writing the literature review for empirical papers

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

The only proofreading tool specialized in correcting academic writing - try for free!

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved April 9, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

• Advanced search
• Peer review
• Record : found
• Abstract : found
• Article : found

Writing the literature review for empirical papers

Read this article at.

• Review article
• Invite someone to review

Abstract Paper aims The purpose of the paper is to offer guidance regarding how to write a Literature Review for empirical papers, that provides adequate background and convincing support. The literature review plays the fundamental role of unveiling the theory, or theories, that underpin the paper argument, sets its limits, and defines and clarifies the main concepts that will be used in the empirical sections of the text. Originality Most papers and books focus on literature review as full articles (systematic reviews, meta analyses and critical analyses) or dissertation, chapters, this paper is focused on literature review for an empirical article. Research method It is a theoretical essay. Main findings The paper summarizes the main steps for performing a literature review and guides how to organize the analyzed literature. Implications for theory and practice Well-crafted literature reviews are the cornerstone of good papers, and this paper offers some guidance on how to write good reviews for empirical papers, and, as a consequence, to produce better quality texts.

Author and article information

Affiliations.

This work is licensed under a Creative Commons Attribution 4.0 International License.

SciELO Brazil

Comment on this article

Ask a Librarian

• Clarify Your Topic
• Research Your Topic
• Write Your Review
• Citing Your Sources
• Other Guides and Resources
• University of Washington Libraries
• Library Guides
• Literature Reviews

Literature Reviews: Write Your Review

Critical analysis.

A critical analysis of your sources is key to creating a quality literature review, and keeping your research question in mind as you read the literature will ensure that you are on track.

• As you read, ask yourself "Why is my topic important?" You must evaluate and interpret the information to discover your own point of view.

Reading and Evaluating Scholarly Literature (Oregon State University)

Evaluating Resources (University of Southern California)

Critically Analyzing Information Sources (Cornell University Library)

Literature Review Model

• What Constitutes a Good Literature Review and Why Does its Quality Matter? A discussion of the state of literature reviews found in scholarly journals. The author discusses the need for clear identification of the "problem domain" or scope of a topic, and the critical need for "identifying and articulating knowledge gaps" in literature reviews. more... less... Maier, H. R. (May 01, 2013). What constitutes a good literature review and why does its quality matter?. Environmental Modelling & Software, 43, 3-4.

Once you identify your topic, check for existing literature reviews in your area of interest that can be used as models.

• Search UW Libraries Catalog using your search terms in conjunction with "literature review" or "methods" or "research" or "bibliography".

What Does a Literature Review Look Like?

This sample literature review from the Purdue University Online Writing Lab (OWL), provides an example in the field of psychology.

Structure of a Literature Review

A literature review has a format similar to other scholarly papers. It contains an introduction, body and conclusion, but is focused exclusively on the research of others.

The Basics of a Literature Review. (2014). Teaching and Learning Center University of Washington Tacoma. Retrieved from  https://www.tacoma.uw.edu/sites/default/files/2021-05/basics-of-lit-review1.pdf 

Guidelines for Writing Your Literature Review

The creation of a literature review involves reading articles , processing the information from the articles, and integrating that information in the larger context of the review

The Basics of a Literature Review. (2014). Teaching and Learning Center University of Washington Tacoma. Retrieved from  https://www.tacoma.uw.edu/sites/default/files/2021-05/basics-of-lit-review1.pdf

Types of Source Materials

Primary Resources - These resources are the basic building blocks for the other types of resources. They include empirical research , firsthand accounts of events and other original materials .

Secondary Resources - These are resources that analyze or interpret primary and other secondary resources .

Tertiary Resources - These include encyclopedias, textbooks, dictionaries, handbooks, and indexes. They provide a summary and definitions of topics and are an effective and efficient way to begin to build your project.

• << Previous: Research Your Topic
• Next: Citing Your Sources >>
• Last Updated: Mar 2, 2023 3:02 PM
• URL: https://guides.lib.uw.edu/tacoma/literaturereview

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

To read this content please select one of the options below:

Please note you do not have access to teaching notes, parameters and decision elements of writing effective literature review papers: empirical evidence from multiple stakeholders on power framework.

Advancing Methodologies of Conducting Literature Review in Management Domain

ISBN : 978-1-80262-372-7 , eISBN : 978-1-80262-371-0

Publication date: 24 November 2023

The study responds to the common concerns of authors, reviewers, and editors on writing and publishing high-quality literature review (LR) studies. First, we evolved the background and decision elements on the five parameters of a quality LR paper: P lanning, O perationalizing, W riting, E mbedding, and R eflecting (POWER), from the editorials and guiding literature. Statistical procedure and refinement of 256 responses from writers, reviewers, and editors revealed 37 decision elements. Finally, a multicriteria-decision-making approach was applied to the detailed responses from the lead editors of ABDC, Scopus, ABS, and WoS journals, and 31 decision elements were found strong enough to represent these five parameters on the quality of LR studies. All five parameters are found important to be considered. However, a high priority is suggested for embedding (the results coming out of the review) and operationalizing (the process of conducting the review), whereas reflection, writing, and planning of LR papers still remain important. The purpose of the POWER framework is to overcome the challenges and decision dilemmas faced by writers and decision-makers. The POWER framework acts as a guiding tool to conduct LR studies in general and business management scholars in specific ways. In addition, this study provides a checklist (Table 6) and template (Appendix A1) of a quality LR study to its stakeholders.

• Literature review
• Review design
• Review style
• Review method
• Writing review

Rana, S. , Singh, J. and Kathuria, S. (2023), "Parameters and Decision Elements of Writing Effective Literature Review Papers: Empirical Evidence From Multiple Stakeholders on POWER Framework", Rana, S. , Singh, J. and Kathuria, S. (Ed.) Advancing Methodologies of Conducting Literature Review in Management Domain ( Review of Management Literature, Vol. 2 ), Emerald Publishing Limited, Leeds, pp. 1-25. https://doi.org/10.1108/S2754-586520230000002001

Emerald Publishing Limited

Copyright © 2024 Sudhir Rana, Jagroop Singh and Sakshi Kathuria. Published under exclusive licence by Emerald Publishing Limited

We’re listening — tell us what you think

Something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

• Submission of manuscripts
• About the journal
• Editorial Board
• Instructions to authors
• Text (English)
• Download PDF (English)
• StambleUpon

Writing the literature review for empirical papers

The purpose of the paper is to offer guidance regarding how to write a Literature Review for empirical papers, that provides adequate background and convincing support. The literature review plays the fundamental role of unveiling the theory, or theories, that underpin the paper argument, sets its limits, and defines and clarifies the main concepts that will be used in the empirical sections of the text.

Originality

Most papers and books focus on literature review as full articles (systematic reviews, meta analyses and critical analyses) or dissertation, chapters, this paper is focused on literature review for an empirical article.

Research method

It is a theoretical essay.

Main findings

The paper summarizes the main steps for performing a literature review and guides how to organize the analyzed literature.

Implications for theory and practice

Well-crafted literature reviews are the cornerstone of good papers, and this paper offers some guidance on how to write good reviews for empirical papers, and, as a consequence, to produce better quality texts.

Keywords Literature review; Metanalyses; Critical analyses; Empirical paper

1. Introduction

As former editors of Production Journal, we have observed that one of the main reasons for the immediate rejection of a paper is what is generally referred as “lack of conceptual (or theoretical) contribution”, i.e., when Editors/Referees are unable to identify which are the proposed additions to the theory authors wish to present in their texts. During 2015-2017 period, the Production Journal rejected 65% of the submissions during the first screening process, mainly for missing theoretical contribution.

The quest for contribution rests on the principle that creating science is a collective and cumulative endeavour, in which each researcher builds upon previously developed knowledge by others, and presents her contribution to the field. The issue of how to make a contribution has already been addressed by several authors from the Organizational Theory field ( Sutton & Staw, 1995 Sutton, R. I., & Staw, B. M. (1995). The relationship between integrating sphere and diffusion theory calculations of fluence rate at the wall of a spherical cavity. Administrative Science Quarterly, 40(3), 1-12. PMid:7708833. http://dx.doi.org/10.2307/2393788. http://dx.doi.org/10.2307/2393788... ; Whetten, 2003 Whetten, D. (2003). O que constitui uma contribuição teórica?. Revista de Administração de Empresas, 43(3), 69-73. ), OM ( Boer et al., 2015 Boer, H., Holweg, M., Kilduff, M., Pagell, M., Schmenner, R., & Voss, C. (2015). Making a meaningful contribution to theory. International Journal of Operations & Production Management, 35(9), 1231-1252. http://dx.doi.org/10.1108/IJOPM-03-2015-0119. http://dx.doi.org/10.1108/IJOPM-03-2015-... ), and it will also be the subject of another paper in this journal, but we believe that weakness is not only related to the paper rationale, where the argument for contribution is developed, but also to how authors construct the theoretical background of their texts. A good literature review supports the paper assertion for contribution, and it is the cornerstone of a successful paper. Even when a paper does make a clear claim for conceptual contribution, it is not uncommon to find that the literature review section does not provide it a consistent foundation, weakening the whole paper, and frequently jeopardizing authors´ efforts. In fact, the absence of a well-crafted theoretical background undermines any paper and wastes effort-intensive fieldwork.

However, despite its importance, it seems that how to build the review of a body of literature is considered a simple, obvious task ( Hart, 1998 Hart, C. (1998). Doing a literature review. London: Sage Publications. ), and thus, there is actually few texts that delve into that effort. To make things harder, indications of how to build strong theory into a paper can be confusing, especially because authors have to deal with many trade-offs: comprehensiveness versus deepness, simplicity versus accuracy ( Sutton & Staw, 1995 Sutton, R. I., & Staw, B. M. (1995). The relationship between integrating sphere and diffusion theory calculations of fluence rate at the wall of a spherical cavity. Administrative Science Quarterly, 40(3), 1-12. PMid:7708833. http://dx.doi.org/10.2307/2393788. http://dx.doi.org/10.2307/2393788... ) and so on. However, there are a number of good references on how to write a literature review for two purposes: (a) as chapters of master theses and doctoral dissertations (e.g. Boote & Beile, 2005 Boote, D. N., & Beile, P. (2005). Scholars before researchers: on the centrality of the dissertation literature review in research preparation. Educational Researcher, 34(6), 3-15. ; Hart, 1998 Hart, C. (1998). Doing a literature review. London: Sage Publications. ), and (b) for literature reviews as full articles (e.g. Kitchenham et al., 2009 Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009). Systematic literature reviews in software engineering: a systematic literature review. Information and Software Technology, 51(1), 7-15. http://dx.doi.org/10.1016/j.infsof.2008.09.009. http://dx.doi.org/10.1016/j.infsof.2008.... ; Tranfield et al., 2003 Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14(3), 207-222. http://dx.doi.org/10.1111/1467-8551.00375. http://dx.doi.org/10.1111/1467-8551.0037... ; Thomé et al., 2016 Thomé, A. M. T., Scavarda, L. F., & Scavarda, A. J. (2016). Conducting systematic literature review in operations management. Production Planning and Control, 27(5), 408-420. http://dx.doi.org/10.1080/09537287.2015.1129464. http://dx.doi.org/10.1080/09537287.2015.... ). This paper draws upon those contributions to suggest some guidelines for authors to write literature review sections for their empirical papers, and provide a convincing rationale to supports this effort, especially from the Operations Management (OM) point of view. It is structured as follows: it briefly presents the different strategies for writing literature reviews as full papers, and then discuss how to write the review section for an empirical paper, based particularly on the literature for theses and dissertations.

2. Literature reviews as full articles

Literature reviews are often presented as full papers, and there are journals which scope is focused on this genre, as the International Journal of Management Review, the Academy Management of Review and the Psychological Review. Reviews offer a comprehensive and thorough analysis of the academic production on a certain topic, and are very useful for both novice and experienced researchers. There are some strategies to build literature review articles: the meta-analysis (MA), the critical analysis (CA), and the integrative or systematic literature review (SLR).

MAs propose one or more research questions and to answer them, analyse large sets of articles on a subject, which authors, titles, keywords, abstracts and references are collected from academic portals and organized in a database. MAs perform statistical analyses on collected data, often using specific software (e.g. Citespace, VOSviewer, Sitkis, etc.), and present an aggregate description of how the academic output has evolved, which are the more prominent papers and authors, how they are connected by the citations they share, and how they answer the research questions. In that sense, MAs synthesize and present a comprehensive view of the literature, and, if the paper set is carefully assembled, offer an overall reliability that cannot be reached from other methods ( Tranfield et al., 2003 Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14(3), 207-222. http://dx.doi.org/10.1111/1467-8551.00375. http://dx.doi.org/10.1111/1467-8551.0037... ). MAs can be used as the first step for a SLR or a CA, or, for instance, to present a bird´s eye picture of the literature in a dissertation. Many academic portals (Scopus, Web of Science, EBSCO, Science Direct, SciElo) and/or softwares support some level of MA, providing descriptive statistics on sets of papers, which can support further analysis. However, if there are no research questions to be discussed, descriptive statistics hardly ever present theoretical contributions per se, and MAs should offer, at minimum, a critique on the field and present opportunities for future research. For further study, Fink (1998 Fink, A. (1998). Conducting research literature review: from paper to the internet. London: Sage. , p. 216) offers a guide on how to write MAs (Seven Steps to a Meta-Analysis).

In a CA, authors examine the main concepts, ideas and relationships of an issue presented by the extant literature, provide a critique, and in several cases, offer research propositions or a framework for future analysis. A CA can summarize a body of literature, report on the evolution of a research field, or reconcile different research strands on the same topic. Generally, paper set selection is subjective, which departs it from MAs: the set of papers is collected by the writers, who decide both what to include and how those ideas will be discussed and summarized. In that sense, it does not intend to be exhaustive, authors are concerned to give the most accurate picture of the field from their point of view. But even in CAs, there are also paper sets defined by statistical sampling (e.g. Burgess et al., 2006 Burgess, K., Singh, P. J., & Koroglu, R. (2006). Supply chain management: a structured literature review and implications for future research. International Journal of Operations & Production Management, 26(7), 703-729. http://dx.doi.org/10.1108/01443570610672202. http://dx.doi.org/10.1108/01443570610672... ). CAs can cover broad or narrow topics: for example, Anderson et al. (1989) Anderson, J. C., Cleveland, G., & Schroeder, R. G. (1989). Operations strategy: a literature review. Journal of Operations Management, 8(2), 133-158. http://dx.doi.org/10.1016/0272-6963(89)90016-8. http://dx.doi.org/10.1016/0272-6963(89)9... provided an overview and an analysis on the literature on Operations strategy, while Ngai et al. (2008) Ngai, E. W. T., Moon, K. K. L., Riggins, F. J., & Yi, C. Y. (2008). RFID research: an academic literature review (1995-2005) and future research directions. International Journal of Production Economics, 12(2), 510-520. https://doi.org/10.1016/j.ijpe.2007.05.004. https://doi.org/10.1016/j.ijpe.2007.05.0... delivered an analysis on the literature on RFID. They can discuss topics from few or several perspectives or lines of research: Rochet & Tirole (2006) Rochet, J., & Tirole, J. (2006). Two-sided markets: a progress report. The RAND Journal of Economics, 37(3), 645-667. offered a report on the evolution of the knowledge on multi-sided markets and platforms, while Mills et al. (2004) Mills, J., Schmitz, J., & Frizelle, G. (2004). A strategic review of “supply networks”. International Journal of Operations & Production Management, 24(10), 1012-1036. reviewed the literature on supply networks. CA offers a view on the state-of-art of knowledge on a subject, provide analytical frameworks and indicate avenues for future research. CAs should follow the same quality principles as any research project, and some guidelines for a good CA are ( Popay et al., 1998 Popay, J., Rogers, A., & Williams, G. (1998). Rationale and standards for the systematic review of qualitative literature in health services research. Qualitative Health Research, 8(3), 341-351. PMID: 10558335. http://dx.doi.org/10.1177/104973239800800305. http://dx.doi.org/10.1177/10497323980080... , 2006 Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., & Duffy, S. (2006). Guidance on the conduct of narrative synthesis in systematic reviews. A product from the ESRC methods programme Version, 1, b92. Retrieved in 2017, July 9, from https://www.researchgate.net/profile/Mark_Rodgers4/publication/233866356_Guidance_on_the_conduct_of_narrative_synthesis_in_systematic_reviews_A_product_from_the_ESRC_Methods_Programme/links/02e7e5231e8f3a6183000000/Guidance-on-the-conduct-of-narrative-synthesis-in-systematic-reviews-A-product-from-the-ESRC-Methods-Programme.pdf https://www.researchgate.net/profile/Mar... ):

Different sources and journals should be explored;

The sample has to be selected in a purposeful way, guided and shaped by theory. It must give attention to the diverse contexts and meanings that the study is aiming to explore;

Interpretation needs to follow a clear and explicit process;

Claims and assertions must be logically supported, theoretically grounded and be amenable of generalization, i.e., they should be applicable in different contexts.

The SLR is a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way, generating new frameworks and perspectives ( Torraco, 2005 Torraco, R. J. (2005). Writing integrative literature reviews: guidelines and examples. Human Resource Development Review, 4(3), 356-367. http://dx.doi.org/10.1177/1534484305278283. http://dx.doi.org/10.1177/15344843052782... ), using a much larger paper set than a CA. It also departs from research questions and requires a great deal of skill and insight ( Torraco, 2005 Torraco, R. J. (2005). Writing integrative literature reviews: guidelines and examples. Human Resource Development Review, 4(3), 356-367. http://dx.doi.org/10.1177/1534484305278283. http://dx.doi.org/10.1177/15344843052782... ), it is not just an aggregation of all existing evidence on a research question; it intends to provide evidence-based guidelines for researchers and practitioners. To Kitchenham et al. (2009) Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009). Systematic literature reviews in software engineering: a systematic literature review. Information and Software Technology, 51(1), 7-15. http://dx.doi.org/10.1016/j.infsof.2008.09.009. http://dx.doi.org/10.1016/j.infsof.2008.... SLR is a literature survey with defined research questions, a clear search and data extraction process, and its presentation (detailed in Appendix Appendix Summarizing the key points of a paper. Authors Objective Findings Research Question Further Research Limitations Methodology Practical Implications Abbariki et al. (2017) The paper analyses whether tacit knowledge was being shared through collective learning routines at two sites where employees were undertaking knowledge-intensive work. Superiors can overlook the presence of embedded knowledge work in tasks environments that are largely static, procedure driven and independent.In such environments, employees need to be grated opportunities to meet together to share tacit knowledge that cannot be accommodated by archiven-based knowledge management platforms. How can employees share tacit knowledge if they are performing tasks independently, and if their superiors discourage face-to-face interactions when they are performing tasks in real time? Enhance confirm ability by examining sites that are similar to Cases, with employees operating within a largely static, solo work and procedure-driven task environment Although the authors sought to obtain a close-up picture of how tacit knowledge was being shared at Case A, the very nature of such knowledge rendered it difficult for employees to provide explanations to an outside interviewer 34 semi structured interviews about task environment and collective learning activities of informants at various hierarchical levelsCase A: 3 directors/9 senior managers/16 operational managersCase B: 1 director, 2 senior managers and 2 operational managers a) It is important for managers to recognize that even procedure-governed tasks require tacit knowledgeb) Managers should encourage and facilitate the sharing of tacit knowledge among employeesc) Computer-based knowledge sharing platforms cannot substitute for face-to-face knowledge sharing forums, since they not help employees to articulate their own tacit knowledge or to decode the tacit knowledge of othersd) If managers prefer to restrict opportunities for employees to engage in ad hoc meetings while they are performing their tasks, they may schedule periodic “off-line” face to face forums Torraco (2005) This article discusses how to organize and write an integrative literature review and cites examples of published integrative literature reviews that illustrate how this type of research has made substantive contributions to the knowledge base of human resource development. It Categorizes Four Forms of Synthesis from Integrative Literature Reviews: a) Research agenda b) Taxonomy c) Alternative models or conceptual frameworks d) Meta-theory.A Checklist for Writing an Integrative Literature Review: a) Before Writing an Integrative Literature Review b) Organizing an Integrative Literature Review c) Writing an Integrative Literature Review Not Available Not Available The paper is focused in Human Resources and could be expanded for others areas The paper discusses 31 papers to provide its findings. This article offers guidelines for writing integrative literature reviews to support authors to do this task. Kitchenham et al. (2009) This study assesses the impact of systematic literature reviews (SLRs), which are the recommended evidence-based software engineering method for aggregating evidence. The authors define evidence as a synthesis of best quality scientific studies on a specific topic or research question, which the main method of synthesis is a systematic literature review (SLR) Not available The authors indicate to extend this study by under- taking a broader automated search for other SLRs over the same time period. They also plan to repeat this study at the end of 2009 to track the progress of SLRs and evidence-based software engineering. This study suffers from a number of limitations; in particular, the authors have restricted themselves to a manual search of international journals and conferences. This study has been undertaken as a systematic literature review based on the original guidelines as proposed by Kitchenham et al. (2009). In this case the goal of the review is to assess systematic literature reviews (which are referred to as secondary studies), so this study is categorized as a tertiary literature review. The authors exampled issues to guide how to do Research Questions (RQ), Limitations and Quality Assessment in their paper, which helps other authors: RQ1. How much SLR activity has there been since 2004? RQ2. What research topics are being addressed? RQ3. Who is leading SLR research? RQ4. What are the limitations of current research?With respect to limitations of SLRs (RQ4) the authors considered issues: RQ4.1. Were the research topics limited? RQ4.2. Is there evidence that the use of SLRs is limited due to lack of primary studies? RQ4.3. Is the quality of SLRs appropriate, if not, is it improving? RQ4.4. Are SLRs contributing to practice by defining practice guidelines?The Quality Assessment criteria are based on four quality assessment (QA) questions:QA1. Are the review’s inclusion and exclusion criteria described and appropriate?QA2. Is the literature search likely to have covered all relevant studies?QA3. Did the reviewers assess the quality/validity of the included studies?QA4. Were the basic data/studies adequately described? ). Tranfield et al. (2003) Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14(3), 207-222. http://dx.doi.org/10.1111/1467-8551.00375. http://dx.doi.org/10.1111/1467-8551.0037... cite it as a replicable, scientific and transparent process that aims to minimize bias through an exhaustive literature search on published and unpublished studies, providing an audit trail of the reviewers decisions, procedures and conclusions. Thus, SLRs are related to locating, appraising and synthesizing evidence ( Petticrew & Roberts, 2006 Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: a practical guide. Malden MA: Blackwell. ). SLRs entail a series of techniques for minimizing bias and error, and thus, SLRs and MAs are widely regarded as providing ‘high-quality’ evidence. The pros and cons of a SLR are indicated in Table 1 .

A SLR starts arguing the need for the review article, the importance of the problem or topic to be examined, and by justifying why a SLR is the appropriate way to address the problem. Then it selects and examines a body of literature, exposing its strengths and deficiencies, to create a better understanding of the topic through synthesis, by integrating existing and new ideas to create a new formulation for the topic or issue ( Torraco, 2005 Torraco, R. J. (2005). Writing integrative literature reviews: guidelines and examples. Human Resource Development Review, 4(3), 356-367. http://dx.doi.org/10.1177/1534484305278283. http://dx.doi.org/10.1177/15344843052782... , p. 362). Unlike a CA, in which authors review the literature from a personal perspective, a SLR makes the reviewing process as structured, transparent, replicable and exhaustive as possible ( Torraco, 2005 Torraco, R. J. (2005). Writing integrative literature reviews: guidelines and examples. Human Resource Development Review, 4(3), 356-367. http://dx.doi.org/10.1177/1534484305278283. http://dx.doi.org/10.1177/15344843052782... ). In order to achieve these aims, a structured process must be followed to design and implement the paper database, from which it is possible to identify patterns and themes of various publications, as well as their frequency and occurrence. Data can provide evidence on changes, shifts and gaps of the literature. Three major phases are required to build a SLR: (i) planning, (ii) execution, and (iii) summarization/reporting. In the first phase, authors must identify the need for a review and create a review protocol 1 1 Protocol is a document that presents an explicit scientific “road map”, detailing the rational and planned methodological and analytical approach of the review (Shamseer et al., 2015, p. 3). . In the second, they should identify and select relevant primary studies, perform data extraction from paper databases and synthesize them. Finally, in the third phase, they should summarize and report the results. There are online manuals for performing MAs and SLRs ( Higgins & Green, 2011 Higgins, J. P. T., & Green, S. Cochrane handbook for systematic reviews of interventions, version 5.1.0. 2011. Retrieved in 2017, December 4, from http://handbook-5-1.cochrane.org/ http://handbook-5-1.cochrane.org/... ) and some guidelines to perform those phases are presented in ( Kitchenham et al., 2009 Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009). Systematic literature reviews in software engineering: a systematic literature review. Information and Software Technology, 51(1), 7-15. http://dx.doi.org/10.1016/j.infsof.2008.09.009. http://dx.doi.org/10.1016/j.infsof.2008.... ; Torraco, 2005 Torraco, R. J. (2005). Writing integrative literature reviews: guidelines and examples. Human Resource Development Review, 4(3), 356-367. http://dx.doi.org/10.1177/1534484305278283. http://dx.doi.org/10.1177/15344843052782... ).

Table 2 presents an overview of the three strategies and offers an analysis based on a representative example.

3. The literature review in an empirical paper

In this section we discuss the literature review as a part of an empirical article. It plays the fundamental role of unveiling the theory, or theories, that underpin the paper argument, or, if there are no such theoretical background, which is the related extant knowledge. It sets the limits of discussion, and defines and clarifies the main concepts that will be used in the empirical sections. A substantive and thorough literature review is the basis for any good research project ( Boote & Beile, 2005 Boote, D. N., & Beile, P. (2005). Scholars before researchers: on the centrality of the dissertation literature review in research preparation. Educational Researcher, 34(6), 3-15. ) and a well-crafted literature review section provides the theoretical foundation that is required to support any argument of contribution.

Theories are systems of concepts that “[...] explain facts and provide stories as to how phenomena work the way that they do [...]” ( Boer et al., 2015 Boer, H., Holweg, M., Kilduff, M., Pagell, M., Schmenner, R., & Voss, C. (2015). Making a meaningful contribution to theory. International Journal of Operations & Production Management, 35(9), 1231-1252. http://dx.doi.org/10.1108/IJOPM-03-2015-0119. http://dx.doi.org/10.1108/IJOPM-03-2015-... , p. 1247), and the first task of a literature review is to reveal which theories are used in the paper´s argument. Boer et al. (2015) Boer, H., Holweg, M., Kilduff, M., Pagell, M., Schmenner, R., & Voss, C. (2015). Making a meaningful contribution to theory. International Journal of Operations & Production Management, 35(9), 1231-1252. http://dx.doi.org/10.1108/IJOPM-03-2015-0119. http://dx.doi.org/10.1108/IJOPM-03-2015-... propose that there are two fundamental ways to make a theoretical contribution: exploratory studies observe and identify interesting phenomena that are not yet well explained by extant literature and propose hypotheses, while confirmatory studies empirically test those hypotheses and confirm, refute or expand them. Knowledge creation rests upon a cycle of testing and amending or refuting existing theory, and the first step to succeed in that task, is to provide a clear picture of the underlying or related theories that support the paper argument. The literature review section opens up the field, showing how the issue under study has been discussed and which are its main concepts, how they have been studied from different points of view and how the field has evolved over time, in order to synthesize them and identify theoretical lacunae. Thus, crafting the literature review section of a paper must be concerned with three goals: setting its theoretical background, identifying gaps in the literature, and defining the key concepts that will be used in the paper:

Establishing the theoretical background : some research projects are related to a single theory, while others work with multiple theories - to set up the background is to position the paper within the theory or theories that support and are used in its argument. It is useful to balance classic texts, which have established the discussion, with contemporary references, that show the current state of the field. Although the number of references should not be the main concern, the analysis must be comprehensive and include what is most relevant, which is not usually obtained by discussing only a few authors. For instance Abbariki et al. (2017) Abbariki, M., Snell, R. S., & Easterby-Smith, M. (2017). Sharing or ignoring tacit knowledge? A comparison of collective learning routines at two sites. Journal of General Management, 42(5), 57-67. http://dx.doi.org/10.1177/0306307017702997. http://dx.doi.org/10.1177/03063070177029... used 84 references to built the theoretical background, and synthesized it in four sections: a relational perspective on workplace knowledge, Processing tacit knowledge, Contextual factors affecting employee collaboration in processing tacit knowledge and Collective learning routines;

Identifying gaps : a theoretical gap refers to a missing point in our current knowledge on a subject, or to an improperly conducted discussion, and the literature review should point to it. Theoretical gaps can be identified based on three rationales: (a) incompleteness – the current literature have not still properly discussed the problem or phenomenon; (b) inadequacy – the extant literature has not yet incorporated different perspectives on the problem or phenomenon, and (c) incommensurability – what is currently known on a subject has taken a wrong path and the existing theoretical discussion is misguided and incorrect ( Locke & Golden-Biddle, 1997 Locke, K., & Golden-Biddle, K. (1997). Constructing opportunities for contribution. Academy of Management Journal, 40(5), 1023-1062. http://dx.doi.org/10.2307/256926. http://dx.doi.org/10.2307/256926... ). Contemporary, recent references are very important to the contribution claim, as they provide topicality;

Defining key concepts : all key concepts used in the paper arguments must be clearly defined, as well as how they relate to each other, if that is the case, based on previous work. However, a literature review should not limit itself to a list of concept and construct definitions ( Sutton & Staw, 1995 Sutton, R. I., & Staw, B. M. (1995). The relationship between integrating sphere and diffusion theory calculations of fluence rate at the wall of a spherical cavity. Administrative Science Quarterly, 40(3), 1-12. PMid:7708833. http://dx.doi.org/10.2307/2393788. http://dx.doi.org/10.2307/2393788... ), it should present how they interconnected and how they will be used to support the argument. (e.g. Abbariki et al. (2017) Abbariki, M., Snell, R. S., & Easterby-Smith, M. (2017). Sharing or ignoring tacit knowledge? A comparison of collective learning routines at two sites. Journal of General Management, 42(5), 57-67. http://dx.doi.org/10.1177/0306307017702997. http://dx.doi.org/10.1177/03063070177029... defined key concepts on the different perspectives of knowledge to support their study).

4. Which are the main characteristics of a good literature review?

A good literature review must address different requirements, covering the relevant literature and synthesizing it with clarity. A journal paper has usually length limitations, thus, the literature review section needs to limit itself to what is important to the argument. Dissertation guides and handbooks (e.g. Boote & Beile, 2005 Boote, D. N., & Beile, P. (2005). Scholars before researchers: on the centrality of the dissertation literature review in research preparation. Educational Researcher, 34(6), 3-15. ) indicate the key features for a literature review, which are also applicable for papers:

Coverage – Relevant references must be covered by the text. It does not mean just citing lots of authors, but identifying and presenting the relevant literature, the main research strands, and building a framework where the paper can be positioned. It is like drawing a map in which key places are identified, and indicating where in that map the paper argument is located. The issue of length must be carefully addressed, and as a general rule, it is always better to have a deeper discussion on the topics the paper delves in, than a superficial overview of several arguments;

Synthesis – A good literature review is not a just a long list of citations, it should summarize and connect relevant references. Synthesis is not just putting references and concepts together. It requires creativity to offer a fresh, new view on the topic, for instance a model or a framework, which reflects the unique knowledge developed by the author ( Torraco, 2005 Torraco, R. J. (2005). Writing integrative literature reviews: guidelines and examples. Human Resource Development Review, 4(3), 356-367. http://dx.doi.org/10.1177/1534484305278283. http://dx.doi.org/10.1177/15344843052782... ). The synthesis can be presented in the form of questions or propositions that have to be either verified or answered by fieldwork. It can also propose a model or a framework, which will be tested or applied in the empirical sections of the paper;

Rhetoric – The text must be clear and coherent, ideas must be presented in a well-articulated text, which does not make unsupported assertions;

Significance – The review also must show which is the practical and theoretical significance of the research problem. We strongly recommend that research problem should evidence theoretical aspects and also register organizational contributions as observed in Abbariki et al. (2017) Abbariki, M., Snell, R. S., & Easterby-Smith, M. (2017). Sharing or ignoring tacit knowledge? A comparison of collective learning routines at two sites. Journal of General Management, 42(5), 57-67. http://dx.doi.org/10.1177/0306307017702997. http://dx.doi.org/10.1177/03063070177029... , which indicated the core of significance in paper’ introduction.

A good literature review balances “[...] appropriate breadth and depth, rigor and consistency, clarity and brevity [...]” ( Hart, 1998 Hart, C. (1998). Doing a literature review. London: Sage Publications. , p. 2), making an effective analysis and synthesis of the identified literature. Consistency and clarity are essential, as they support a coherent argument, while depth and rigor show how authors master the subject, providing a well-developed argument. Finally, brevity is also essential, as all relevant literature must be presented, analysed and articulated in a limited space. When one finishes reading the literature review, one should be able to answer the following questions:

What are the main sources on the subject under study?

Which are the key theories and ideas that support the paper´s assertions?

How the paper argument relates to a major issue or debate on the topic?

What are the key concepts of the paper´s argument and how are they are defined?

5. How to write a literature review for empirical papers?

Novice researchers may find the task of writing the literature review a quite overwhelming task after reading this paper to this point. To offer some practical steps to start it, we suggest that the researcher starts with a preliminary CA. To perform it, we suggest starting with a small set of 8 to 10 papers. To select papers, talking to supervisors and seasoned researchers is always the first action to take. Also, papers can be obtained from academic portals, such as Web of Knowledge, Scopus, Science Direct, EBSCO, JSTOR, SciELO, which strengths and weaknesses are indicated in Table 3 . As each portal has its selection of sources, it may be convenient to search in more than one to get better coverage. Also, portals usually rank papers by number of citation/relevance, which is useful to identify fundamental papers and authors. However, it is important to remember that the older the paper, the greater the probability of citation, so if only number of citations is considered, one may end up with an older, and sometimes out-dated, set of papers. Finally, that paper set is just a preliminary selection, which needs to be refined as the researcher keeps studying the subject, and she will find that citations from papers are always a good source for further reading.

After assembling the initial set of papers, the aims, research questions, methods, findings, limitations and suggestions for future research can be extracted from each paper. Empirical papers usually are structured in at least five sections: (1) introduction, (2) literature review, (3) empirical methods, (4) data analysis, discussion and findings, and (5) conclusions. The introduction section usually presents the paper´s aim, the theoretical gaps it addresses and its research questions, and the conclusion section discuss its limitations and opportunities for future research. Thus, reading first those sections usually helps to better grasp the whole paper.

After reading each paper, we suggest to summarize data in a table, as recorded in the Appendix Appendix Summarizing the key points of a paper. Authors Objective Findings Research Question Further Research Limitations Methodology Practical Implications Abbariki et al. (2017) The paper analyses whether tacit knowledge was being shared through collective learning routines at two sites where employees were undertaking knowledge-intensive work. Superiors can overlook the presence of embedded knowledge work in tasks environments that are largely static, procedure driven and independent.In such environments, employees need to be grated opportunities to meet together to share tacit knowledge that cannot be accommodated by archiven-based knowledge management platforms. How can employees share tacit knowledge if they are performing tasks independently, and if their superiors discourage face-to-face interactions when they are performing tasks in real time? Enhance confirm ability by examining sites that are similar to Cases, with employees operating within a largely static, solo work and procedure-driven task environment Although the authors sought to obtain a close-up picture of how tacit knowledge was being shared at Case A, the very nature of such knowledge rendered it difficult for employees to provide explanations to an outside interviewer 34 semi structured interviews about task environment and collective learning activities of informants at various hierarchical levelsCase A: 3 directors/9 senior managers/16 operational managersCase B: 1 director, 2 senior managers and 2 operational managers a) It is important for managers to recognize that even procedure-governed tasks require tacit knowledgeb) Managers should encourage and facilitate the sharing of tacit knowledge among employeesc) Computer-based knowledge sharing platforms cannot substitute for face-to-face knowledge sharing forums, since they not help employees to articulate their own tacit knowledge or to decode the tacit knowledge of othersd) If managers prefer to restrict opportunities for employees to engage in ad hoc meetings while they are performing their tasks, they may schedule periodic “off-line” face to face forums Torraco (2005) This article discusses how to organize and write an integrative literature review and cites examples of published integrative literature reviews that illustrate how this type of research has made substantive contributions to the knowledge base of human resource development. It Categorizes Four Forms of Synthesis from Integrative Literature Reviews: a) Research agenda b) Taxonomy c) Alternative models or conceptual frameworks d) Meta-theory.A Checklist for Writing an Integrative Literature Review: a) Before Writing an Integrative Literature Review b) Organizing an Integrative Literature Review c) Writing an Integrative Literature Review Not Available Not Available The paper is focused in Human Resources and could be expanded for others areas The paper discusses 31 papers to provide its findings. This article offers guidelines for writing integrative literature reviews to support authors to do this task. Kitchenham et al. (2009) This study assesses the impact of systematic literature reviews (SLRs), which are the recommended evidence-based software engineering method for aggregating evidence. The authors define evidence as a synthesis of best quality scientific studies on a specific topic or research question, which the main method of synthesis is a systematic literature review (SLR) Not available The authors indicate to extend this study by under- taking a broader automated search for other SLRs over the same time period. They also plan to repeat this study at the end of 2009 to track the progress of SLRs and evidence-based software engineering. This study suffers from a number of limitations; in particular, the authors have restricted themselves to a manual search of international journals and conferences. This study has been undertaken as a systematic literature review based on the original guidelines as proposed by Kitchenham et al. (2009). In this case the goal of the review is to assess systematic literature reviews (which are referred to as secondary studies), so this study is categorized as a tertiary literature review. The authors exampled issues to guide how to do Research Questions (RQ), Limitations and Quality Assessment in their paper, which helps other authors: RQ1. How much SLR activity has there been since 2004? RQ2. What research topics are being addressed? RQ3. Who is leading SLR research? RQ4. What are the limitations of current research?With respect to limitations of SLRs (RQ4) the authors considered issues: RQ4.1. Were the research topics limited? RQ4.2. Is there evidence that the use of SLRs is limited due to lack of primary studies? RQ4.3. Is the quality of SLRs appropriate, if not, is it improving? RQ4.4. Are SLRs contributing to practice by defining practice guidelines?The Quality Assessment criteria are based on four quality assessment (QA) questions:QA1. Are the review’s inclusion and exclusion criteria described and appropriate?QA2. Is the literature search likely to have covered all relevant studies?QA3. Did the reviewers assess the quality/validity of the included studies?QA4. Were the basic data/studies adequately described? (Authors, Objective, Findings, Research-questions, Further Research, Limitations, Methodology, Practical Implications). For further study, Perkmann et al. (2013 Perkmann, M., Tartari, V., McKelvey, M., Autioa, E., Broströmc, A., D’Este, P., Finif, R., Geuna, A., Grimaldif, R., Hughes, A., Krabel, S., Kitsong, M., Llerenai, P., Lissoni, F., Salter, A., & Sobrerof, M. (2013). Academic engagement and commercialisation: a review of the literature on university–industry relations. Research Policy, 42(2), 423-442. http://dx.doi.org/10.1016/j.respol.2012.09.007. http://dx.doi.org/10.1016/j.respol.2012.... , p. 434-439) applied similar strategy to summarize 133 articles, recording them in a table with the following columns: Research questions, Data Method, Dependent variables, and Findings.

From this initial step, the author can assess if there is good coverage, which papers support the claims that will be made, and which present concept definitions. At this point, the author will realize which are subjects need to be further studied, and will start a new round of paper search and reading. It is a cyclical process, and ideally, in each cycle a clearer and deeper perspective is reached.

As academic output is huge and keeps growing literally everyday, there is the danger of losing focus and the researcher must be cautious about when to finish the literature search, as sometimes one is tempted to keep looking for more texts. Here again, guidance from supervisors and help from seasoned researchers is always a good way to check is the required coverage and deepness has been reached. Finally, the researcher must sum up her work on a text, which must offer a good synthesis and good rhetoric. Novice researchers will find that the first draft is never a good text, and a well-crafted piece of writing is the outcome of several revisions. Figure 1 summarizes the main steps for performing a literature review:

6. Conclusions

Well-crafted literature reviews are the cornerstone of good papers, however it is not uncommon to find weak, or even absent, literature reviews among OM submissions to Production. That weakness jeopardizes any claim for contribution authors might have, and frequently undermines all effort put into fieldwork and data analysis. However, as an applied field, research in OM should not start exclusively from theory, practice has been and should continue to be a major source for research ( Boer et al., 2015 Boer, H., Holweg, M., Kilduff, M., Pagell, M., Schmenner, R., & Voss, C. (2015). Making a meaningful contribution to theory. International Journal of Operations & Production Management, 35(9), 1231-1252. http://dx.doi.org/10.1108/IJOPM-03-2015-0119. http://dx.doi.org/10.1108/IJOPM-03-2015-... ). Interesting, real world phenomena should motivate us to study and solve them, but our efforts should be aimed towards not only its solution, but also to how it can be generalized. There is no contribution case that stands the lack of a proper literature review and we hope to have offered some guidance on how to write good reviews for empirical papers, and, as a consequence, to produce better quality texts.

Appendix Summarizing the key points of a paper.

• 1 Protocol is a document that presents an explicit scientific “road map”, detailing the rational and planned methodological and analytical approach of the review ( Shamseer et al., 2015 Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ (Clinical Research Ed.), 349, g7647. PMid:25555855. , p. 3).
• How to cite this article: Nakano, D., & Muniz Jr., J. (2018). Writing the literature review for an empirical paper. Production, 28 , e20170086. http://dx.doi.org/10.1590/0103-6513.20170086
• Abbariki, M., Snell, R. S., & Easterby-Smith, M. (2017). Sharing or ignoring tacit knowledge? A comparison of collective learning routines at two sites. Journal of General Management , 42 (5), 57-67. http://dx.doi.org/10.1177/0306307017702997 » http://dx.doi.org/10.1177/0306307017702997
• Anderson, J. C., Cleveland, G., & Schroeder, R. G. (1989). Operations strategy: a literature review. Journal of Operations Management , 8 (2), 133-158. http://dx.doi.org/10.1016/0272-6963(89)90016-8 » http://dx.doi.org/10.1016/0272-6963(89)90016-8
• Boer, H., Holweg, M., Kilduff, M., Pagell, M., Schmenner, R., & Voss, C. (2015). Making a meaningful contribution to theory. International Journal of Operations & Production Management , 35 (9), 1231-1252. http://dx.doi.org/10.1108/IJOPM-03-2015-0119 » http://dx.doi.org/10.1108/IJOPM-03-2015-0119
• Boote, D. N., & Beile, P. (2005). Scholars before researchers: on the centrality of the dissertation literature review in research preparation. Educational Researcher , 34 (6), 3-15.
• Burgess, K., Singh, P. J., & Koroglu, R. (2006). Supply chain management: a structured literature review and implications for future research. International Journal of Operations & Production Management , 26 (7), 703-729. http://dx.doi.org/10.1108/01443570610672202 » http://dx.doi.org/10.1108/01443570610672202
• Easterby-Smith, M., Thorpe, R., & Jackson, P. R. (2012). Management research London: Sage.
• Fink, A. (1998). Conducting research literature review: from paper to the internet London: Sage.
• Hart, C. (1998). Doing a literature review London: Sage Publications.
• Higgins, J. P. T., & Green, S. Cochrane handbook for systematic reviews of interventions, version 5.1.0 2011. Retrieved in 2017, December 4, from http://handbook-5-1.cochrane.org/ » http://handbook-5-1.cochrane.org/
• Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009). Systematic literature reviews in software engineering: a systematic literature review. Information and Software Technology , 51 (1), 7-15. http://dx.doi.org/10.1016/j.infsof.2008.09.009 » http://dx.doi.org/10.1016/j.infsof.2008.09.009
• Locke, K., & Golden-Biddle, K. (1997). Constructing opportunities for contribution. Academy of Management Journal , 40 (5), 1023-1062. http://dx.doi.org/10.2307/256926 » http://dx.doi.org/10.2307/256926
• Mills, J., Schmitz, J., & Frizelle, G. (2004). A strategic review of “supply networks”. International Journal of Operations & Production Management , 24 (10), 1012-1036.
• Ngai, E. W. T., Moon, K. K. L., Riggins, F. J., & Yi, C. Y. (2008). RFID research: an academic literature review (1995-2005) and future research directions. International Journal of Production Economics , 12 (2), 510-520. https://doi.org/10.1016/j.ijpe.2007.05.004 » https://doi.org/10.1016/j.ijpe.2007.05.004
• Nonaka, I., & Peltokorpi, V. (2006). Objectivity and subjectivity in knowledge management: a review of 20 top articles. Knowledge and Process Management , 13 (2), 73-82. http://dx.doi.org/10.1002/kpm.251 » http://dx.doi.org/10.1002/kpm.251
• Perkmann, M., Tartari, V., McKelvey, M., Autioa, E., Broströmc, A., D’Este, P., Finif, R., Geuna, A., Grimaldif, R., Hughes, A., Krabel, S., Kitsong, M., Llerenai, P., Lissoni, F., Salter, A., & Sobrerof, M. (2013). Academic engagement and commercialisation: a review of the literature on university–industry relations. Research Policy , 42 (2), 423-442. http://dx.doi.org/10.1016/j.respol.2012.09.007 » http://dx.doi.org/10.1016/j.respol.2012.09.007
• Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: a practical guide Malden MA: Blackwell.
• Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., & Duffy, S. (2006). Guidance on the conduct of narrative synthesis in systematic reviews. A product from the ESRC methods programme Version, 1, b92 Retrieved in 2017, July 9, from https://www.researchgate.net/profile/Mark_Rodgers4/publication/233866356_Guidance_on_the_conduct_of_narrative_synthesis_in_systematic_reviews_A_product_from_the_ESRC_Methods_Programme/links/02e7e5231e8f3a6183000000/Guidance-on-the-conduct-of-narrative-synthesis-in-systematic-reviews-A-product-from-the-ESRC-Methods-Programme.pdf » https://www.researchgate.net/profile/Mark_Rodgers4/publication/233866356_Guidance_on_the_conduct_of_narrative_synthesis_in_systematic_reviews_A_product_from_the_ESRC_Methods_Programme/links/02e7e5231e8f3a6183000000/Guidance-on-the-conduct-of-narrative-synthesis-in-systematic-reviews-A-product-from-the-ESRC-Methods-Programme.pdf
• Popay, J., Rogers, A., & Williams, G. (1998). Rationale and standards for the systematic review of qualitative literature in health services research. Qualitative Health Research , 8 (3), 341-351. PMID: 10558335. http://dx.doi.org/10.1177/104973239800800305 » http://dx.doi.org/10.1177/104973239800800305
• Rochet, J., & Tirole, J. (2006). Two-sided markets: a progress report. The RAND Journal of Economics , 37 (3), 645-667.
• Serenko, A., & Bontis, N. (2004). Meta-review of knowledge management and intellectual capital literature: citation impact and research productivity ranking. Knowledge and Process Management , 11 (3), 185-198. http://dx.doi.org/10.1002/kpm.203 » http://dx.doi.org/10.1002/kpm.203
• Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ (Clinical Research Ed.) , 349 , g7647. PMid:25555855.
• Sutton, R. I., & Staw, B. M. (1995). The relationship between integrating sphere and diffusion theory calculations of fluence rate at the wall of a spherical cavity. Administrative Science Quarterly , 40 (3), 1-12. PMid:7708833. http://dx.doi.org/10.2307/2393788 » http://dx.doi.org/10.2307/2393788
• Thomé, A. M. T., Scavarda, L. F., & Scavarda, A. J. (2016). Conducting systematic literature review in operations management. Production Planning and Control , 27 (5), 408-420. http://dx.doi.org/10.1080/09537287.2015.1129464 » http://dx.doi.org/10.1080/09537287.2015.1129464
• Torraco, R. J. (2005). Writing integrative literature reviews: guidelines and examples. Human Resource Development Review , 4 (3), 356-367. http://dx.doi.org/10.1177/1534484305278283 » http://dx.doi.org/10.1177/1534484305278283
• Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management , 14 (3), 207-222. http://dx.doi.org/10.1111/1467-8551.00375 » http://dx.doi.org/10.1111/1467-8551.00375
• Whetten, D. (2003). O que constitui uma contribuição teórica?. Revista de Administração de Empresas , 43 (3), 69-73.

Publication Dates

• Publication in this collection 2018
• Received 24 Oct 2017
• Accepted 22 Jan 2018

• * [email protected]

SCIMAGO INSTITUTIONS RANKINGS

Figures | tables.

• Figures (1)

Figure 1   Steps for a LR.

Table 1   Pros and cons of a systematic literature review.

• Source: Easterby-Smith et al. (2012 Easterby-Smith, M., Thorpe, R., & Jackson, P. R. (2012). Management research. London: Sage. , p. 109).

Table 2   MA, CA and SLR Examples and Main Points.

Table 3   overview of sources..

• Source: adapted from Easterby-Smith et al. (2012) Easterby-Smith, M., Thorpe, R., & Jackson, P. R. (2012). Management research. London: Sage. .

How to cite

Pdf version for download, related articles.

• Google Scholar

Versões e tradução automática

• Google Translator
• Microsoft Translator

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Writing the literature review for empirical papers 2018 Nakano etal Production

Related Papers

Jorge Muniz Junior

International Journal of P R O F E S S I O N A L Business Review

With a view to examining the entire proposed structure for an empirical article, this editorial focuses on the Literature Review, also known as the Theoretical Framework. The literature review may be defined as “a documented review of published or unpublished works (articles, books, etc.) in specific fields of interest to the work of the researcher” (Ferreira, 2015: 36). It is to be found in conceptual articles such as empirical articles, whether qualitative or quantitative. It has a clear link to the article as a whole and provides support for the section on the development of the concept and the hypotheses/propositions that follow it in the structure of an empirical article.

Publications

Cherley C Du Plessis

The ability to conduct an explicit and robust literature review by students, scholars or scientists is critical in producing excellent journal articles, academic theses, academic dissertations or working papers. A literature review is an evaluation of existing research works on a specific academic topic, theme or subject to identify gaps and propose future research agenda. Many postgraduate students in higher education institutions lack the necessary skills and understanding to conduct in-depth literature reviews. This may lead to the presentation of incorrect, false or biased inferences in their theses or dissertations. This study offers scientific knowledge on how literature reviews in different fields of study could be conducted to mitigate against biased inferences such as unscientific analogies and baseless recommendations. The literature review is presented as a process that involves several activities including searching, identifying, reading, summarising, compiling, analysing, interpreting and referencing. We hope this article serves as reference material to improve the academic rigour in the literature review chapters of postgraduate students' theses or dissertations. This article prompts established scholars to explore more innovative ways through which scientific literature reviews can be conducted to identify gaps (empirical, knowledge, theoretical, methodological, application and population gap) and propose a future research agenda.

Notchie Angeles

Journal of Asian Development

Erni Murniarti

A debatable consensus on and appropriate approaches to literature review function as the theoretical background of the paper. It redefines the literature review substance, synthesis, and procedure to literature matrix, and literature review assessment. In addition, some implications are to interpret the discussion. Finally, finding of the present study is applicable to any study fields at it generally provides matrix for which writing literature review can be easily conducted. Suggestions, identical to the potential of further studies and its application by ongoing researcher and writer, are holistically provided.

Auxiliadora Padilha

Nirmal Kumar Raut

Abdullah Ramdhani , Tatam Chiway , Muhammad Ali Ramdhani

Coaching: An International Journal of Theory, Research and Practice

Mark N K Saunders , Céline Rojon

Exploring and evaluating findings from previous research is an essential aspect of all research projects enabling the work to be set in the context of what is known and what is not known. This necessitates a critical review of the literature in which existing research is discussed and evaluated, thereby contextualising and justifying the project. In this research note we consider what is understood by being critical when reviewing prior to outlining the key attributes of a critical literature review. We conclude with a summary checklist to help ensure a literature review is critical.

Rebekka Tunombili

RELATED PAPERS

Canadian Urological Association Journal

Alessandra Mosca

Paedagoria | FKIP UMMat

Kartika Dian

Image Analysis and Processing–ICIAP …

Revista Albanon

Shefqet Deliallisi

Nature Communications

Mrinmoy Pal

HIV/AIDS - Research and Palliative Care

Olayinka Ogunleye

Boletín de la SECAH

Jose Luis Portillo Sotelo , Rafael Jiménez-Camino Álvarez , Leandro Fantuzzi

East African Journal of Sciences

Zinabu Nigusie

Perisai : Islamic Banking and Finance Journal

Helmi Muhammad

Istanbul University - DergiPark

Natalia de la Cruz

Al-Ulum : Jurnal Penelitian dan Pemikiran Ke Islaman

Syamsul Rijal

Journal of Vacuum Science & Technology A: Vacuum, Surfaces, and Films

Shakil awan

Surgery for Obesity and Related Diseases

Orit Blumenfeld

Educational Studies

Jurnal Agribisnis Dan Ekonomi Pertanian

Camelia Putri

Ganaya : Jurnal Ilmu Sosial dan Humaniora

202010115174 NOVITA SARI

Shanlax Publication

Dr. J. Suresh Kumar

Teoria e prática da pedagogia social, da mediação intercultural e da intervenção social

José Alberto Gallardo-López

Miklos Mezosi

International Journal of Mineral Processing

Ranjan Dwari

Zerbitzuan (Vitoria-Gasteiz)

Amaia Inza Bartolomé

International Journal of Scientific & Technology Research

Tari Purwinda Tarigan Tari P Tarigan

BMC Veterinary Research

Kamal Eltom

Communications Medicine

Andreia Francesli Negri

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024