critique and literature review

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved April 2, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, what is your plagiarism score.

Literature reviews

• Introduction
• Conducting your search
• Store and organise the literature

Evaluate the information you have found

Critique the literature.

• Different subject areas
• Find literature reviews

When conducting your searches you may find many references that will not be suitable to use in your literature review.

• Skim through the resource - a quick read through the table of contents, the introductory paragraph or the abstract should indicate whether you need to read further or whether you can immediately discard the result.
• Evaluate the quality and reliability of the references you find - our page on evaluating information outlines what you need to consider when evaluating the books, journal articles, news and websites you find to ensure they are suitable for use in your literature review.

Critiquing the literature involves looking at the strength and weaknesses of the paper and evaluating the statements made by the author/s.

Books and resources on reading critically

• CASP Checklists Critical appraisal tools designed to be used when reading research. Includes tools for Qualitative studies, Systematic Reviews, Randomised Controlled Trials, Cohort Studies, Case Control Studies, Economic Evaluations, Diagnostic Studies and Clinical Prediction Rule.
• How to read critically - business and management From Postgraduate research in business - the aim of this chapter is to show you how to become a critical reader of typical academic literature in business and management.
• Learning to read critically in language and literacy Aims to develop skills of critical analysis and research design. It presents a series of examples of `best practice' in language and literacy education research.
• Critical appraisal in health sciences See tools for critically appraising health science research.

• << Previous: Store and organise the literature
• Next: Different subject areas >>
• Last Updated: Dec 15, 2023 12:09 PM
• URL: https://guides.library.uq.edu.au/research-techniques/literature-reviews

Literature Reviews

• Introduction
• Tutorials and resources
• Step 1: Literature search
• Step 2: Analysis, synthesis, critique
• Step 3: Writing the review

If you need any assistance, please contact the library staff at the Georgia Tech Library Help website . 

Analysis, synthesis, critique

Literature reviews build a story. You are telling the story about what you are researching. Therefore, a literature review is a handy way to show that you know what you are talking about. To do this, here are a few important skills you will need.

Skill #1: Analysis

Analysis means that you have carefully read a wide range of the literature on your topic and have understood the main themes, and identified how the literature relates to your own topic. Carefully read and analyze the articles you find in your search, and take notes. Notice the main point of the article, the methodologies used, what conclusions are reached, and what the main themes are. Most bibliographic management tools have capability to keep notes on each article you find, tag them with keywords, and organize into groups.

Skill #2: Synthesis

After you’ve read the literature, you will start to see some themes and categories emerge, some research trends to emerge, to see where scholars agree or disagree, and how works in your chosen field or discipline are related. One way to keep track of this is by using a Synthesis Matrix .

Skill #3: Critique

As you are writing your literature review, you will want to apply a critical eye to the literature you have evaluated and synthesized. Consider the strong arguments you will make contrasted with the potential gaps in previous research. The words that you choose to report your critiques of the literature will be non-neutral. For instance, using a word like “attempted” suggests that a researcher tried something but was not successful. For example: 

There were some attempts by Smith (2012) and Jones (2013) to integrate a new methodology in this process.

On the other hand, using a word like “proved” or a phrase like “produced results” evokes a more positive argument. For example:

The new methodologies employed by Blake (2014) produced results that provided further evidence of X.

In your critique, you can point out where you believe there is room for more coverage in a topic, or further exploration in in a sub-topic.

Need more help?

If you are looking for more detailed guidance about writing your dissertation, please contact the folks in the Georgia Tech Communication Center .

• << Previous: Step 1: Literature search
• Next: Step 3: Writing the review >>
• Last Updated: Apr 2, 2024 11:21 AM
• URL: https://libguides.library.gatech.edu/litreview

IOE Writing Centre

• Writing a Critical Review

Writing a Critique

A critique (or critical review) is not to be mistaken for a literature review. A 'critical review', or 'critique', is a complete type of text (or genre), discussing one particular article or book in detail.  In some instances, you may be asked to write a critique of two or three articles (e.g. a comparative critical review). In contrast, a 'literature review', which also needs to be 'critical', is a part of a larger type of text, such as a chapter of your dissertation.

Most importantly: Read your article / book as many times as possible, as this will make the critical review much easier.

1. Read and take notes 2. Organising your writing 3. Summary 4. Evaluation 5. Linguistic features of a critical review 6. Summary language 7. Evaluation language 8. Conclusion language 9. Example extracts from a critical review 10. Further resources

Read and Take Notes

To improve your reading confidence and efficiency, visit our pages on reading.

Further reading: Read Confidently

After you are familiar with the text, make notes on some of the following questions. Choose the questions which seem suitable:

• What kind of article is it (for example does it present data or does it present purely theoretical arguments)?
• What is the main area under discussion?
• What are the main findings?
• What are the stated limitations?
• Where does the author's data and evidence come from? Are they appropriate / sufficient?
• What are the main issues raised by the author?
• What questions are raised?
• How well are these questions addressed?
• What are the major points/interpretations made by the author in terms of the issues raised?
• Is the text balanced? Is it fair / biased?
• Does the author contradict herself?
• How does all this relate to other literature on this topic?
• How does all this relate to your own experience, ideas and views?
• What else has this author written? Do these build / complement this text?
• (Optional) Has anyone else reviewed this article? What did they say? Do I agree with them?

^ Back to top

Organising your writing

You first need to summarise the text that you have read. One reason to summarise the text is that the reader may not have read the text. In your summary, you will

• focus on points within the article that you think are interesting
• summarise the author(s) main ideas or argument
• explain how these ideas / argument have been constructed. (For example, is the author basing her arguments on data that they have collected? Are the main ideas / argument purely theoretical?)

In your summary you might answer the following questions:     Why is this topic important?     Where can this text be located? For example, does it address policy studies?     What other prominent authors also write about this?

Evaluation is the most important part in a critical review.

Use the literature to support your views. You may also use your knowledge of conducting research, and your own experience. Evaluation can be explicit or implicit.

Explicit evaluation

Explicit evaluation involves stating directly (explicitly) how you intend to evaluate the text. e.g. "I will review this article by focusing on the following questions. First, I will examine the extent to which the authors contribute to current thought on Second Language Acquisition (SLA) pedagogy. After that, I will analyse whether the authors' propositions are feasible within overseas SLA classrooms."

Implicit evaluation

Implicit evaluation is less direct. The following section on Linguistic Features of Writing a Critical Review contains language that evaluates the text. A difficult part of evaluation of a published text (and a professional author) is how to do this as a student. There is nothing wrong with making your position as a student explicit and incorporating it into your evaluation. Examples of how you might do this can be found in the section on Linguistic Features of Writing a Critical Review. You need to remember to locate and analyse the author's argument when you are writing your critical review. For example, you need to locate the authors' view of classroom pedagogy as presented in the book / article and not present a critique of views of classroom pedagogy in general.

Linguistic features of a critical review

The following examples come from published critical reviews. Some of them have been adapted for student use.

Summary language

•     This article / book is divided into two / three parts. First...
•     While the title might suggest...
•     The tone appears to be...
•     Title is the first / second volume in the series Title, edited by...The books / articles in this series address...
•     The second / third claim is based on...
•     The author challenges the notion that...
•     The author tries to find a more middle ground / make more modest claims...
•     The article / book begins with a short historical overview of...
•     Numerous authors have recently suggested that...(see Author, Year; Author, Year). Author would also be once such author. With his / her argument that...
•     To refer to title as a...is not to say that it is...
•     This book / article is aimed at... This intended readership...
•     The author's book / article examines the...To do this, the author first...
•     The author develops / suggests a theoretical / pedagogical model to…
•     This book / article positions itself firmly within the field of...
•     The author in a series of subtle arguments, indicates that he / she...
•     The argument is therefore...
•     The author asks "..."
•     With a purely critical / postmodern take on...
•     Topic, as the author points out, can be viewed as...
•     In this recent contribution to the field of...this British author...
•     As a leading author in the field of...
•     This book / article nicely contributes to the field of...and complements other work by this author...
•     The second / third part of...provides / questions / asks the reader...
•     Title is intended to encourage students / researchers to...
•     The approach taken by the author provides the opportunity to examine...in a qualitative / quantitative research framework that nicely complements...
•     The author notes / claims that state support / a focus on pedagogy / the adoption of...remains vital if...
•     According to Author (Year) teaching towards examinations is not as effective as it is in other areas of the curriculum. This is because, as Author (Year) claims that examinations have undue status within the curriculum.
•     According to Author (Year)…is not as effective in some areas of the curriculum / syllabus as others. Therefore the author believes that this is a reason for some school's…

Evaluation language

•     This argument is not entirely convincing, as...furthermore it commodifies / rationalises the...
•     Over the last five / ten years the view of...has increasingly been viewed as 'complicated' (see Author, Year; Author, Year).
•     However, through trying to integrate...with...the author...
•     There are difficulties with such a position.
•     Inevitably, several crucial questions are left unanswered / glossed over by this insightful / timely / interesting / stimulating book / article. Why should...
•     It might have been more relevant for the author to have written this book / article as...
•     This article / book is not without disappointment from those who would view...as...
•     This chosen framework enlightens / clouds...
•     This analysis intends to be...but falls a little short as...
•     The authors rightly conclude that if...
•     A detailed, well-written and rigorous account of...
•     As a Korean student I feel that this article / book very clearly illustrates...
•     The beginning of...provides an informative overview into...
•     The tables / figures do little to help / greatly help the reader...
•     The reaction by scholars who take a...approach might not be so favourable (e.g. Author, Year).
•     This explanation has a few weaknesses that other researchers have pointed out (see Author, Year; Author, Year). The first is...
•     On the other hand, the author wisely suggests / proposes that...By combining these two dimensions...
•     The author's brief introduction to...may leave the intended reader confused as it fails to properly...
•     Despite my inability to...I was greatly interested in...
•     Even where this reader / I disagree(s), the author's effort to...
•     The author thus combines...with...to argue...which seems quite improbable for a number of reasons. First...
•     Perhaps this aversion to...would explain the author's reluctance to...
•     As a second language student from ...I find it slightly ironic that such an anglo-centric view is...
•     The reader is rewarded with...
•     Less convincing is the broad-sweeping generalisation that...
•     There is no denying the author's subject knowledge nor his / her...
•     The author's prose is dense and littered with unnecessary jargon...
•     The author's critique of...might seem harsh but is well supported within the literature (see Author, Year; Author, Year; Author, Year). Aligning herself with the author, Author (Year) states that...
•     As it stands, the central focus of Title is well / poorly supported by its empirical findings...
•     Given the hesitation to generalise to...the limitation of...does not seem problematic...
•     For instance, the term...is never properly defined and the reader left to guess as to whether...
•     Furthermore, to label...as...inadvertently misguides...
•     In addition, this research proves to be timely / especially significant to... as recent government policy / proposals has / have been enacted to...
•     On this well researched / documented basis the author emphasises / proposes that...
•     Nonetheless, other research / scholarship / data tend to counter / contradict this possible trend / assumption...(see Author, Year; Author, Year).
•     Without entering into detail of the..., it should be stated that Title should be read by...others will see little value in...
•     As experimental conditions were not used in the study the word 'significant' misleads the reader.
•     The article / book becomes repetitious in its assertion that...
•     The thread of the author's argument becomes lost in an overuse of empirical data...
•     Almost every argument presented in the final section is largely derivative, providing little to say about...
•     She / he does not seem to take into consideration; however, that there are fundamental differences in the conditions of…
•     As Author (Year) points out, however, it seems to be necessary to look at…
•     This suggest that having low…does not necessarily indicate that…is ineffective.
•     Therefore, the suggestion made by Author (Year)…is difficult to support.
•     When considering all the data presented…it is not clear that the low scores of some students, indeed, reflects…

Conclusion language

•     Overall this article / book is an analytical look at...which within the field of...is often overlooked.
•     Despite its problems, Title offers valuable theoretical insights / interesting examples / a contribution to pedagogy and a starting point for students / researchers of...with an interest in...
•     This detailed and rigorously argued...
•     This first / second volume / book / article by...with an interest in...is highly informative...

Example extracts from a critical review

Writing critically.

If you have been told your writing is not critical enough, it probably means that your writing treats the knowledge claims as if they are true, well supported, and applicable in the context you are writing about. This may not always be the case.

In these two examples, the extracts refer to the same section of text. In each example, the section that refers to a source has been highlighted in bold. The note below the example then explains how the writer has used the source material.    

There is a strong positive effect on students, both educationally and emotionally, when the instructors try to learn to say students' names without making pronunciation errors (Kiang, 2004).

Use of source material in example a: 

This is a simple paraphrase with no critical comment. It looks like the writer agrees with Kiang. (This is not a good example for critical writing, as the writer has not made any critical comment).        

Kiang (2004) gives various examples to support his claim that "the positive emotional and educational impact on students is clear" (p.210) when instructors try to pronounce students' names in the correct way. He quotes one student, Nguyet, as saying that he "felt surprised and happy" (p.211) when the tutor said his name clearly . The emotional effect claimed by Kiang is illustrated in quotes such as these, although the educational impact is supported more indirectly through the chapter. Overall, he provides more examples of students being negatively affected by incorrect pronunciation, and it is difficult to find examples within the text of a positive educational impact as such.

Use of source material in example b: 

The writer describes Kiang's (2004) claim and the examples which he uses to try to support it. The writer then comments that the examples do not seem balanced and may not be enough to support the claims fully. This is a better example of writing which expresses criticality.

^Back to top

Further resources

You may also be interested in our page on criticality, which covers criticality in general, and includes more critical reading questions.

Further reading: Read and Write Critically

We recommend that you do not search for other university guidelines on critical reviews. This is because the expectations may be different at other institutions. Ask your tutor for more guidance or examples if you have further questions.

IOE Writing Centre Online

Self-access resources from the Academic Writing Centre at the UCL Institute of Education.

Anonymous Suggestions Box

Information for Staff

Academic Writing Centre

Academic Writing Centre, UCL Institute of Education [email protected] Twitter:   @AWC_IOE Skype:   awc.ioe

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• CAREER FEATURE
• 04 December 2020
• Correction 09 December 2020

How to write a superb literature review

Andy Tay is a freelance writer based in Singapore.

You can also search for this author in PubMed   Google Scholar

Literature reviews are important resources for scientists. They provide historical context for a field while offering opinions on its future trajectory. Creating them can provide inspiration for one’s own research, as well as some practice in writing. But few scientists are trained in how to write a review — or in what constitutes an excellent one. Even picking the appropriate software to use can be an involved decision (see ‘Tools and techniques’). So Nature asked editors and working scientists with well-cited reviews for their tips.

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

doi: https://doi.org/10.1038/d41586-020-03422-x

Interviews have been edited for length and clarity.

Updates & Corrections

Correction 09 December 2020 : An earlier version of the tables in this article included some incorrect details about the programs Zotero, Endnote and Manubot. These have now been corrected.

Hsing, I.-M., Xu, Y. & Zhao, W. Electroanalysis 19 , 755–768 (2007).

Article   Google Scholar  

Ledesma, H. A. et al. Nature Nanotechnol. 14 , 645–657 (2019).

Article   PubMed   Google Scholar  

Brahlek, M., Koirala, N., Bansal, N. & Oh, S. Solid State Commun. 215–216 , 54–62 (2015).

Choi, Y. & Lee, S. Y. Nature Rev. Chem . https://doi.org/10.1038/s41570-020-00221-w (2020).

Download references

Related Articles

• Research management

Africa’s postdoc workforce is on the rise — but at what cost?

Career Feature 02 APR 24

Impact factors are outdated, but new research assessments still fail scientists

World View 02 APR 24

How scientists are making the most of Reddit

Career Feature 01 APR 24

Adopt universal standards for study adaptation to boost health, education and social-science research

Correspondence 02 APR 24

How can we make PhD training fit for the modern world? Broaden its philosophical foundations

Allow researchers with caring responsibilities ‘promotion pauses’ to make research more equitable

Will the Gates Foundation’s preprint-centric policy help open access?

News 04 APR 24

The corpse of an exploded star and more — March’s best science images

News 28 MAR 24

How papers with doctored images can affect scientific reviews

Global Scientist Interdisciplinary Forum & Recruitment

Southern University of Science and Technology, School of Medicine

Shenzhen, Guangdong, China

Research Associate - Neuroscience and Respiratory Physiology

Houston, Texas (US)

Baylor College of Medicine (BCM)

Histology Laboratory Manager

Postdoctoral scholar - research-pediatrics.

Memphis, Tennessee

The University of Tennessee Health Science Center (UTHSC)

Postdoctoral Scholar

Postdoctoral Scholar - PHAST Investigate Novel Roles of Linear Deubiquitinase Otulin In NeuroDegenerato

Memphis, Tennessee (US)

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Apr 4, 2024 11:06 AM
• URL: https://libguides.usc.edu/writingguide

We use cookies on this site to enhance your experience

By clicking any link on this page you are giving your consent for us to set cookies.

A link to reset your password has been sent to your email.

Back to login

We need additional information from you. Please complete your profile first before placing your order.

Thank you. payment completed., you will receive an email from us to confirm your registration, please click the link in the email to activate your account., there was error during payment, orcid profile found in public registry, download history, difference between a literature review and a critical review.

• Charlesworth Author Services
• 08 October, 2021

As you read research papers, you may notice that there are two very different kinds of review of prior studies. Sometimes, this section of a paper is called a literature review, and at other times, it is referred to as a critical review or a critical context . These differences may be more commonly seen across different fields. Although both these sections are about reviewing prior and existing studies, this article aims to clarify the differences between the two.

Literature review

A literature review is a summary of prior or existing studies that are related to your own research paper . A literature review can be a part of a research paper or can form a paper in itself . For the former, the literature review is designed as a basis upon which your own current study is designed and built. The latter forms a synthesis of prior studies and is a way to highlight future research agendas or a framework.

Writing a literature review

In a literature review, you should attempt to discuss the arguments and findings in prior studies and then work to build on these studies as you develop your own research. You can also highlight the connection between existing and prior literature to demonstrate how the current study you are presenting can advance your knowledge in the field .

When performing a literature review, you should aim to summarise your discussions using a specific aspect of the literature, such as by topic, time, methodology/ design and findings . By doing so, you should be able to establish an effective way to present the relevant literature and demonstrate the connection between prior studies and your research.

Do note that a literature review does not include a presentation or discussion of any results or findings – this should come at a later point in the paper or study. You should also not impose your subjective viewpoints or opinions on the literature you discuss. 

Critical review

A critical review is also a popular way of reviewing prior and existing studies. It can cover and discuss the main ideas or arguments in a book or an article, or it can review a specific concept, theme, theoretical perspective or key construct found in the existing literature .

However, the key feature that distinguishes a critical review from a literature review is that the former is more than just a summary of different topics or methodologies. It offers more of a reflection and critique of the concept in question, and is engaged by authors to more clearly contextualise their own research within the existing literature and to present their opinions, perspectives and approaches .

Given that a critical review is not just a summary of prior literature, it is generally not considered acceptable to follow the same strategy as for a literature review. Instead, aim to organise and structure your critical review in a way that would enable you to discuss the key concepts, assert your perspectives and locate your arguments and research within the existing body of work. 

Structuring a critical review

A critical review would generally begin with an introduction to the concepts you would like to discuss. Depending on how broad the topics are, this can simply be a brief overview or it could set up a more complex framework. The discussion that follows through the rest of the review will then address and discuss your chosen themes or topics in more depth. 

Writing a critical review

The discussion within a critical review will not only present and summarise themes but also critically engage with the varying arguments, writings and perspectives within those themes. One important thing to note is that, similar to a literature review , you should keep your personal opinions, likes and dislikes out of a review. Whether you personally agree with a study or argument – and whether you like it or not – is immaterial. Instead, you should focus upon the effectiveness and relevance of the arguments , considering such elements as the evidence provided, the interpretations and analysis of the data, whether or not a study may be biased in any way, what further questions or problems it raises or what outstanding gaps and issues need to be addressed.

In conclusion

Although a review of previous and existing literature can be performed and presented in different ways, in essence, any literature or critical review requires a solid understanding of the most prominent work in the field as it relates to your own study. Such an understanding is crucial and significant for you to build upon and synthesise the existing knowledge, and to create and contribute new knowledge to advance the field .

Read previous (fourth) in series: How to refer to other studies or literature in the different sections of a research paper

Maximise your publication success with Charlesworth Author Services .

Charlesworth Author Services, a trusted brand supporting the world’s leading academic publishers, institutions and authors since 1928. 

To know more about our services, visit:  Our Services

Share with your colleagues

Related articles.

Conducting a Literature Review

Charlesworth Author Services 10/03/2021 00:00:00

Important factors to consider as you Start to Plan your Literature Review

Charlesworth Author Services 06/10/2021 00:00:00

How to Structure and Write your Literature Review

Charlesworth Author Services 07/10/2021 00:00:00

Related webinars

Bitesize Webinar: How to write and structure your academic article for publication- Module 3: Understand the structure of an academic paper

Charlesworth Author Services 04/03/2021 00:00:00

Bitesize Webinar: How to write and structure your academic article for publication: Module 5: Conduct a Literature Review

Bitesize Webinar: How to write and structure your academic article for publication: Module 7: Write a strong theoretical framework section

Charlesworth Author Services 05/03/2021 00:00:00

Bitesize Webinar: How to write and structure your academic article for publication: Module 11: Know when your article is ready for submission

Literature search.

Why and How to do a literature search

Charlesworth Author Services 17/08/2020 00:00:00

Best tips to do a PubMed search

Charlesworth Author Services 26/08/2021 00:00:00

Best tips to do a Scopus search

A Step-by-Step Guide to Writing a Stellar Literature Review (with Help from AI)

Table of contents

Aren’t all of us mini versions of Sherlock Holmes when browsing data and archives for a research piece? As we go through the process, a comprehensive literature review is an essential toolkit to make your research shine.

A literature review consists of scholarly sources that validate the content. Its primary objective is to offer a concise summary of the research and to let you explore relevant theories and methodologies. Through this review, you can identify gaps in the existing research and bridge them with your contribution. 

The real challenge is how to write an excellent literature review. Let’s learn.

What is the purpose of a literature review?

A literature review is an introduction to your research. It helps you put your perspective to the table, along with a summary of the theme.

What does my literature review communicate?

• Explanation of your research: how the information was collected, the research method, the justification of the chosen data sources, and an overview of the data analysis.
• Framework: the journey from where the concept began and how it is presented.
• Connects the previous and current research: 

It presents the broader scope of your research by connecting it to the existing data and debates and underlining how your content fits the prevailing studies. 

In an era of information overload, a literature review must be well-structured. 

Let’s learn all about the structure and style of a literature review that’ll help you strengthen your research.

Literature review– structure and style

Begin with a question and end it with the solution– the key to structuring a literature review. It resembles an essay’s format, with the first paragraph introducing the readers to the topic and the following explaining the research in-depth.

The conclusion reiterates the question and summarizes the overall insights of your research. There’s no word count restriction. —it depends on the type of research. For example, a dissertation demands lengthy work, whereas a short paper needs a few pages. 

In a literature review, maintaining high quality is vital, with a focus on academic writing style. Informal language should be avoided in favor of a more formal tone. 

The content avoids contractions, clearly differentiating between previous and current research through the use of past and present tense. Wordtune assists in establishing a formal tone, enhancing your work with pertinent suggestions. This AI-powered tool ensures your writing remains genuine, lucid, and engaging. 

The option of refining the tonality offers multiple possibilities for rephrasing a single sentence. Thus, pick the best and keep writing.

Get Wordtune for free > Get Wordtune for free >

Your friendly step-by-step guide to writing a literary review (with help from AI)

Do you find it challenging to begin the literature review? Don’t worry! We’re here to get you started with our step-by-step guide.

1. Narrow down the research scope

Simply begin with the question: What am I answering through my research?

Whether it’s cooking or painting, the real challenge is the prep-up for it rather than performing the task. Once you’re done, it smoothly progresses. Similarly, for your literature review, prepare the groundwork by narrowing down the research scope.

Browse and scoop out relevant data inclining well with your research. While you can’t cover every aspect of your research, pick a topic that isn’t too narrow nor too broad to keep your literature review well-balanced. 

2. Hunt relevant literature

The next question: Does this data align with the issue I’m trying to address?

As you review sources of information, hunt out the best ones. Determine which findings help in offering a focused insight on your topic. The best way to pick primary sources is to opt for the ones featured in reliable publications. You can also choose secondary sources from other researchers from a reasonable time frame and a relevant background.

For example, if your research focuses on the Historical Architecture of 18th-century Europe, the first-hand accounts and surveys from the past would hold more weight than the new-age publications. 

3. Observe the themes and patterns in sources

Next comes: What is the core viewpoint in most of the research? Has it stayed constant over time, or have the authors differed in their points of view?

Ensure to scoop out the essential aspects of what each source represents. Once you have collected all this information, combine it and add your interpretations at the end. This process is known as synthesis.

Synthesize ideas by combining arguments, findings and forming your new version.

4. Generate an outline

The next question: How can I organize my review effectively? When navigating multiple data sources, you must have noticed a structure throughout the research. Develop an outline to make the process easier. An outline is a skeletal format of the review, helping you connect the information more strategically.

Here are the three different ways to organize an outline– Chronologically, Thematically, or by Methodology.You can develop the outline chronologically, starting from the older sources and leading to the latest pieces. Another way of organizing is to thematically divide the sections and discuss each under the designated sub-heading.

You can even organize it per the research methods used by the respective authors. The choice of outline depends on the subject. For example, in the case of a science paper, you can divide the information into sections like introduction, types of equipment, method, procedure, findings, etc. In contrast, it’s best to present it in divisions based on timelines like Ancient, Middle Ages, Industrial revolutions, etc., for a history paper.

If you’re confused about how to structure the data, work with Wordtune. 

With the Generate with AI feature, you can mention your research topic and let Wordtune curate a comprehensive outline for your study.

Having a precise prompt is the key to getting the best results.

5. Start filling!

Your next question must be: Am I ready to compose all the parts of the literature review?

Once you’re ready with the basic outline and relevant sources, start filling in the data. Go for an introductory paragraph first to ensure your readers understand the topic and how you will present it. Ensure you clearly explain the section in the first sentence.

However, if beginning from the first paragraph seems intimidating, don’t worry! Add the main body content to the sub-headings, then jump to the introduction. 

Add headings wherever possible to make it more straightforward and guide your readers logically through different sources. Lastly, conclude your study by presenting a key takeaway and summarizing your findings. To make your task easier, work with Wordtune. It helps align your content with the desired tone and refine the structure.

6. Give attention to detail and edit

The last question: Am I satisfied with the language and content written in the literature review? Is it easy to understand?

Once you’re done writing the first draft of a literature review, it’s time to refine it. Take time between writing and reading the draft to ensure a fresh perspective. It makes it easier to spot errors when you disconnect from the content for some time. Start by looking at the document from a bird's eye to ensure the formatting and structure are in order. 

After reviewing the content format, you must thoroughly check your work for grammar, spelling, and punctuation. One of the best approaches to editing and proofreading is to use Wordtune . It helps simplify complex sentences, enhance the content quality, and gain prowess over the tonality.

The dos and don’ts of writing a literature review

Writing a stellar literature review requires following a few dos and don'ts. Just like Sherlock Holmes would never overlook a hint, you must pay attention to every minute detail while writing a perfect narrative. To help you write, below are some dos and don'ts to remember.

Composing a literature review demands a holistic research summary, each part exhibiting your understanding and approach. As you write the content, make sure to cover the following points:

• Keep a historical background of the field of research. Highlight the relevant relation between the old studies and your new research.
• Discuss the core issue, question, and debate of your topic.
• Theories lay the foundation of research. While you’re writing a literature review, make sure to add relevant concepts and ideas to support your statements.
• Another critical thing to keep in mind is to define complex terminologies. It helps the readers understand the content with better clarity. 

Examples of comprehensive literature reviews

Aren’t good examples the best way to understand a subject? Let’s look into a few examples of literature reviews and analyze what makes them well-written.

1. Critical Thinking and Transferability: A Review of the Literature (Gwendolyn Reece)

An overview of scholarly sources is included in the literature review, which explores critical thinking in American education. The introduction stating the subject’s importance makes it a winning literature review. Following the introduction is a well-defined purpose that highlights the importance of research.

As one keeps reading, there is more clarity on the pros and cons of the research. By dividing information into parts with relevant subheadings, the author breaks a lengthy literature review into manageable chunks, defining the overall structure.

Along with other studies and presented perspectives, the author also expresses her opinion. It is presented with minimal usage of ‘I,’ keeping it person-poised yet general. Toward the conclusion, the author again offers an overview of the study. A summary is further strengthened by presenting suggestions for future research as well. 

2. The Use of Technology in English Language Learning: A Literature Review

This literature review is thematically organized on how technology affects language acquisition. The study begins with an introduction to the topic with well-cited sources. It presents the views of different studies to help readers get a sense of different perspectives. After giving these perspectives, the author offers a personalized opinion.

One of the critical aspects that makes this a good literature review is a dedicated paragraph for definitions. It helps readers proceed further with a clear understanding of the crucial terminologies. There’s a comparison of the modern and previous studies and approaches to give an overall picture of the research.

Once the main body is composed, the author integrates recommendations for action-based tips. Thus, the literature review isn’t just summarizing the sources but offering actions relevant to the topics. Finally, the concluding paragraph has a brief overview with key takeaways.

Wordtune: your writing buddy!

A literature review demands the right balance of language and clarity. You must refine the content to achieve a formal tone and clear structure. Do you know what will help you the most? Wordtune !. 

The real-time grammar checker leaves no scope for errors and lets you retain precision in writing. This writing companion is all you need for stress-free working and comprehensive literature review development.

Let the narrative begin

A literary review isn't just about summarizing sources; it's about seamlessly bringing your perspective to the table. Always remember to set a narrative for added interest and a brilliant composition. With structure and style being the pillars of a stellar literature review, work with Wordtune to ensure zero compromises on the quality.

Share This Article:

How to Properly Conduct Research with AI: Tools, Process, and Approach

What’s a Double Negative? + How To Fix It

The Official Wordtune Guide

Looking for fresh content, thank you your submission has been received.

Advertisement

Supported by

editors’ choice

7 New Books We Recommend This Week

Suggested reading from critics and editors at The New York Times.

• Share full article

Where are all the books about work? That question lands in our inbox from time to time, and no wonder: In terms of hours and paychecks and the sense of identity they impart, jobs are a consuming part of our lives that authors do indeed too often neglect. So this week we recommend three books that put the world of paid labor front and center: Adelle Waldman’s novel “Help Wanted” is set in a suburban box store, Hamilton Nolan’s “The Hammer” assesses the current state of union organizing, and Jane Kamensky’s “Candida Royalle and the Sexual Revolution” takes the measure of a proto-girlboss who went from starring in pornographic movies to launching her own production company with a feminist slant.

Also recommended this week: a look at Saddam Hussein’s state of mind as America and Iraq approached war in 2003, a study of African American literature as a reflection of Black history, a warning about the impacts of climate-fueled migration and, in fiction, Percival Everett’s sparkling riff on the story of Huck Finn, this time centering the character of Huck’s fellow runaway Jim. Happy reading. — Gregory Cowles

JAMES Percival Everett

In this reworking of the “Adventures of Huckleberry Finn,” Jim, the enslaved man who accompanies Huck down the Mississippi River, is the narrator, and he recounts the classic tale in a language that is his own and with surprising details that reveal a far more resourceful, cunning and powerful character than we knew.

“Luxuriates in language. Everett, like Twain, is a master of American argot. … This is Everett’s most thrilling novel, but also his most soulful.”

From Dwight Garner’s review

Doubleday | $28

CANDIDA ROYALLE AND THE SEXUAL REVOLUTION: A History From Below Jane Kamensky

In 1984, Candida Royalle changed the porn industry when she co-founded the female-targeted Femme Productions. As Kamensky convincingly argues in this scholarly and engaging tribute, the performer, producer and director was more than a feminist pioneer; her life mirrored that of the sexual revolution itself.

“Her rigor and thoroughness demand that the reader take seriously an underdog who made her name in a stigmatized industry. This book is a labor of empathy that refuses to simplify or valorize its subject.”

From Rich Juzwiak’s review

Norton | $35

HELP WANTED Adelle Waldman

Waldman’s long-anticipated follow-up to her 2013 debut, “The Love Affairs of Nathaniel P.,” applies her sharp sense of relational drama and dark comedy to the retail work space. The big-box store is Town Square, and the cast of characters who toil there are as surprising and varied as the merchandise they stock.

“Waldman is skilled at building momentum and tension through intricacies of plot. The book shines whenever the group is together, concocting plans … in search of a shared sense of hope.”

From Alexandra Chang’s review

Norton | $28.99

THE HAMMER: Power, Inequality, and the Struggle for the Soul of Labor Hamilton Nolan

The longtime labor reporter and former Gawker journalist’s lively account of the current landscape of the American labor movement paints colorful portraits of union organizers from across the country alongside a pointed critique of the A.F.L.-C.I.O.

“Offers an impressive array of scenes from the front lines of the 21st-century economy. … As ‘The Hammer’ shows, the kind of solidarity that might naturally arise from shared frustrations on the conveyor belt doesn’t necessarily translate to the broader movement all on its own.”

From Willa Glickman’s review

Hachette | $30

THE ACHILLES TRAP: Saddam Hussein, the C.I.A., and the Origins of America’s Invasion of Iraq Steve Coll

Coll’s book stretches from Hussein’s earliest days in power to the U.S. invasion of Iraq in March 2003, tracking the dictator’s state of mind with the help of 2,000 hours of rarely accessed audio from high-level meetings that Hussein “recorded as assiduously as Richard Nixon,” Coll says.

“Most of the story is vivid and sometimes even funny. … Unlike his main character, Coll succeeds in part because he has an eye for dramatic irony.”

From Noreen Malone’s review

Penguin Press | $35

ON THE MOVE: The Overheating Earth and the Uprooting of America Abrahm Lustgarten

The climate is changing, says the author, a climate scientist — and drought, fire and heat waves are going to cause massive demographic shifts. To get a sense of the scale of these changes, the author examines studies and models that simulate future migration scenarios, and combines his insights with first-person reportage. The results are often alarming and admittedly speculative, but never less than compelling.

“The author’s eloquent personal insights … are astonishing as well as gripping, presenting an intimate understanding of why poor agricultural workers, beset by droughts and calamitous economic circumstances, risk everything.”

From Jon Gertner’s review

Farrar, Straus & Giroux | $30

THE BLACK BOX: Writing the Race Henry Louis Gates Jr.

In his latest book, the Harvard scholar shows how African American writers have used the written word to shape their reality despite constraints imposed on them from outside, using the metaphor of the box to reflect ordeals withstood and survived since Africans were first brought to this continent.

“The allure of this book, and the reason for its existence, are the narrative links he draws. … This is a literary history of Black America, but it is also an argument that African American history is inextricable from the history of African American literature.”

From Tope Folarin’s review

Penguin Press | $30

Explore More in Books

Want to know about the best books to read and the latest news start here..

Stephen King, who has dominated horror fiction for decades , published his first novel, “Carrie,” in 1974. Margaret Atwood explains the book’s enduring appeal .

The actress Rebel Wilson, known for roles in the “Pitch Perfect” movies, gets vulnerable about her weight loss, sexuality and money  in her new memoir.

“City in Ruins” is the third novel in Don Winslow’s Danny Ryan trilogy and, he says, his last book. He’s retiring in part to invest more time into political activism .

​​Jonathan Haidt, the social psychologist and author of “The Anxious Generation,” is “wildly optimistic” about Gen Z. Here’s why .

Do you want to be a better reader?   Here’s some helpful advice to show you how to get the most out of your literary endeavor .

Each week, top authors and critics join the Book Review’s podcast to talk about the latest news in the literary world. Listen here .

• Open access
• Published: 08 December 2023

Participatory approaches and methods in gender equality and gender-based violence research with refugees and internally displaced populations: a scoping review

• Michelle Lokot 1 ,
• Erin Hartman 1 &
• Iram Hashmi 1  

Conflict and Health volume  17 , Article number:  58 ( 2023 ) Cite this article

1411 Accesses

1 Citations

13 Altmetric

Metrics details

A Correction to this article was published on 03 January 2024

This article has been updated

Using participatory approaches or methods are often positioned as a strategy to tackle power hierarchies in research. Despite momentum on decolonising aid, humanitarian actors have struggled to describe what ‘participation’ of refugees and internally displaced persons (IDPs) means in practice. Efforts to promote refugee and IDP participation can be tokenistic. However, it is not clear if and how these critiques apply to gender-based violence (GBV) and gender equality—topics that often innately include power analysis and seek to tackle inequalities. This scoping review sought to explore how refugee and IDP participation is conceptualised within research on GBV and gender equality. We found that participatory methods and approaches are not always clearly described. We suggest that future research should articulate more clearly what constitutes participation, consider incorporating feminist research methods which have been used outside humanitarian settings, take more intentional steps to engage refugees and IDPs, ensure compensation for their participation, and include more explicit reflection and strategies to address power imbalances.

Introduction

Within research, ‘participation’ has often been understood as the process of directly involving people who are affected by a particular issue, in the process of research [ 1 ]. Humanitarian actors, including international non-governmental organisations (NGOs), UN actors and local NGOs assert the importance of participation of populations affected by crises—refugees and internally displaced persons (IDPs)—in humanitarian activities. The Humanitarian Accountability Partnership’s (HAP) 2013 standard—a key humanitarian guideline—positions participation as vital to humanitarian accountability. HAP defines participation as: ‘listening and responding to feedback from crisis-affected people when planning, implementing, monitoring and evaluating programmes, and making sure that crisis-affected people understand and agree with the proposed humanitarian action and are aware of its implications’ [ 2 ].

The concept of ‘participatory research’ is sometimes used when discussing how to enhance participation in research. Caroline Lenette and colleagues suggest that when talking about participatory research, there is a difference between taking a ‘holistic approach’ within a broader ‘participatory paradigm’ and using methods identified as ‘participatory’ such as PhotoVoice, that is, a difference between methodology (or approach) and method [ 1 ]. In this paper, we use their framing of approach versus method to distinguish between efforts to embed participatory strategies within research holistically, in contrast with using participatory research methods, while also recognising that both of these framings may co-exist within a research project. Examples of taking a holistic approach include ‘community-based participatory research’ (CBPR) and Participatory Action Research (PAR). CBPR has been used to ensure refugees/IDPs are involved at every stage of the research process, and focuses on ensuring that research practices address unequal power hierarchies and adhere to ethical principles [ 3 , 4 ]. PAR also represents a research paradigm/approach focused on working with populations affected by an issue to generate momentum for change. Scholars urge that care is taken with implementing PAR, because of the risk of creating false hope that action will be taken based on the research [ 5 ]. Research may be labelled as using PAR without real meaning: ‘The trend of putting the terms “participatory” and “action” before “research” has led to co-option: not every project labelled PAR is “participatory” research…’ [ 6 ]. Different to this holistic approach, certain research methods are often associated with being participatory, for example PhotoVoice, theatre or arts-based methods. Scholars have observed the ‘glorification’ of arts-based methods, which may be implemented blindly because they are seen as participatory, creative and innovative—without consideration of the relevance of these methods for affected populations [ 7 ].

The concept of participation has become more common within the humanitarian sector as a result of how it has been operationalised within international development, including through the work of practitioners such as Robert Chambers [ 8 ]. In the development sector, participation was a means of shifting power back to communities, for example, through approaches like ‘participatory rural appraisal’ [ 9 ]. Some have critiqued these efforts, labelling them unsuccessful in shifting power dynamics within international development [ 10 ]. Others point to external shifts that have decreased the focus on hearing directly from affected populations, including mandates from donors that development and humanitarian actors deliver impact and value for money [ 11 ]. Despite participation sometimes being connected to improving efficiency [ 12 ], in humanitarian settings the capacity to be participatory is often pitted against the urgency of responding to crises. For example, taking the time to listen to refugees/IDPs is seen as too challenging with the limited funding offered by short-term emergency projects [ 13 ]. There may also be a distinction between listening to refugees/IDPs and actively involving them in design and analysis of research, especially when listening occurs in an extractive way [ 7 ]. Further complicating matters, the term ‘participation’ is sometimes used interchangeably with other terms, such as inclusion, engagement and involvement [ 14 , 15 ]. Outside of international development and humanitarian action, participatory approaches and methods are recognised as holding important potential for shifting power [ 1 , 16 ], transforming knowledge production [ 17 ], increasing equity [ 18 , 19 ], ensuring marginalised populations are reached [ 20 , 21 , 22 ], and enabling innovative research practice and methods [ 21 , 23 , 24 ].

Humanitarian actors have sought to create processes to enhance the participation of refugees and IDPs within humanitarian activities, including research. Research with refugees and IDPs may be conducted by academic or humanitarian actors, and may include baselines, assessments, evaluations and specific research studies. Within such research, efforts to promote participation may include training refugees and IDPs to collect data themselves, consulting them on their needs, and ensuring that they share their perspectives during evaluations. Humanitarian actors invoke the concept of participation to varying degrees: in instrumental ways to achieve better outcomes, and in practical ways such as through their relationships with refugees and IDPs [ 25 ].

Efforts to enhance refugee/IDP participation in research have been criticised for being tokenistic, stemming from the concept of participation being ‘externally imposed’ [ 15 ]. Involvement of refugees within research has been described as ‘exploitative’, whereby refugees are treated as merely sources of data rather than as individuals [ 26 ]. Conflict-affected populations have expressed frustration with being convened for ‘consultations’ when humanitarian actors have already made decisions about their needs and identified solutions [ 27 ]. Humanitarian actors have also been criticised for only promoting women’s participation to improve efficiency [ 9 ] and for failing to recognise how gender, age, ethnicity, economic status and other power hierarchies might constrain participation in humanitarian settings [ 28 ], which increases the influence of power-holders like refugee elites [ 29 ]. These critiques are not necessarily new, but demonstrate there is lack of clarity on what it means for research to reflect ‘refugee voices’ [ 30 ]. Efforts to be ‘participatory’ often lack clarity on what this means [ 31 ].

Critiques of poor implementation of participation have not specifically been applied to gender equality research. Gender equality research—which includes research on gender-based violence (GBV)—often involves consideration of power dynamics, thus often positions participation as a pre-cursor for gender equality [ 9 ]. Participatory research and feminist research share common goals of empowering marginalised populations [ 1 ]. Understanding how participation occurs within research on gender equality may provide important lessons for how participation is being used in research which already uses power as a key lens. For example, while not among refugees and IDPs, recent examples of feminist participatory research with other populations have considerably advanced scholarship through piloting new methods such as body mapping to understand inequity [ 32 ], digital mapping to conceptualise street harassment [ 33 ] and participatory video to provide new insights on gender inequalities [ 34 ]. Feminists have provided critical new insights for participatory research, such as through emphasising not just women’s voices but also their silences during the research process [ 35 ], and reframing ethics from women’s perspective [ 36 ]. Evaluation practice has also been transformed through use of feminist participatory action research approaches that position evaluation participants as co-researchers, challenging the power dynamics often built into evaluation processes [ 37 , 38 ]. Feminist participatory research has provided particular insights for research on violence, including agenda-setting on the use of trauma-informed approaches [ 39 , 40 ], integrating feminist principles into quantitative studies on violence [ 41 ] and using indigenous feminist approaches to reframe women’s safety [ 42 ]. Feminist research approaches and methods continue to push the boundaries of what it means to be ‘participatory’ in diverse settings [ 43 ].

This scoping review explores academic and grey literature on gender equality and GBV among refugees and IDPs which describes itself as ‘participatory’. Specifically, the objectives of this review were to: (1) describe the contexts, approaches and methods used in gender and GBV research with refugees and IDPs; (2) outline the rationale and impacts of promoting refugee/IDP participation in research; (3) describe how refugee/IDP participation is conceptualised, including how participatory approaches and methods are used in research.

We followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) to conduct and report on this scoping review [ 44 ]. We conducted a scoping review rather than a systematic review to recognise that the body of evidence on refugee/IDP participation in research on gender and GBV is still emerging, and to acknowledge that we must understand how the literature defines participation, what methods are used and what evidence currently exists on the topic. Since we were focused on understanding the concept of participation rather than addressing effectiveness or appropriateness [ 45 ], a scoping review was deemed the best approach. In line with Chang’s approach for scoping reviews [ 46 ], instead of summarising and assessing the quality of evidence, we explored the literature, identified key definitions and themes and identified the type and nature of evidence available.

Search strategy

We searched five academic databases (Medline, PsycINFO, Academic Search Complete, Web of Science and Scopus) in February 2022. The database searches included search terms related to three main concepts: (1) gender equality and GBV, (2) refugees/IDPs, and (3) participation. Table 1 outlines the key search terms used for each database.

We supplemented the academic database search by searching Google and Google Scholar using the following search strings: “refugee participation” AND gender; “refugee participation” AND gender-based violence; refugee AND gender AND participatory research; displaced AND gender AND participatory research. We limited results for Google and Google Scholar to the first 200 hits per search and cleared browsing data after each search. All searches were conducted without signing into Google to prevent tailoring of results by location and search history [ 47 ]. We searched institutional websites of organisations working on gender, GBV and refugee/IDP research, specifically: UNFPA, UN Women, UNHCR, Women’s Refugee Commission and International Center for Research on Women. We also asked practitioners and researchers in this field to send articles that may fulfill inclusion criteria through the Sexual Violence Research Initiative and Forced Migration mailing lists. We hand-searched the reference lists of included papers to identify additional records for inclusion. In order to prevent publication bias and avoid excluding knowledge produced by non-academic actors, we intentionally searched sources outside of academic databases [ 48 ].

Inclusion and exclusion criteria

Articles in English from any time period and country involving empirical research with refugees/IDPs on gender equality or GBV were included. We included high-income settings where refugees are resettled like the United States, Australia, European countries and Canada, recognising firstly that there has been considerable investment in participatory research and emerging scholarship on what it means to be ‘participatory’ from these settings; and secondly that the challenges in active conflict and humanitarian settings would likely prevent participatory research from occurring.

Screening occurred in two stages using Covidence. First, we screened titles and abstracts, excluding non-empirical research, studies unrelated to gender equality or GBV and studies that collected data only amongst host populations or amongst practitioners, rather than refugee/IDP populations,were also excluded. During the full-text review, we narrowed our criteria to search full texts for descriptions of efforts to promote participation of refugees/IDPs. Studies that did not incorporate this term or various forms of it (e.g. ‘participatory’ and ‘involvement’) were excluded. Where multiple records by the same author existed for the same research, only the earliest record was included. During the title/abstract screening and full-text review process, all articles were double-screened with regular meetings held between the three researchers to reach consensus. The first author reviewed all articles at both stages. Table 2 outlines the inclusion and exclusion criteria.

Data analysis

For each included article, we extracted information on: (a) study design (country, type of population, nationality of refugees/IDPs, sample size, research methods), (b) type of gender equality or GBV issue, and (c) participation (level of focus on participation, definitions of participation, rationale for participatory approach, recommendations for future participatory research, impacts of participation). For population type, we classified based on how the populations were described in the study, rather than using legal definitions of refugees, IDPs, migrants or asylum seekers. We defined ‘gender equality’ using UN Women’s definition as ‘equal rights, responsibilities and opportunities of women and men and girls and boys’ [ 49 ]. We initially classified studies on three levels according to the degree to which the participation was a focus: (1) low: participation in research is mentioned in passing/without further discussion or explanation, (2) medium: participation in research is referenced only in the methods section, or (3) high: participation in research is referenced in the methods section as well as throughout the paper.

Data was extracted using Covidence. Each article was extracted by two authors, with the first author extracting every article. We analysed extracted data to identify: whether and how participation was defined and to what extent it was a focus; the types of methods and strategies used to ensure participation of refugees/IDPs in research; the rationale for promoting participation, including how power dynamics were framed; the impacts of participation; and recommendations for improving participation.

Limitations

Our review has a few limitations. Firstly, due to time and staffing constraints, we only searched for a few key concepts related to participation in academic databases, rather than specifically searching for methods or methodologies commonly identified as participatory. This may have limited the studies that were identified in the database search. Secondly, our review is limited by whatever content authors chose to include in their papers, which may not have been fully representative of the holistic approach taken to participation or to the participatory methods used. Authors may have been constrained by their journal requirements, and may not have been able to include the full level of detail. In at least two cases [ 50 , 51 ], methods sections were shorter because the authors subsequently published a solely methods-focused paper—which fell outside the scope of our review. As with any review, our analysis is confined to what authors describe, which may only be a snapshot of what occurred in their research. Finally, our ranking approach was not a straight-forward process and often required judgments be made about the level of content on participation included by authors. While we made decisions about rankings together, it is possible that the lines between categories are more blurred.

Final sample

Out of 2641 results from five academic databases, 1092 were duplicates, resulting in 1549 unique records being screened.

Alongside the academic database records, 88 additional records were identified and screened from Google Scholar (n = 50), Google (n = 26), institutional websites (n = 1), practitioners (n = 7) and through hand-searching references from included papers (n = 4). After screening, 35 of these were included in the full-text review and 8 of these were deemed eligible.

We assessed 244 full-text papers from academic databases for eligibility. Of these, 206 studies (84%) were excluded due to not being empirical research (n = 11), not including refugees/IDPs (n = 3), not being about gender/GBV (n = 34), or not mentioning referencing being participatory in approach or using a participatory method (n = 158). Among studies from other sources, 27 studies (77%) were excluded due to not being about gender/GBV (n = 17) or not being about promoting participation (n = 10). In total, 46 studies were included, specifically 38 from academic databases and 8 studies from other sources. Figure  1 outlines the scoping review process at different stages using an adapted PRISMA framework.

Adapted PRISMA framework

Study types and design

Out of the 46 included studies, 39 adopted a qualitative design and the remaining seven employed quantitative (n = 3) and mixed methods (n = 4). The qualitative studies utilized various methods, including semi-structured interviews, focus group discussions (FGD), ‘participatory group discussions’, and participatory mapping and ranking approaches. In total, eight studies used photography as a research method, with three explicitly mentioning using ‘PhotoVoice’ and the rest adopting a participatory and ethnographic photographic approach. Quantitative studies mainly used surveys, whereas mixed method studies employed interviews and FGDs in addition to surveys. Table 3 shows the characteristics of the studies in this review.

Study settings, populations and funders

Included studies were conducted in 29 countries, with the most studies conducted in the United States (n = 8), followed by Australia (n = 7) and Uganda (n = 5). Most studies were conducted in only one country (n = 40) only, while a smaller number were conducted in three countries (n = 2) or two countries (n = 2). One study was conducted in 8 countries and another in 5 countries. According to geographical region, North America (n = 12), Australia/Asia (n = 13) and sub-Saharan Africa (n = 13) were the most (and equally) represented, followed by Europe/Caucuses (n = 12), and the Middle East and North Africa (n = 8). Only two studies were conducted in South America, both in Colombia.

Overall, close to half the studies (n = 21) collected data solely from refugees. A further 17 studies included some combination of refugees with other populations such as IDPs, (n = 2), practitioners (n = 3), practitioners and other stakeholders (n = 2), migrants (n = 1), asylum seekers (n = 3), undocumented migrants and asylum seekers (n = 1), immigrants (n = 3), immigrants and practitioners (n = 1), and IDPs and practitioners (n = 1). In total 6 studies focused solely on IDP populations, while a further 2 focused on IDPs and practitioners (n = 1) and IDPs, practitioners and other stakeholders (n = 1).

Study methods

Studies employed several different, and sometimes mixed, research methods.

Qualitative methods were most commonly used (93% of included studies used qualitative methods alone or in combination with other methods), and were predominantly structured as interviews or focus group discussions. Interviews were conducted with refugees/IDPs or other community-based actors and took the form of in-depth, semi-structured or biographical interviews. Focus group discussions were formal and informal, stratified by age and gender, or designed as workshops or anecdote circles. Researchers employed varied—and creative and participatory—methods within such interviews and focus group discussions to collect data and learn about the nuances of refugee/IDPs lives and experiences. These techniques included: storytelling, oral histories, and vignettes, safety, community, dream, and body mapping, free listing, timelining, ranking, sorting, and venn-diagramming, art making, document analysis, photo-elicitation, diaries and role play. Studies also used qualitative methods such as observations and methodologies such as ethnographies.

Studies also employed PhotoVoice (or derivatives of participant or auto-photography) and artistic co-creation. Through the taking of photos and their presentation and discussion, photo-based methods enable community strengths, issues, and concerns to be documented and can promote critical dialogue [ 55 ]. Types of artistic co-creation included song, written tests, deejay sets, ‘Grindr poetry’, video poetry, performance, drag, and graphic design [ 67 ].

Researchers also utilised quantitative or mixed-qualitative and quantitative methods for data collection. Three studies used quantitative methods alone, including a knowledge, attitudes and practices survey, a randomised household survey with ‘heads of households’ [ 79 ] and an attitude survey incorporating the ‘Gender Equitable Men’ scale [ 85 ]. Two of these quantitative studies described their participatory approach as involving the creation of advisory groups consisting of refugees who were involved in decision-making about the research [ 71 , 85 ], however the third mentioned using a ‘participatory approach’ and ‘participatory method’ without further explanation [ 79 ]. Further, in this third study, only sampling household heads is limiting as this often results in over-representation of men, limiting women’s participation as research participants. Mixed methods included prioritization exercises (with numerical rankings) and the use of the ‘Sensemaker’ method, which documents micro-narratives of refugees/IDPs lived experiences and, then from these narratives, using a signification framework, participants then create their own set of questions to analyze such narratives [ 78 ].

As will be discussed in later sections, some of these methods were explicitly framed as being participatory. These research methods are distinct from the broader participatory approaches employed.

Gender and GBV focus

In total, 68% of included studies (n = 32) focused on GBV. This included 14 studies that focused solely on GBV, and 18 studies which looked at GBV along with other themes specifically: GBV and adolescent girls (n = 4), GBV and LGBTQIA + (n = 4), GBV and sexual and reproductive health (n = 2), and various combinations of GBV with other topics including economic development, maternal and child health, economic development, division of labour/gender roles, decision-making/leadership and masculinities.

The remaining 32% of included studies (n = 15) focused on topics related to gender equality more broadly without discussing GBV. These topics included LGBTQIA + (n = 4), division of labour/gender roles (n = 2), sexual and reproductive health (n = 2), masculinities (n = 1), and various combinations of division of labour/gender roles with other topics (n = 6). The greater proportion of studies focused on GBV rather than gender equality more broadly may reflect the fact that researching GBV requires greater sensitivity and care (which participatory approaches and methods may help with). For included papers focusing on humanitarian settings (rather than high-income countries hosting refugees), the emphasis on gender equality may also reflect the greater focus within the humanitarian sector on GBV compared to other gender-related issues.

Definitions of participation and ‘participatory’ research

Across all included studies, no definition of the core concept of ‘participation’ was discussed, despite recognition that participation is important. Existing frameworks and definitions were not referenced in these studies.

However, included studies do describe or define different participatory approaches to research. For example, Lenette and colleagues [ 73 ] describe participatory research as research that ‘begins from a social, ethical and moral commitment not to treat people as objects of research but rather, to recognise and value the diverse experiences and knowledges of all those involved (…) Participatory research is often seen as a method that promotes cultural continuity and values gender-specific standpoints’ (757). Feminist participatory research is described by Thompson [ 92 ] as ‘a conscious break with research programs grounded in empiricism (…) Feminist participatory research, then, is not just neutral on the topic of women. It is instead openly committed to a diverse range of women's experiences and women's struggles. It is guided by feminist critiques of science and employs methods that preserve women's experiences in context’ (31).

The concept of ‘Participatory Action Research’ (PAR) was also described in several studies, with a focus on principles of PAR [ 50 , 51 , 52 , 60 , 65 ]. Community-based participatory research (CBPR) principles were also discussed in a few studies [ 64 , 68 , 69 ]. Other concepts that were described were PhotoVoice [ 74 ], action research [ 66 ] and ‘community participatory methodology’ [ 57 ].

Rationale for promoting refugee/IDP participation

Reviewed papers provided several rationales for promoting participation of refugees/IDPs in their research including: their identity as a refugee/IDP, their gender, and their position within power hierarchies. For example, various papers (n = 9) voiced that the experiences and qualities that are intrinsic to refugee/IDP status mandated their active participation in research. With a consensus that there is an overall lack of attention to this population [ 64 ], coupled with their rapidly increasing numbers [ 74 ], authors believed it was especially important to include those with “local, individual and marginalized viewpoints” [ 59 ] that are often outside of traditional “Western” research [ 68 , 69 ], in order to capture a holistic view of their lives [ 94 ]. Authors also viewed their participation as an empowering process, which could counter act often romanticized perceptions and representations of their lives, such as that they are all traumatized [ 95 ]. Participatory research was also positioned as responding to the fact that research with refugees does not use strengths-based approaches [ 55 ]. Thompson believed participation—via the recall and collection of their stories—could help participants to reconstruct their lives [ 92 ].

Further, several reviewed papers (n = 8) cited feminist theory as a rationale for promoting participation amongst research with refugees/IDPs [ 28 , 55 , 67 , 70 , 74 , 77 , 92 , 94 ]. Most referred to encouraging women and girls to join the research process. However, one paper purposely included adolescent boys so to understand their perspectives on issues around gender inequality and marriage [ 78 ] and a few purposely included LGBTQIA + refugees/IDPs (n = 2). Overall, rationales for including women and girls were two-fold. First, they either conceptualized knowledge as a (feminist) process of emancipation and social change [ 95 ], and thus, included women and girls to address gender stereotypes that persist in research [ 74 ]. For example, they recognized that women and girls are less likely to participate in mixed-gendered research spaces and that their contributions to knowledge are often viewed as less valuable [ 95 ]. Secondly, the rationale used for including women and girls was in order to ensure that research recommendations would be centred on their specific needs and experiences, for example, to ensure that their specific safety concerns would be included.

Moreover, many studies (n = 10) sought to include refugees/IDPs within the research process to address the power imbalances that are often present within research and ensure more democratic equitable research. This often included descriptions of how power dynamics can make research ‘exploitative’ [ 95 ]. In Pangcoga and Gambir’s study, the Sensemaker method made the research more ‘democratic’, enabling participants voices to be centred while addressing power imbalances [ 78 ], while others identified how their choice of methods such as participatory photography, visual methods and production of artistic outputs helped to reduce power dynamics [ 28 , 67 , 94 ]. In both ‘Empowered Aid’ studies, PAR was stated as a means of recognising and tackling power imbalances [ 50 , 51 ]. Other studies also took a holistic approach to being participatory through strategies such as asking open-ended questions [ 28 , 73 ], reflecting on power and positionality [ 28 , 92 , 94 ], spending more time with refugees and thinking about how best to represent their lives [ 28 , 72 , 94 ]. Studies acknowledged that it was challenging to fully address power imbalances [ 95 ].

Level of focus on participation

As part of the extraction process, we classified included studies according to how authors’ described their study’s focus on on participation. This was driven by our recognition—also discussed in literature—that the concept of participation has often been co-opted by authors [ 6 ] when describing their methods, without due consideration to the fidelity and robustness of participation. Firstly, we classified 15% of studies (n = 7) as having ‘low’ content on participation—describing studies where being participatory was mentioned in passing only, without further explanation. We then used the framing by Lenette et al. [ 1 ] to contrast the use of a participatory ‘approach’ (i.e. a holistic process made up of multiple strategies to embed participation across the research process), and the use of a participatory ‘method’ (i.e. the use of a specific research method such as PhotoVoice or video). We created three categories to classify the studies that were not categorised as ‘low’: studies that only use participatory method(s), studies that only use a participatory approach, or studies that use both a participatory method and participatory approach. We suggest that simply using a participatory method is not always sufficient to address power hierarchies within research, rather using a more holistic participatory approach encompassing multiple strategies is more helpful.

Content classified as ‘low’ (n=7) tended to involve singular references to participation or being participatory without any further explanation [ 61 , 79 , 80 , 81 , 86 , 96 , 97 ]. For example, using the term ‘participatory qualitative design’ only in the abstract with no additional reference in the text [ 61 ], or referring to a ‘participatory approach’ or ‘participatory research’ without further explanation [ 79 , 86 , 97 ].

One study classified as low referred to ‘ethnographic participatory fieldwork’ [ 81 ] and listed classroom interactions and language portraits as examples, without explaining these methods further. It is unclear if the methods alone were the reason for using the term ‘participatory’ or if something related to the methodology of the ethnographic fieldwork was participatory. Similarly, another study mentioned ‘participatory FGDs’ and said this involved drawing and poetry, but did not provide further detail on this approach [ 96 ], seeming to reflect Ozkul’s [ 7 ] critique that arts-based methods are sometimes automatically assumed to be participatory.

Some of these examples may reflect what Cornwall and Brock [ 98 ] refer to as ‘buzzwords’. Using the term participation or participatory may invoke positive associations without resulting in refugees or IDPs meaningfully participating in research processes. However, we also recognise that the level of content included to describe participatory approaches and methods are not always reflective of whether studies actually used these approaches. For example, disciplinary styles of writing, journal requirements and feedback from peer reviewers may all result in less (or more) description being added about methods.

Studies that only use participatory approaches

In total, 17 studies used a holistic participatory approach in isolation—without also mentioning use of a participatory method. The table below outlines the types of strategies used to enhance participation. We took a broad approach in categorising these studies as taking a participatory approach, recognising that not all practices were explicitly labelled as participatory. For example, one study [ 72 ] only mentioned the word ‘participatory’ in passing, yet the practices described in the methods (including having an advisory committee that was connected to the community) align with participatory approaches.

In a few cases, it was not clear if studies also used a participatory method. For example, two studies included community members at each stage of the research as part of the broader participatory approach, but it was unclear if the use of video-elicitation within the FGDs constituted a participatory method [ 68 , 69 ]. In another case where a feminist participatory approach was described, it was not clear if the use of ‘dream narratives’ may constitute use of a participatory method [ 92 ].

Studies that only use participatory methods

In total, 11 studies used participatory methods in isolation [ 53 , 54 , 55 , 59 , 62 , 63 , 74 , 83 , 84 , 87 , 89 ]. The methods chosen included participatory photography including participatory mapping [ 84 , 89 ], PhotoVoice [ 62 ] and participatory ranking methodologies [ 53 ]. A few studies did not fully explain their use of participatory methods. One study mentioned the use of ‘participatory workshop methods’ multiple times without explaining what this meant [ 83 ]. One study used PAR meetings with refugees to gather data [ 54 ] and another used ‘participatory learning and action’ (PLA) [ 59 ], but neither outlined in detail the PAR and PLA approaches, though Akash & Chalmiers noted that they describe their methodology in another paper [ 54 ].

In a few cases, studies were stated as using a participatory approach, however in reality these described methods and were counted within the ten studies above that only talked about methods. In two studies, CBPR was stated as the methodology but only PhotoVoice [ 55 ] or only PhotoVoice and interviews [ 74 ] were used as the method—and there was no other indication that a broader CBPR approach was taken. Elsewhere PAR was stated as the methodology, but in one study only the use of photo-ethnography as method rather than PAR more broadly was evident based on the paper description [ 87 ]. Another study mentioned the use of PAR meetings which were also described as creating space for women ‘to enable women to flexibly tell their own stories of marriage using a life events-narrative approach’ [ 54 ]—which sounds less like a participatory method and more like a life history interview.

Studies that use both participatory methods and participatory approaches

In total, 11 studies clearly stated the use of both a participatory approach as well as a participatory method [ 28 , 50 , 51 , 56 , 58 , 67 , 73 , 78 , 82 , 91 , 94 ].

Strategies used to enhance participation

The most common strategy used within the 27 studies that took a participatory approach was involving participants in design, data collection and analysis (including through an advisory group), which 17 studies mentioned. Other strategies included refugees/IDPs only participating in design/influencing the research agenda (n = 5), refugees/IDPs only participating in analysis/feedback (n = 3), using peer data collectors (n = 4) and providing in-kind or financial compensation for refugees/IDPs who participated (n = 3) (Table 4 ).

While this list (which is not mutually-exclusive) represents a helpful indication of the ways in which GBV and gender equality research has sought to promote refugee/IDP participation, it is important to note the challenges in using these strategies which many studies discussed. The time and financial cost associated with participatory approaches can be significant; and it is not always possible to compensate refugees/IDPs for their time [ 66 ]. Even if researchers intend to promote participation, refugees/IDPs may not always be accustomed to or comfortable with participating and may not engage as much as hoped [ 95 ]. Efforts to enable participants to co-create outputs may not always be successful as participants may be not used to having more autonomy and voice [ 67 ]. These challenges complicate efforts to promote refugee/IDP participation.

Impacts of participation of refugees/IDPs in research

Some studies explicitly commented on the impacts of using participatory methods and strategies. For example, studies stated that using this approach to research increased participants’ well-being and confidence [ 55 , 94 ]. Participants reported feeling heard [ 64 ]. Participatory research also created opportunities for socialisation amongst participants [ 73 ]. Engaging communities throughout the research enabled communities to create knowledge and develop local strategies for change [ 66 ].

Other studies did not specifically comment on concrete impacts but discussed the potential of participatory methods and strategies to contribute towards increasing solidarity [ 95 ], creating transformative experiences for participants [ 74 ], preventing research fatigue [ 95 ], and improving research rigour and ethics [ 69 ].

This scoping review explored how the concept of participation is operationalised in research with refugees and IDPs. Our review highlights how despite recognition that participation of refugees/IDPs is important for research, the concept of participation continues to be used tokenistically, as a ‘buzzword’ [ 98 ] that is misappropriated to describe a myriad of research approaches and methods.

In our study, we found that while many studies use gender (including specifically drawing on feminist theory), or refugee/IDP status to explain the reason for taking a participatory approach, in many cases there was not a concerted effort to understand and outline the reasons why participation is important—and even less effort to document the impacts of using participatory approaches and methods. The power hierarchies within research more generally do provide a strong incentive for researchers to tackle imbalances inherent within the research process, however these dynamics were not often discussed in included papers. We suggest that conducting power analysis more broadly—including analysing power dynamics within research, gendered power dynamics and dynamics between refugess/IDPs and researchers—may provide stronger rationale for promoting participation, making it easier to identify concrete opportunities for refugee/IDP participation in research.

While only a small number of studies were classified as having limited/passing references to being participatory, those that did include references to either using participatory methods or participatory approaches more broadly, at times did not fully explain what exactly was participatory about the research. Methods like FGDs were described as being participatory, without it being clear what made this approach participatory. Even when approaches like CBPR or PAR were referenced, the descriptions of research practices were sometimes limited. Some of this gap is due to journal and peer reviewer expectations, as well as practices within research disciplines—rather than necessarily reflecting that participatory methods and approaches are not being used. Thus, we recommend more robust descriptions of how researchers action participation within research outputs, so that the wider research community can learn not only what they have accomplished, but how they accomplished it.

Where participatory approaches were used, we found that the use of specific strategies to promote participation tended to focus on involving refugees/IDPs in providing advice across the research process—a positive sign. In some cases, refugees were engaged as ‘peer researchers,’ though this strategy has also been critiqued by others as containing potential for exploitation [ 26 , 99 ]. Importantly, engaging refugees/IDPs during analysis was less common, representing a gap in current strategies to promote participation, which others have also identified [ 100 ]. Thus, we suggest aiming to involve refugees and IDPs more in analysis, all whilst recognising also the additional burden on this engagement might place on refugees by seeking to find less time-intensive ways of seeking input on the findings and ensuring renumeration for this participation. Moreover, providing some kind of incentive or benefit for refugees/IDPs to participate was only mentioned in a few studies, although this would have meaningful impact for refugees/IDPs. While this review highlights that among refugees and IDPs there are limited examples of the systematic use of both participatory approaches across a research process, and use of participatory methods, we suggest much can be learnt from feminist participatory research among other populations. Feminist participatory research continues to provide innovative ways of understanding power, challenging how knowledge is produced (and by whom) and framing issues from women’s perspective [ 33 , 36 , 43 ]. However, many of these methodological advancements are yet to be tested in settings with refugees and IDPs. We suggest that particularly in humanitarian emergencies, the default assumption may be that using innovative methods is less realistic. Indeed, the urgent nature of the humanitarian response has at times acted as a justification for not considering issues of power in sufficient depth or not spending enough time to understand issues before responding [ 28 , 101 ]. In the same way, the limited level of innovation within research methods among refugees and IDPs may be driven by assumptions about what is possible to implement within a humanitarian emergency. Notwithstanding the challenges in obtaining research funding for research in humanitarian settings that uses innovative methods, we suggest more work needs to be done to consider the value of participatory methods—beyond PhotoVoice—for research among refugees and IDPs.

We recommend that future research among refugees and IDPs should:

More explicitly detail how researchers sought to promote participation of refugees/IDPs, including clearer conceptualisations of what constitutes refugee/IDP participation and how they operationalised this.

Consider the use of innovative, feminist research methods that can challenge power dynamics and provide new opportunities for refugees and IDPs to share their lived experience. Learning from feminist participatory research methods used outside of refugee and IDP populations may provide important lessons to bring innovative research methods into the humanitarian sector.

Continue to engage refugees and IDPs in research design and analysis in particular, and use other strategies such as in-kind and financial compensation to recognise the contribution refugees and IDPs make towards research.

Include more explicit reflection on how power affects the research process and deliberately incorporate participatory approaches and methods to address this, including drawing on feminist and participation frameworks applied in other settings to ensure refugee/IDP participation is meaningful and not solely lip service. This should include consideration of how participatory approaches and methods align with key principles of rigorous, ethical research.

Seek to analyse the impacts of incorporating participatory approaches and methods on refugees/IDPs themselves, to help with documenting both positive impacts and unintended/negative impacts.

Availability of data and materials

All data generated or analysed during this study are included in this published article and its supplementary information files.

Change history

03 january 2024.

A Correction to this paper has been published: https://doi.org/10.1186/s13031-023-00559-0

Lenette C, Stavropoulou N, Nunn CA, Kong ST, Cook T, Coddington K, Banks S. Brushed under the carpet: examining the complexities of participatory research. Res All. 2019;3:161–79.

Article   Google Scholar  

Humanitarian Accountability Partnership. 2013 Humanitarian Accountability Report. 2013.

Filler T, Benipal PK, Torabi N, Minhas RS. A chair at the table: a scoping review of the participation of refugees in community-based participatory research in healthcare. Global Health. 2021;17:1–11.

Google Scholar  

Kia-Keating M, Juang LP. Participatory science as a decolonizing methodology: leveraging collective knowledge from partnerships with refugee and immigrant communities. Cult Divers Ethn Minor Psychol. 2022. https://doi.org/10.1037/cdp0000514 .

Martin SB, Burbach JH, Benitez LL, Ramiz I. Participatory action research and co-researching as a tool for situating youth knowledge at the centre of research. London Rev Educ. 2019;17:297–313.

Starodub A. Horizontal participatory action research: Refugee solidarity in the border zone. Area. 2019;51:166-17S3.

Ozkul D. Participatory research: still a one-sided research agenda? Migr Lett. 2020;17:229–37.

Chambers R. Whose reality counts? Putting the first last. Practical Action Publishing Ltd, London. 1997.

Cornwall A. Whose voices? Whose choices? Reflections on gender and participatory development. World Dev. 2003;31:1325–42.

Cooke B, Kothari U. Participation: the new tyranny? New York: Zed; 2001.

Eyben R. Uncovering the politics of “evidence” and “results”. A framing paper for development practitioners. 2013.

Olivius E. (Un)governable subjects: the limits of refugee participation in the promotion of gender equality in humanitarian aid. J Refug Stud. 2014;27:42–61.

Brown D, Donini A, Knox Clarke P. Engagement of crisis-affected people in humanitarian action. 2014.

Anderson MB, Brown D, Jean I. Time to listen. Hearing people on the receiving end of international aid, 1st ed. CDA Collaborative Learning Projects, Cambridge, MA. 2012.

Women’s Refugee Commission. Understanding past experiences to strengthen feminist responses to crises and forced displacement. New York. 2021.

Vaughn LM, Jacquez F. Participatory research methods—choice points in the research process. J Particip Res Methods. 2020. https://doi.org/10.35844/001c.13244

Fine M, Torre ME, Oswald AG, Avory S. Critical participatory action research: methods and praxis for intersectional knowledge production. J Couns Psychol. 2021;68:344–56.

Article   PubMed   Google Scholar  

Rodriguez Espinosa P, Verney SP. The underutilization of community-based participatory research in psychology: a systematic review. Am J Community Psychol. 2021;67:312–26.

Wallerstein N, Oetzel JG, Sanchez-Youngman S, et al. Engage for equity: a long-term study of community-based participatory research and community-engaged research practices and outcomes. Heal Educ Behav. 2020;47:380–90.

McFarlane SJ, Occa A, Peng W, Awonuga O, Morgan SE. Community-based participatory research (CBPR) to enhance participation of racial/ethnic minorities in clinical trials: a 10-year systematic review. Health Commun. 2022;37:1075–92.

Patel MR, TerHaar L, Alattar Z, Rubyan M, Tariq M, Worthington K, Pettway J, Tatko J, Lichtenstein R. Use of storytelling to increase navigation capacity around the affordable care act in communities of color. Prog Commun Heal Partnerships Res Educ Action. 2018;12:307–19.

Sharpe L, Coates J, Mason C. Participatory research with young people with special educational needs and disabilities: a reflective account. Qual Res Sport Exerc Heal. 2022;14:460–73.

Lenette C, Brough M, Schweitzer RD, Correa-Velez I, Murray K, Vromans L. ‘Better than a pill’: digital storytelling as a narrative process for refugee women. Media Pract Educ. 2019;20:67–86.

Duran B, Oetzel J, Magarati M, et al. Toward health equity: a national study of promising practices in community-based participatory research. Prog Community Heal Partnerships Res Educ Action. 2019;13:337–52.

Bottomley B. NGO strategies for sex and gender-based violence protection and accountability in long-term displacement settings: Reviewing women’s participation in humanitarian programmes in Dadaab refugee complex. RLI Work Pap No. 2021;56:1–20.

Pincock K, Bakunzi W. Power, participation, and ‘peer researchers’: addressing gaps in refugee research ethics guidance. J Refug Stud. 2021;34:2333–48.

Anderson K. Tearing down the walls. Confronting the barriers to internally displaced women and girls’ participation in humanitarian settings. 2019.

Lokot M. The space between us: feminist values and humanitarian power dynamics in research with refugees. Gend Dev. 2019. https://doi.org/10.1080/13552074.2019.1664046 .

Omata N. Unwelcome participation, undesirable agency? Paradoxes of de-politicisation in a refugee camp. Refug Surv Q. 2017;36:108–31.

Doná G. The microphysics of participation in refugee research. J Refug Stud. 2007;20:210–29.

Rass E, Lokot M, Brown FL, Fuhr DC, Asmar MK, Smith J, McKee M, Orm IB, Yeretzian JS, Roberts B. Participation by conflict-affected and forcibly displaced communities in humanitarian healthcare responses: a systematic review. J Migr Heal. 2020;1–2: 100026.

Mayra K. Birth mapping: a visual arts-based participatory research method embedded in feminist epistemology. Int J Qual Methods. 2022;21:160940692211053.

Fileborn B. Digital mapping as feminist method: critical reflections. Qual Res. 2023;23:343–61.

Cin FM, Süleymanoğlu-Kürüm R. Participatory video as a tool for cultivating political and feminist capabilities of women in Turkey. In: Particip Res Capab Epistemic Justice. Springer International Publishing, Cham; 2020, pp 165–188.

Lykes MB, Bianco ME, Távara G. Contributions and limitations of diverse qualitative methods to feminist participatory and action research with women in the wake of gross violations of human rights. Methods Psychol. 2021;4: 100043.

Buchanan K, Geraghty S, Whitehead L, Newnham E. Woman-centred ethics: a feminist participatory action research. Midwifery. 2023;117: 103577.

Crupi K, Godden NJ. Feminist evaluation using feminist participatory action research: guiding principles and practices. Am J Eval. 2023. https://doi.org/10.1177/10982140221148433 .

Mcdiarmid T, Pineda A, Scothern A. We are women! We are ready! Amplifying women’s voices through feminist participatory action research. Eval J Australas. 2021;21:85–100.

Jumarali SN, Nnawulezi N, Royson S, Lippy C, Rivera AN, Toopet T. Participatory research engagement of vulnerable populations: employing survivor-centered, trauma-informed approaches. J Particip Res Methods. 2021. https://doi.org/10.35844/001c.24414

Anderson KM, Karris MY, DeSoto AF, Carr SG, Stockman JK. Engagement of sexual violence survivors in research: trauma-informed research in the THRIVE study. Violence Against Women. 2023;29:2239–65.

Leung L, Miedema S, Warner X, Homan S, Fulu E. Making feminism count: integrating feminist research principles in large-scale quantitative research on violence against women and girls. Gend Dev. 2019;27:427–47.

Dorries H, Harjo L. Beyond safety: refusing colonial violence through indigenous feminist planning. J Plan Educ Res. 2020;40:210–9.

Guy B, Arthur B. Feminism and participatory research: exploring intersectionality, relationships, and voice in participatory research from a feminist perspective. SAGE Handb. Particip. Res. Inq. 2021.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169:467–73.

Munn Z, Peters M, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18:1–7.

Chang S. Scoping reviews and systematic reviews: Is it an either/or question? Ann Intern Med. 2018;169:502–3.

Piasecki J, Waligora M, Dranseika V. Google search as an additional source in systematic reviews. Sci Eng Ethics. 2017. https://doi.org/10.1007/s11948-017-0010-4 .

Article   PubMed   PubMed Central   Google Scholar  

Adams RJ, Smart P, Huff AS. Shades of grey: guidelines for working with the grey literature in systematic reviews for management and organizational studies. Int J Manag Rev. 2017;19:432–54.

UN Women Concepts and definitions. https://www.un.org/womenwatch/osagi/conceptsandefinitions.htm . Accessed 4 Apr 2023.

Potts A, Fattal L, Hedge E, Hallak F, Reese A. Empowered Aid: Transforming gender and power dynamics in the delivery of humanitarian aid. Participatory action research with refugee women & girls to better prevent sexual exploitation & abuse—Lebanon Results Report. 2020.

Potts A, Kolli H, Hedge E, Ullman C. Empowered aid: transforming gender and power dynamics in the delivery of humanitarian aid. Participatory action research with refugee women & girls to better prevent sexual exploitation & abuse—Uganda Results Report. 2020.

Affleck W, Thamotharampillai U, Jeyakumar J, Whitley R. “If One Does Not Fulfil His Duties, He Must Not Be a Man”: masculinity, mental health and resilience amongst Sri Lankan Tamil refugee men in Canada. Cult Med Psychiatry. 2018;42:840–61.

Ager A, Bancroft C, Berger E, Stark L. Local constructions of gender-based violence amongst IDPs in northern Uganda: analysis of archival data collected using a gender- and age-segmented participatory ranking methodology. Confl Health. 2018;12:10.

Al Akash R, Chalmiers MA. Early marriage among Syrian refugees in Jordan: exploring contested meanings through ethnography. Sex Reprod Heal Matters. 2021;29:287–302.

Dantas J, Lumbus A, Gower S. Empowerment and health promotion of refugee women: the photovoice project. 2018.

Edström J, Dolan C. Breaking the spell of silence: collective healing as activism amongst Refugee male survivors of sexual violence in Uganda. J Refug Stud. 2019;32:175–96.

Ellis BH, MacDonald HZ, Klunk-Gillis J, Lincoln A, Strunin L, Cabral HJ. Discrimination and mental health among Somali refugee adolescents: the role of acculturation and gender. Am J Orthopsychiatry. 2010;80:564–75.

Elnakib S, Hussein SA, Hafez S, Elsallab M, Hunersen K, Metzler J, Robinson WC. Drivers and consequences of child marriage in a context of protracted displacement: a qualitative study among Syrian refugees in Egypt. BMC Public Health. 2021;21:674.

Fincham K. Constructions, contradictions and reconfigurations of ‘Manhood’ among youth in Palestinian camps in Lebanon. Int J Educ Dev. 2014;37:48–56.

Fineran S, Kohli HK. Muslim refugee women’s perspectives on intimate partner violence. J Fam Soc Work. 2020;23:199–213.

Fisher C. Changed and changing gender and family roles and domestic violence in African refugee background communities post-settlement in Perth, Australia. Violence Against Women. 2013;19:833–47.

Fobear K. Nesting bodies: exploration of the body and embodiment in LGBT refugee oral history and participatory photography. Soc Altern. 2016;35:33–43.

Gibb C. The evacuation camp as paradoxical space for women. Polit Geogr. 2022;93: 102546.

Green BA, Latifi Y. No one smiles at me: the double displacement of Iranian migrant men as refugees who use drugs in Australia. Soc Sci. 2021;10:85.

Guerin PB, Allotey P, Elmi FH, Baho S. Advocacy as a means to an end: assisting refugee women to take control of their reproductive health needs. Women Health. 2006;43:7–25.

Gustafson DT, Iluebbey V. “Traditional discipline” or domestic violence": participatory action research with a Sudanese refugee community. J Cult Divers Preview Publ details. 2013;20:51–6.

Holle F, Rast MC, Ghorashi H. Exilic (art) narratives of queer refugees challenging dominant hegemonies. Front Sociol. 2021. https://doi.org/10.3389/fsoc.2021.641630 .

Johnson-Agbakwu CE, Helm T, Killawi A, Padela AI. Perceptions of obstetrical interventions and female genital cutting: insights of men in a Somali refugee community. Ethn Health. 2014;19:440–57.

Johnson CE, Ali SA, Shipp MP-L. Building community-based participatory research partnerships with a Somali refugee community. Am J Prev Med. 2009;37:S230–6.

Keygnaert I, Dialmy A, Manço A, Keygnaert J, Vettenburg N, Roelens K, Temmerman M. Sexual violence and sub-Saharan migrants in Morocco: a community-based participatory assessment using respondent driven sampling. Global Health. 2014;10:32.

Keygnaert I, Dias SF, Degomme O, Deville W, Kennedy P, Kovats A, De Meyer S, Vettenburg N, Roelens K, Temmerman M. Sexual and gender-based violence in the European asylum and reception sector: a perpetuum mobile? Eur J Public Health. 2015;25:90–6.

Lee EOJ, Brotman S. Identity, refugeeness, belonging: experiences of sexual minority refugees in Canada. Can Rev Sociol Can Sociol. 2011;48:241–74.

Lenette C, Maror A, Manwaring S. Mothers & daughters : redefining cultural continuity through South Sudanese women’s artistic practices. J Intercult Stud. 2019;40:751–71.

McMorrow S, Saksena J. Voices and views of congolese refugee women: a qualitative exploration to inform health promotion and reduce inequities. Heal Educ Behav. 2017;44:769–80.

Mehta PK, Saia K, Mody D, Crosby SS, Raj A, Maru S, Piwowarczyk L. Learning from UJAMBO: Perspectives on Gynecologic Care in African Immigrant and Refugee Women in Boston, Massachusetts. J Immigr Minor Health. 2018;20(2):380–7. https://doi.org/10.1007/s10903-017-0659-4 .

Mulé NJ. Mental health issues and needs of LGBTQ+ asylum seekers, refugee claimants and refugees in Toronto, Canada. Psychol Sex. 2022;13(5):1168–78. https://doi.org/10.1080/19419899.2021.1913443 .

Murray L, Warr D, Chen J, Block K, Murdolo A, Quiazon R, Davis E, Vaughan C. Between ‘here’ and ‘there’: family violence against immigrant and refugee women in urban and rural Southern Australia. Gender Place Cult. 2019;26:91–110.

Pangcoga AH, Gambir K. Our voices, our future: understanding risks and adaptive capacities to prevent and respond to child marriage in the Bangsamoro Autonomous Region in Muslim Mindanao (BARMM). 2022.

Rahamtalla NM, Saeed AM. Gender analysis of the impacts of displacement on Western Sudanese Migrants in Khartoum State, Sudan. Ahfad J. 2009;26:15–46.

Rees S, Pease B. Domestic violence in refugee families in Australia. J Immigr Refug Stud. 2007;5:1–19.

Rezaian A, Daskalaki I, Apostolidou A. Gendered spaces and educational expectations: the case of the former refugee camp “Elliniko” in athens. J Int Migr Integr. 2020;21:155–70.

Ritterbusch AE. Mobilities at gunpoint: the geographies of (Im)mobility of transgender sex workers in Colombia. Ann Am Assoc Geogr. 2016; 1–12

Rothkegel S, Poluda J, Wonani C, Papy J, Engelhardt-Wendt E, Weyermann B, Hennig R. Evaluation of UNHCR’s efforts to prevent and respond to sexual and gender-based violence in situations of forced displacement. 2008.

Schulte J, Rizivi Z. In Search of Safety and Solutions : Somali Refugee Adolescent Girls at Sheder and Aw Barre Camps, Ethiopia. 2012.

Scott J, Averbach S, Modest AM, Hacker M, Cornish S, Spencer D, Murphy M, Parmar P. An assessment of attitudes toward gender inequitable sexual and reproductive health norms in South Sudan: a community-based participatory research approach. Confl Health. 2013;7:1–8.

Scott J, Averbach S, Modest AM, Hacker MR, Cornish S, Spencer D, Murphy M, Parmar P. An assessment of gender inequitable norms and gender-based violence in South Sudan: a community-based participatory research approach. Confl Health. 2013;7:1.

Shanahan F, Veale A. How mothers mediate the social integration of their children conceived of forced marriage within the lord’s resistance army. Child Abuse Negl. 2016;51:72–86.

Simbandumwe L, Bailey K, Denetto S, Migliardi P, Bacon B, Nighswander M. Family violence prevention programs in immigrant communities: perspectives of immigrant men. J Commun Psychol. 2008;36(7):899–914. https://doi.org/10.1002/jcop.20263 .

Stark L, Sommer M, Davis K, Asghar K, Assazenew Baysa A, Abdela G, Tanner S, Falb K. Disclosure bias for group versus individual reporting of violence amongst conflict-affected adolescent girls in DRC and Ethiopia. PLoS ONE. 2017;12: e0174741.

Sullivan M, Bhuyan R, Senturia K, Shiu-Thornton S, Ciske S. Participatory action research in practice: a case study in addressing domestic violence in nine cultural communities. J Interpers Violence. 2005;20(8):977-95. https://doi.org/10.1177/0886260505277680 .

Tanabe M, Nagujjah Y, Rimal N, Bukania F, Krause S. Intersecting sexual and reproductive health and disability in humanitarian settings: risks, needs, and capacities of refugees with disabilities in Kenya, Nepal, and Uganda. Sex Disabil. 2015;33:411–27.

Thompson JL. Exploring gender and culture with Khmer refugee women: reflections on participatory feminist research. Adv Nurs Sci. 1991;13:30–48.

Article   CAS   Google Scholar  

Vloeberghs E, van der Kwaak A, Knipscheer J, van den Muijsenbergh M. Coping and chronic psychosocial consequences of female genital mutilation in the Netherlands. Ethn Health. 2012;17(6):677-95. https://doi.org/10.1080/13557858.2013.771148 .

Weber S. Participatory visual research with displaced persons: “listening” to post-conflict experiences through the visual. J Refug Stud. 2019;32:417–35.

Weber S. Participatory visual research with displaced persons: ‘listening’ to post-conflict experiences through the visual. J Refug Stud. 2019;32:417–35.

Whiting-Collins L, Tawk M, Karp C, Robinson WC, Metzler J. Fostering protective assets among syrian refugee girls who experience child marriage: findings from a formative program evaluation. J Immigr Minor Heal. 2022;24:614–25.

Mulé NJ. Mental health issues and needs of LGBTQ+ asylum seekers, refugee claimants and refugees in Toronto, Canada. Psychol Sex. 2022;13:1168–78.

Cornwall A, Brock K. What do buzzwords do for development policy? A critical look at ‘Participation’, ‘Empowerment’ and ‘Poverty Reduction.’ Third World Q. 2005;26:1043–60.

Sukarieh M, Tannock S. Subcontracting academia: alienation, exploitation and disillusionment in the UK overseas Syrian refugee research industry. Antipode. 2019;51:664–80.

Zreik T, El Masri R, Chaar S, Ali R, Meksassi B, Elias J, Lokot M. Collaborative coding in multi-national teams: benefits, challenges and experiences promoting equitable research. Int J Qual Methods. 2022;21:160940692211394.

Smith J. Overcoming the ‘tyranny of the urgent’: integrating gender into disease outbreak preparedness and response. Gend Dev. 2019;27:355–69.

Download references

This research was funded by the Wellcome Trust’s Institutional Strategic Support Fund, Grant Number: 204928/Z/16/Z.

Author information

Authors and affiliations.

London School of Hygiene and Tropical Medicine, London, UK

Michelle Lokot, Erin Hartman & Iram Hashmi

You can also search for this author in PubMed   Google Scholar

Contributions

ML designed the study and conducted the searches for literature. All authors jointly conducted title/abstract screening, full-text review and data extraction. All authors drafted the manuscript and all were involved in revising the transcript. All authors approved the final draft of the manuscript.

Corresponding author

Correspondence to Michelle Lokot .

Ethics declarations

Ethics approval and consent to participate.

Ethical approvals were not needed for this scoping review.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

“The original online version of this article has been revised”: the caption of table 1 is corrected.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Lokot, M., Hartman, E. & Hashmi, I. Participatory approaches and methods in gender equality and gender-based violence research with refugees and internally displaced populations: a scoping review. Confl Health 17 , 58 (2023). https://doi.org/10.1186/s13031-023-00554-5

Download citation

Received : 04 May 2023

Accepted : 22 November 2023

Published : 08 December 2023

DOI : https://doi.org/10.1186/s13031-023-00554-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Internally displaced person
• Participation
• Gender-based violence

Conflict and Health

ISSN: 1752-1505

• Submission enquiries: [email protected]

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

• Live In The D
• Newsletters

Why Better Made is suing several Michigan marijuana companies

Lawsuit could have impact on cannabis product branding.

Cassidy Johncox , Senior News Editor

DETROIT – Better Made is suing companies behind several Michigan marijuana dispensaries that they say are selling products with an altered, and unauthorized, version of the snack food company’s logo.

The popular Michigan potato chip company filed a lawsuit Wednesday, March 27 against nearly 20 companies and the marijuana dispensaries they operate in the state. Better Made says the businesses are selling a cannabis product labeled “Better Smoke,” whose logo is a spin off their own.

Better Made claims the use of their distinct logo design by Better Smoke is damaging to their brand, since customers may wrongly assume the cannabis products are affiliated with the food company. The defendants named in the lawsuit are being accused of federal trademark infringement and dilution, among other things.

Included among the defendants are several companies and their dispensaries from across all of Michigan -- from Traverse City, to Grand Rapids, to Adrian, and throughout Metro Detroit. Better Made is asking the court to make these companies give all of their profits from Better Smoke products to the actual snack foods company.

It’s clear that the Better Smoke logo is a play on Better Made’s iconic design, which has been around since 1930. But trademark parodies are actually pretty common among cannabis products -- at least at Michigan dispensaries.

Here’s a look at what trademark parodies are, and why Better Made may have a case.

Why is Better Smoke using Better Made’s logo?

The makers of Better Smoke aren’t directly using Better Made’s logo. They are, however, using a nearly identical design, but with different words that indicate they are selling a different product under a different name.

This is what’s known as a trademark parody: When someone reuses an existing brand or logo in a way that is humorous, satirical, or critical of the original brand. This practice is largely done with logos or bands that are widely known.

Parodies of other creations like songs and films may be more common, or at least more familiar. But, for the most part, they’re legal, too -- so long as the parodies are an exaggerated, or creative, or comedic take on someone’s work that’s meant to critique or comment on that work.

It appears that Better Smoke’s packaging and name are meant to be a humorous take on Better Made’s brand, but that depends on who you ask. At some point, Better Smoke’s logo featured a marijuana plant leaf rather than the Better Made’s specific shape, though it’s unclear when that branding was released.

There is a fine legal line when it comes to parodies -- especially trademark parodies.

Why Better Made is claiming trademark infringement

A trademark is known as a “source identifier,” which is a word, phrase, logo, slogan, general appearance or another item or concept that literally identifies the source of a good or service. Trademarks are meant to protect and verify a product’s source for consumers -- such as Nike’s swoosh or Adidas’ three stripes. When a shopper sees those designs, they know what brand the product is associated with.

Those designs, or whatever constitutes the trademark, are also referred to as “marks.”

In this case, Better Made argues that Better Smoke’s logo use is actually confusing to consumers, which is considered a viable claim for trademark infringement. Better Made claims the parody logo can cause buyers to be unsure about where the product originates.

Because the logos are so similar, Better Made argues that consumers might incorrectly believe the cannabis products and the food company’s products are affiliated with one another.

The defendants’ actions “deprive Better Made of the ability to control consumer perception of its goods and services offered under its Better Made marks, placing the valuable goodwill and reputation of Better Made into the hands of defendants,” the lawsuit reads.

The snack food company also argues Better Smoke’s version and utilization of a nearly identical design “dilutes” Better Made’s distinctiveness as a brand. That is considered a viable argument for federal trademark dilution.

In the lawsuit, Better Made specifically calls out defendant House Brands Distro, a cannabis distribution company in Michigan that makes the Better Smoke products. The lawsuit alleges that House Brands Distro “willfully violated Better Made’s rights” by manufacturing, distributing and selling a product using a logo “design that is an unmistakable reproduction of the Better Made design.”

Better Made claims its branding has become famous over the last nearly 100 years, and that the defendants have used it without authorization. Because those behind Better Smoke are making money from the products, they may be given less leeway than if their designs were for noncommercial use, experts say.

What Better Made wants out of lawsuit

The snack foods company is asking the court to take several actions in response to the lawsuit.

Better Made wants the defendants barred from using their design without authorization. The court is being asked to force the marijuana dispensaries to destroy all product labels, literature, online content, and advertising materials that feature the Better Smoke logo called into question.

Most notably, though, Better Made wants the companies to hand over all of the profits that came from selling Better Smoke products.

The plaintiff wants all defendants to “account for and pay to Better Made all profits derived by defendants resulting from their use of the Better Made marks,” the lawsuit reads.

Requests for comment were left with most of the listed defendants. No such requests had been returned as of Monday morning.

Cannabis companies use trademark parodies a lot

Those behind Better Smoke aren’t the only ones manufacturing or selling cannabis products with brands that look ... familiar.

Whether you shop for cannabis products online or in person in Michigan, you’re bound to witness a plethora of trademark parodies. The packages and brand names of several types of marijuana flower, cannabis edibles, and THC vapes are meant to evoke familiar brands -- especially those related to candy.

The company Clout King is well known for naming and designing their products around existing, high-profile candy brands. They sell a strain of marijuana named Air Headz, similar to Airheads candy. They also sell multiple strains named Terdz, a play on the candy Nerds.

Manufacturer Monster Xtracts is known for its microdose-sized cannabis edibles PoTartz, which are similar to SweeTarts, and Pot Dots, which resemble M&Ms.

VBJG Mt. Clemens LLC manufactures marijuana-infused apple cider drinks labeled Armada Cannabis Co. The design of the cans strongly resemble the design of hard cider drinks produced by Blake’s, an orchard and cider mill located in Armada, Michigan.

It’s unknown how or if Better Made’s lawsuit will affect the packaging for cannabis companies across Michigan in the future.

The U.S. Supreme Court heard a similar case last year , when Jack Daniel’s brought counter claims against a dog toy company for trademark infringement. VIP Products was selling a dog toy called Bad Spaniels that resembled the popular whiskey brand’s bottle. The toy explicitly stated that it was not affiliated with Jack Daniel’s.

The Supreme Court did not decide whether VIP Products committed trademark infringement. The high court instead o verturned part of a lower court’s ruling that largely sided with VIP product, telling the lower court to review the case again.

In its published, unanimous decision, the Supreme Court ordered the lower court to focus on the question of whether the dog toy’s branding was likely to cause confusion for consumers. Trademark parody for a commercial product is specifically subject to something called the “likelihood-of-confusion analysis” under the Lanham Act, which is the U.S.’ core federal trademark law.

---> More on that case: Ruff day in court: Supreme Court sides with Jack Daniel’s in dispute with makers of dog toy

Experts say the Supreme Court’s ruling in that case could strengthen the rules for trademark owners, making it more difficult for parody trademarks to exist.

The Supreme Court case is known as Jack Daniel’s Properties, Inc. v. VIP Products LLC, 22-148.

Read: Lawsuit filed by Better Made

Below is the entire complaint filed by Better Made on March 27, 2024.

Copyright 2024 by WDIV ClickOnDetroit - All rights reserved.

About the Author

Cassidy johncox.

Cassidy Johncox is a senior digital news editor covering stories across the spectrum, with a special focus on politics and community issues.

Click here to take a moment and familiarize yourself with our Community Guidelines.

Recommended Videos

CS 224C Logistics

Stanford / spring 2024.

• Project (55%): Divided into proposal (10%), midway report (15%), final submission (25%), project pitch (2.5%), and poster presentation (2.5%).
• Presentation (10%): Engage with your subject matter and peers through presentations.
• Homework (20%): Four assignments, each worth 5%, designed to reinforce core material.
• Participation (2%): Involvement in discussions, presentations, and project teamwork.
• Reading Responses (13.5%): 9 responses, each contributing 1.5%, to foster engagement and critical thinking.

This is the system we will use at the end of the quarter to map numerical final grades to letter grades. No curve is applied, and there are no other factors shaping the mapping from weighted averages ( details here ) to letter grades.

Reading Responses

A short (2-paragraph) written response to each reading to be posted the day before class.

The response should not focus on summarizing the papers, but instead raise questions that would be appropriate for discussion, or propose ideas to think about. The goal is to get you to think critically about the research that a paper presents and why that research is important.

• why the work is/isn't novel/important in terms of data/method/topic
• critique certain features of the reading
• identify potentially important issues not covered in the reading
• how the paper has changed your opinion or outlook on a topic
• suggest new research questions inspired by the reading, or think about new ways to improve the work
• Check-minus (85%) : Surface-level engagement with the readings, or a repeat of a style of critique that the staff told the class to avoid. Examples of surface-level engagement include: comments about whether the student likes or would use the technology, a summary of the paper rather than a reflections on the ideas, or critiques that engage only obliquely with the paper or indicate that the author didn't fully read it.
• Check (100%) : Effective engagement with the readings. Example responses involving check grades often indicate that they understand the main ideas of the papers, and the reflections are reasonably nontrivial observations worth discussing.
• Check-plus (105%) : Excellent engagement with the readings. Check-plus grades are reserved for rare instances where a reading response really hits on an interesting, unique, and insightful point of view worth sharing.

Role-Playing Paper Reading & Discussion

More information and details: Role-Playing Seminar

You are the instructor who needs to provide a comprehensive overview of the paper. What did it study? What’s the novelty? What’s the methods and results?

The papers have not been published yet and is currently submitted to a top conference where you’ve been assigned as a peer reviewer. Complete a full review of the paper answering all prompts “strengths, weaknesses, questions”

You’re a researcher who is working on a new project in this area. Propose an imaginary follow-up project not just based on the current but only possible due to the existence and success of the current paper.

You work at a company or organization developing an application or product of your choice (that has not already been suggested in a prior session). Bring a convincing pitch for why you should be paid to implement the method in the paper, and discuss at least one positive and negative impact of this application.

Identify how this paper self-assesses its (likely positive) impact on the world. Have any additional positive social impacts left out? What are possible negative social impacts that were overlooked or omitted?

You’re a hacker who needs a demo of this paper ASAP. Implement a small part or simplified version of the paper on a small dataset or toy problem. Prepare to share the core code of the algorithm to the class and demo your implementation. Do not simply download and run an existing implementation – though you are welcome to use (and give credit to) an existing implementation for “backbone” code.

Presentation

• The group works together to deliver a presentation
• Work together to well cover the material
• Make easy-to-understand slides
• Be prepared for Q&As
• Please send your draft slides to the instructor/TA 1 day before the class

Grading: 5 points (group), 5 points (each role)

Class Participation

• Class participation grades are based on whether you productively contribute to the classroom discussions in lecture. This grade also captures your contributions in the presentation and leading discussion section. Finally, participation grades will also take into account how actively you contribute to the success of your project alongside your teammates.
• Please check out the details of the course project here .
• Please discuss your project idea with instructor/TA early on in the course.
• Literature Review
• Experiment Protocol
• Final Paper
• Literature Review: Due Monday, Jan 30, 5:00PM Thursday, Jan 26, 5:00pm
• Experiment Protocol: Due Thursday, Feb 16, 5:00pm
• Final Paper: Due Tuesday, March 21, 5:00pm

Academic Honesty

Please familiarize yourself with Stanford's honor code . We will adhere to it and follow through on its penalty guidelines.

• It is expected that you accurately represent your own work and the work of others in this class. Ideas should be your own. Any use of tools (e.g., ChatGPT) should be limited to clarity and credited appropriately in your submission.
• Each student will have a total of 6 free late (calendar) days applicable to any assignment (including the lit review and project milestone) except the final project paper . Final project papers cannot be turned in late under any circumstances.
• Free late days can be used at any time, no questions asked. Each 24 hours or part thereof that a homework is late uses up one full late day. Once these late days are exhausted, any homework or quiz turned in late will be penalized 10% per late day.
• If a group's assignment is late n days, then each group member is charged n late days.
• Late days are never transferrable between students, even students in the same group.
• Late days do not apply to the final submission of the course project/
• Reading responses do not have late days (since they are a prerequisite for coming to class, so responses posted after 5pm the day before class will count as not turned in.)

Policy on Submitting Related Final Projects to Multiple Classes

On the one hand, we want to encourage you to pursue unified interdisciplinary projects that weave together themes from multiple classes. On the other hand, we need to ensure that final projects for this course are original and involve a substantial new effort.

To try to meet both these demands, we are adopting the following policy on joint submission: if your final project for this course is related to your final project for another course, you are required to submit both projects to us by our final project due date. If we decide that the projects are too similar, your project will receive a failing grade. To avoid this extreme outcome, we strongly encourage you to stay in close communication with us if your project is related to another you are submitting for credit, so that there are no unhappy surprises at the end of the term. Since there is no single objective standard for what counts as "different enough", it is better to play it safe by talking with us.

Fundamentally, we are saying that combining projects is not a shortcut. In a sense, we are in the same position as professional conferences and journals, which also need to watch out for multiple submissions. You might have a look at the ACL/NAACL policy , which strives to ensure that any two papers submitted to those conferences make substantially different contributions – our goal here as well.

It is very important to us that all assignments are properly graded. The teaching staff works extremely hard to grade fairly and to turn around assignments quickly. We know what you work hard, and we respect that. Occasionally, mistakes happen, and it's important to us to correct them. If you believe there is an error in your assignment grading, please submit an explanation in writing to the staff within seven days of receiving the grade. We will regrade the entire assignment to ensure quality.

• No regrade requests will be accepted orally, and no regrade requests will be accepted more than seven days after receipt of the assignment. Regrade requests must be respectful; we will not consider any regrade requests containing disrespectful language.

Names and Pronouns

• Use the names and pronouns (e.g., they/them, she/her, he/him, just a name, or something else) indicated by your classmates for themselves. If you don’t want to share a set of pronouns for yourself, that is perfectly acceptable, too. If your name or pronouns change during the course, we invite you to share this with us and/or other students, so we may talk with you and refer to your ideas in discussion as you would wish.