gray literature in research

Public Health: Defining Grey Literature

• Getting Started
• Public Health Books in the Library
• Top Public Health Journals
• Health Data and Statistics
• Populations & Communities
• Vital Statistics
• More Public Health Resources
• Software in the Library
• How to Search the Literature for Evidence
• How to Search Scholarly Databases
• How to Search for Textbooks & Other References
• Defining Grey Literature
• Key Sources & Databases for Grey Literature
• Evaluation Report Resources
• Evaluating Evidence-Based Interventions
• General Guidance on Evaluating Information
• Citation Styles & Management
• Get More Help

Guidance Documents for Searching Gray Literature

• Grey Matters: A Practical Search Tool for Evidence-Based Medicine This online manual provides a thorough list of sources for gray literature in medicine and a helpful checklist to help systematize your process.
• Googling the Greys: Tips for Searching Beyond Health Databases and Turning Information into Insights This PDF presentation by Sarah Bonato, a health science librarian at the Centre for Addiction and Mental Health in Toronto, ON, provides an excellent guide to searching Google effectively to find gray literature.
• Duke University Medical Center Guide to Resource for Searching the Gray Literature A more thorough guide to gray literature, including resources for trial registries, pharmacological studies, conference abstracts, government document and more.

What Grey Literature Is -- and How to Search It

What is grey literature?   Grey (or gray) literature is defined by the  Cochrane Handbook for Systematic Reviews of Interventions  as "...literature that is not formally published in sources such as books or journal articles." This can include information such as government reports, conference proceedings, graduate dissertations, unpublished clinical trials, and much more.  

Why search the grey literature if it hasn't been peer-reviewed?   There is a strong bias in scientific publishing toward publishing studies that show some sort of significant effect.  Meanwhile, many studies and trials that show no effect end up going unpublished.  But knowing that an intervention had no effect is just as important as knowing that it did have an effect when it comes to making decisions for practice and policy-making.  Thus, the grey literature can be critical.

How do I search the grey literature if it's unpublished and not in the scholarly databases?    Its true--finding grey literature and searching it systematically is challenging.  But there are a few approaches that you can take to add some structure to your search of this type of information:

• Identify and record  the sources you will search, including some indication of search terms used if appropriate.  The sources you search will be informed by your research question and where you expect to find information related to your question.
• Try searching  databases that specialize in grey literature  like  OpenGrey .  See the box below for more information.
• Conference proceedings :  Identify professional organizations that have conferences at which researchers might be presenting work related to your topic.  Search those conference proceedings on the organization's website or by contacting organizational boards for access to past proceedings that may not be online.
• Theses and dissertations :  There are a number of databases dedicated to theses and dissertations, which you can search using your search terms.  See the box below for links to these resources. 
• Identify government agencies, and international and non-governmental organizations , that might publish technical papers and reports on your topic.  Search their websites or any online libraries that they may provide.  For example, the  WHO  has a number of searchable online collections and the  World Bank  now makes all of their publications openly available online.
• Contact known researchers  in the field to determine if there are any ongoing or unpublished studies that s/he may be aware of.
• << Previous: Searching Grey Literature
• Next: Key Sources & Databases for Grey Literature >>
• Last Updated: Jan 24, 2024 1:52 PM
• URL: https://guides.library.cornell.edu/public-health

• Library Catalogue

Grey literature: What it is & how to find it

On this page, what is grey literature, where to search for grey literature, using google to search for grey literature, google custom searches, google scholar, other sources.

• By geographic coverage
• Statistics and data sources

Further resources and strategies for searching for grey literature

Grey literature is information produced outside of traditional publishing and distribution channels, and can include reports, policy literature, working papers, newsletters, government documents, speeches, white papers, urban plans, and so on.

This information is often produced by organizations "on the ground" (such as government and inter-governmental agencies, non-governmental organisations, and industry) to store information and report on activities, either for their own use or wider sharing and distribution, and without the delays and restrictions of commercial and academic publishing. For that reason, grey literature can be more current than literature in scholarly journals.

However, because grey literature (usually) does not go through a peer review process, the quality can vary a great deal. Be sure to critically evaluate your source .

For an extensive list of the types of documents that might be considered grey literature, see Document types in grey literature (from GreyNet International ).

To search across the websites of governments and organizations, use the resources below.

Another way to find grey literature is to identify organizations that might be publishing this type of information on topics that you're interested in and then to search their websites — paying close attention to website sections with names like "Documents", "Reports", and "Library". Consider government agencies, non-profits, professional associations, research institutes, and other organisations, based on the research you have already done. 

Use Google's site limits to only search a government or institution's site or top-level domain, for instance use  site:gov.bc.ca to limit your search to British Columbia government sites. This technique paired with keywords in Google is a quick way to locate grey literature. Example search: site:gov.bc.ca housing affordable

For additional advanced searching techniques see  Search tips for Google and Google Scholar , our quick overview of basic and more advanced techniques.

Type your search terms into these pre-built searches  to limit results to specific organisation types. Use Search tips for Google and Google Scholar  to create your search strings. 

NGO Search NGO Search is a Google Custom Search that searches across hundreds non-governmental organization (NGO) websites. NGO Search is a project of the International Documents Taskforce (IDTF) and the Government Documents Roundtable (GODORT) of the American Library Association (ALA).  This is a spreadsheet of NGOs included in the project . 

IGO Search International governmental organizations (IGOs) are organizations made up of more than one national government—examples include NATO (North Atlantic Treaty Organization) and the WHO (World Heath Organization). The governments are the members. IGO Search is a Google Custom Search that searches across IGO websites. It is a project of the International Documents Taskforce (IDTF) and the Government Documents Roundtable (GODORT) of the American Library Association (ALA). Here is the growing spreadsheet of IGO urls  included in the search. 

There is some grey literature in Google Scholar — mainly theses and dissertations in institutional repositories. Be sure to access  Google Scholar via the Library's website  to ensure you see the "Where can I get this?" links that will take you to the full text SFU offers. 

Use  Search tips for Google and Google Scholar  to create your search strings. 

Policy Commons   This database brings together grey literature -- reports, working papers, policy briefs, data sources, and media -- from NGOS, think tanks, and research centers around the world.  

 By geographic coverage

Canada Commons Includes some full text public policy documents from Canadian institutes, think-tanks, and research groups.

CHODARR (Community Health Online Digital Archive & Research Resource) An archived virtual library of community-based and government publications related to health and social welfare policy, including grey literature, with emphasis on Vancouver-area resources and organisations. Note: This source is no longer being updated. You can browse the individual collections, or use Google's site search function to search across it. Example search in Google:  site:chodarr.org AIDS 

International

OpenGrey  (Europe) A search portal for European grey literature. It covers Science, Technology, Biomedical Science, Economics, Social Science and the Humanities.

 Statistics and data sources

Statistical resources: Data & statistics information (SFU Library) Use these resources for processed data drawn from national agencies, intergovernmental organizations, and other official sources of statistics.

Microdata resources: Data & statistics information (SFU Library) Use microdata when you need to do analysis, or generate visualizations such as tables and maps. 

For more about the value of grey literature and further strategies for finding it, you can also search the Library Catalogue for terms such as  grey literature methodology .

Grey literature: An important resource in systematic reviews

Affiliation.

• 1 Nuffield Department of Primary Care Health Sciences, Centre for Evidence-Based Medicine, Department for Continuing Education, The University of Oxford, Oxford, UK.
• PMID: 29266844
• DOI: 10.1111/jebm.12265

Systematic reviews aid the analysis and dissemination of evidence, using rigorous and transparent methods to generate empirically attained answers to focused research questions. Identifying all evidence relevant to the research questions is an essential component, and challenge, of systematic reviews. Grey literature, or evidence not published in commercial publications, can make important contributions to a systematic review. Grey literature can include academic papers, including theses and dissertations, research and committee reports, government reports, conference papers, and ongoing research, among others. It may provide data not found within commercially published literature, providing an important forum for disseminating studies with null or negative results that might not otherwise be disseminated. Grey literature may thusly reduce publication bias, increase reviews' comprehensiveness and timeliness and foster a balanced picture of available evidence. Grey literature's diverse formats and audiences can present a significant challenge in a systematic search for evidence. However, the benefits of including grey literature may far outweigh the cost in time and resource needed to search for it, and it is important for it to be included in a systematic review or review of evidence. A carefully thought out grey literature search strategy may be an invaluable component of a systematic review. This narrative review provides guidance about the benefits of including grey literature in a systematic review, and sources for searching through grey literature. An illustrative example of a search for evidence within grey literature sources is presented to highlight the potential contributions of such a search to a systematic review. Benefits and challenges of grey literature search methods are discussed, and recommendations made.

Keywords: evidence; grey literature; publication bias; search; systematic review.

© 2017 Chinese Cochrane Center, West China Hospital of Sichuan University and John Wiley & Sons Australia, Ltd.

An official website of the United States government

Here's how you know

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Effective March 14, 2024, HHS Virtual Library customers will have a streamlined login , with a common URL for all HHS customers using a PIV card. IHS and other HHS customers without a PIV card will continue to login using temporary credentials provided by the NIH Library. Further guidance is available here .

Literature Search: Databases and Gray Literature

The literature search.

• A systematic review search includes a search of databases, gray literature, personal communications, and a handsearch of high impact journals in the related field.  See our list of recommended databases and gray literature sources on this page.
• a comprehensive literature search can not be dependent on a single database, nor on bibliographic databases only.
• inclusion of multiple databases helps avoid publication bias (georaphic bias or bias against publication of negative results).
• The Cochrane Collaboration recommends PubMed, Embase and the Cochrane Central Register of Controlled Trials (CENTRAL) at a minimum.     
• NOTE:  The Cochrane Collaboration and the IOM recommend that the literature search be conducted by librarians or persons with extensive literature search experience. Please contact the NIH Librarians for assistance with the literature search component of your systematic review. 

Cochrane Library

A collection of six databases that contain different types of high-quality, independent evidence to inform healthcare decision-making. Search the Cochrane Central Register of Controlled Trials here.

European database of biomedical and pharmacologic literature.

PubMed comprises more than 21 million citations for biomedical literature from MEDLINE, life science journals, and online books.

Largest abstract and citation database of peer-reviewed literature and quality web sources. Contains conference papers.

Web of Science

World's leading citation databases. Covers over 12,000 of the highest impact journals worldwide, including Open Access journals and over 150,000 conference proceedings. Coverage in the sciences, social sciences, arts, and humanities, with coverage to 1900.

Subject Specific Databases

APA PsycINFO

Over 4.5 million abstracts of peer-reviewed literature in the behavioral and social sciences. Includes conference papers, book chapters, psychological tests, scales and measurement tools.

CINAHL Plus

Comprehensive journal index to nursing and allied health literature, includes books, nursing dissertations, conference proceedings, practice standards and book chapters.

Latin American and Caribbean health sciences literature database

Gray Literature

• Gray Literature  is the term for information that falls outside the mainstream of published journal and mongraph literature, not controlled by commercial publishers
• hard to find studies, reports, or dissertations
• conference abstracts or papers
• governmental or private sector research
• clinical trials - ongoing or unpublished
• experts and researchers in the field     
• Library catalogs
• Professional association websites
• Google Scholar  - Search scholarly literature across many disciplines and sources, including theses, books, abstracts and articles.
• Dissertation Abstracts - dissertation and theses database - NIH Library biomedical librarians can access and search for you.
• NTIS  - central resource for government-funded scientific, technical, engineering, and business related information.
• AHRQ  - agency for healthcare research and quality
• Open Grey  - system for information on grey literature in Europe. Open access to 700,000 references to the grey literature.
• World Health Organization  - providing leadership on global health matters, shaping the health research agenda, setting norms and standards, articulating evidence-based policy options, providing technical support to countries and monitoring and assessing health trends.
• New York Academy of Medicine Grey Literature Report  - a bimonthly publication of The New York Academy of Medicine (NYAM) alerting readers to new gray literature publications in health services research and selected public health topics. NOTE: Discontinued as of Jan 2017, but resources are still accessible.
• Gray Source Index
• OpenDOAR - directory of academic repositories
• International Clinical Trials Registery Platform  - from the World Health Organization
• Australian New Zealand Clinical Trials Registry
• Brazilian Clinical Trials Registry
• Chinese Clinical Trial Registry - 
• ClinicalTrials.gov   - U.S.  and international federally and privately supported clinical trials registry and results database
• Clinical Trials Registry  - India
• EU clinical Trials Register
• Japan Primary Registries Network  
• Pan African Clinical Trials Registry

• Langson Library
• Science Library
• Grunigen Medical Library
• Law Library
• Connect From Off-Campus
• Accessibility
• Gateway Study Center

Email this link

Systematic reviews & evidence synthesis methods.

• Schedule a Consultation / Meet our Team
• What is Evidence Synthesis?
• Types of Evidence Synthesis
• Evidence Synthesis Across Disciplines
• Finding and Appraising Existing Systematic Reviews
• 0. Develop a Protocol
• 1. Draft your Research Question
• 2. Select Databases
• 3. Select Grey Literature Sources
• 4. Write a Search Strategy
• 5. Register a Protocol
• 6. Translate Search Strategies
• 7. Citation Management
• 8. Article Screening
• 9. Risk of Bias Assessment
• 10. Data Extraction
• 11. Synthesize, Map, or Describe the Results
• Open Access Evidence Synthesis Resources

Grey Literature

Grey literature is literature produced by individuals or organizations outside of commercial and/or academic publishers. This can include information such as government reports, conference proceedings, graduate dissertations, unpublished clinical trials, and much more. The sources you select will be informed by your research question and field of study, but should likely include, at a minimum, theses and dissertations.

Why Search the Grey Literature?

The intent of an evidence synthesis is to synthesize all available evidence that is applicable to your research question. There is a strong bias in scientific publishing toward publishing studies that show some sort of significant effect. Meanwhile, many studies and trials that show no effect end up going unpublished. But knowing that an intervention had no effect is just as important as knowing that it did have an effect when it comes to making decisions for practice and policy-making. While not peer-reviewed, grey literature represents a valuable body of information that is critical to consider when synthesizing and evaluating all available evidence.

Searching the Grey Literature

Finding grey literature and searching it systematically is challenging. But there are a few approaches that you can take to add some structure to your search of this type of information:

• Refer to grey literature sources used for related evidence syntheses. Refer to both published evidence syntheses and registered protocols.
• Ask experts in the field for relevant grey literature sources. If you are an expert, include important grey literature sources, and ask colleagues for their recommendations.
• Search databases that specialize in grey literature: See the "Grey Literature Sources" box at the bottom of this page for more information.
• Search for theses and dissertations: There are a number of databases dedicated to theses and dissertations, which you can search using your search terms. See the "Grey Literature Sources" box at the bottom of this page for links to these resources.
• Search clinical trials: There may be clinical trials being conducted that are relevant to your research question, but that haven't been published yet or never were published. See the "Grey Literature Sources" box at the bottom of this page for links to these resources.
• Identify government agencies and international and non-governmental organizations that might publish technical papers and reports on your topic. Search their websites or any online libraries that they may provide. See the "Grey Literature Sources" box at the bottom of this page for links to some examples.
• Search conference proceedings and newsletters: Identify professional organizations that have and/or conferences at which researchers might be presenting work related to your topic. Search those conference proceedings or newsletters on the organization's website or by contacting organizational boards for access to past proceedings that may not be online. See the "Grey Literature Sources" box at the bottom of this page for some examples.
• Contact known researchers in the field to determine if there are any ongoing or unpublished studies that s/he may be aware of.
• Search professional and trade magazines. Professional magazines contain literature that is written by professionals in the field for other professionals in the field, but that may not be about research. Trade magazines contain advertisements and news very specific to a topic or industry.

How to Manage the Grey Literature Search

• Identify and record the sources you will search. The sources you search will be informed by your research question and where you expect to find information related to your question.
• Document where you are searching and your search strategies, including document resource name, URL, search terms, and date searched.
• Collect citation information as you go.
• Adhere to your established inclusion and exclusion criteria when selecting sources.

See below for guidance documents specific to grey literature searching.

Grey Literature Sources

• Grey Literature Databases
• Theses and Dissertations
• Clinical Trials
• Public Policy
• Conference Proceedings
• Preprint Repositories
• Other Resources
• NY Academy of Medicine Grey Literature Report This report is a bimonthly publication of The New York Academy of Medicine (NYAM) alerting readers to new grey literature publications in health services research and selected public health topics. The database platform is keyword searchable and serves as an archive for the cataloged reports.
• WHO Library Database The WHO library database includes governing documents, reports and technical documentation.
• Global Think Tanks A database of reports, papers, and other resources published by NGOs, think tanks, IGOs and research centers from across the world. It also includes a directory of thousands of organizations involved in public and social policy in most subject area. A module in the Policy Commons database.
• MedNar MedNar searches across more than 60 medical research sources, including commercial databases, medical societies, NIH resources, and other government resources.
• Global Index Medicus The Global Index Medicus (GIM) provides worldwide access to biomedical and public health literature produced by and within low-middle income countries. The main objective is to increase the visibility and usability of this important set of resources. The material is collated and aggregated by WHO Regional Office Libraries on a central search platform allowing retrieval of bibliographical and full text information.
• National Grey Literature Collection - UK Funded by Health Education England this holds the catalogue for the hard copy grey literature collection of the former North Grey Literature Service. It is collecting an increasing range of electronic Grey Literature.

• Networked Digital Library of Theses and Dissertations (NDLT) Free international resource for theses and dissertations.
• Center for Research Libraries A resource for institutions outside of the U.S. and Canada.
• EThOS Electronic Theses Online Service contains doctoral theses from candidates in the UK.
• Open Access Theses and Dissertations Resource for finding open access graduate theses and dissertations published around the world. Metadata (information about the theses) comes from over 1100 colleges, universities, and research institutions. OATD currently indexes over 4 million theses and dissertations.
• DART European - Digital Access to Research Theses Europe The aim of DART is to provide researchers with a single European Portal for the discovery of Electronic Theses and Dissertations (ETDs). Note that only theses available in electronic form are included.
• Australia New Zealand Clinical Trials Registry Online register of clinical trials being undertaken in Australia, New Zealand and elsewhere. The ANZCTR includes trials from the full spectrum of therapeutic areas of pharmaceuticals, surgical procedures, preventive measures, lifestyle, devices, treatment and rehabilitation strategies, and complementary therapies.
• ClinicalTrials.gov The US registry for clinical trials. Includes new, ongoing, and completed human clinical trials both in the US and countries around the world.
• Cochrane Central Register of Controlled Trials A highly concentrated source of reports of randomized and quasi-randomized controlled trials. The majority of CENTRAL records are taken from bibliographic databases (mainly MEDLINE and Embase), but records are also derived from other published and unpublished sources.
• EU Clinical Trials Register The European Union Clinical Trials Register allows you to search for protocol and results information on interventional clinical trials that are conducted in the European Union (EU) and the European Economic Area (EEA) and clinical trials conducted outside the EU / EEA that are linked to European paediatric-medicine development.
• WHO International Clinical Trials Registry Platform ICTRP provides access to a central database containing the trial registration data sets provided by numerous international registries. It also provides links to the full original records.
• Wikipedia Clinical Trial Registries List This Wikipedia entry contains a list of links to individual clinical trial registries by country.
• WHO Institutional Repository for Information Sharing (IRIS) Institutional WHO database of intergovernmental policy documents and technical reports. Can search by IRIS by region (Africa, Americas, Eastern Mediterranean, Europe, South-East Asia, Western Pacific).
• Health Research Web A wiki with descriptors of national health research systems, ethics review committees, local and regional policies and research priorities and more.

This list of conference proceedings is not comprehensive, but are ideas of where to search. We recommend that you search proceedings for the three most relevant conferences for your evidence synthesis.

• OCLC PapersFirst OCLC PapersFirst is an OCLC index of papers presented at conferences worldwide.
• BIOSIS Previews A subcollection of Web of Science, BIOSIS Previews searches across journals, meetings, patents, and books in the life sciences and biomedical fields.

• arXiv arXiv is a free distribution service and an open-access archive for scholarly articles in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics.
• medRxiv medRxiv (pronounced "med-archive") is a free online archive and distribution server for complete but unpublished manuscripts (preprints) in the medical, clinical, and related health sciences.
• PsyArXiv A free preprint service for the psychological sciences. It is maintained by The Society for the Improvement of Psychological Science.
• OSF Preprints A general preprint repository that covers architecture, business, engineering, life sciences, physical sciences and mathematics, arts and humanities, education, law, medicine and health sciences and social and behavioral sciences, among others.
• TechRxiv Hosted by IEEE
• Engineering Archive Created in 2016. Accepts all types of manuscripts, including but not limited to systematic reviews, hypotheses, ‘negative’ results, and data and methods papers.Includes 2 major types of articles: (1) Preprints of articles in parallel to traditional journal submissions. (2) Previously published articles that you wish to make Open Access
• Preprints.org This is a multidiscipline platform providing preprint service that is dedicated to making early versions of research outputs permanently available and citable. We post original research articles and comprehensive reviews, and papers can be updated by authors as long as the updated content has not been published online.
• Grey Matters: A practical search tool for evidence-based medicine This online manual provides a thorough list of sources for grey literature in medicine and a helpful checklist to help systematize your process.
• Searching the grey literature: A handbook for searching reports, working papers, and other unpublished research Searching the Grey Literature is for librarians and information professionals interested in learning more about grey literature. This book will aid with crafting a grey lit search successfully, from start to finish. Many types of librarians will find the content of this book useful, particularly those in health or social science.
• Duke University Medical Center Guide to Resource for Searching the Grey Literature Resources for trial registries, pharmacological studies, conference abstracts, government document and more.
• Gray Literature Resources for Agriculture Evidence Syntheses This is a comprehensive list of agricultural grey literature sources, one of several evidence synthesis worksheets and tools provided to help in non-medical evidence synthesis development and implementation.
• Searching for studies: A guide to information retrieval for Campbell Systematic Reviews This document gives comprehensive guidance in searching for evidence syntheses, and offers a robust list of grey literature sources for social science disciplines.
• Finding Grey Literature Evidence and Assessing for Outcome and Analysis Reporting Biases When Comparing Medical Interventions: AHRQ and the Effective Health Care Program An introduction to clinical resources and strategies in approaching systematic reviews.

Video: Grey literature (3:10 minutes)

• << Previous: 2. Select Databases
• Next: 4. Write a Search Strategy >>
• Last Updated: Feb 14, 2024 4:20 PM
• URL: https://guides.lib.uci.edu/evidence-synthesis

Off-campus? Please use the Software VPN and choose the group UCIFull to access licensed content. For more information, please Click here

Software VPN is not available for guests, so they may not have access to some content when connecting from off-campus.

• Subject guides
• Grey Literature
• What is grey literature

Grey Literature: What is grey literature

• Finding grey literature
• Evaluating grey literature
• Grey Sources

Other useful guides

• Citing and referencing
• Literature reviews

​Grey literature is "Information produced on all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body."

CGL Luxembourg definition , 1997-expanded in New York, 2004

Grey literature is published informally or non-commercially, or remains unpublished. It can appear in many forms, including government reports, statistics, patents, conference papers and even non-written resources such as posters and infographics.

Grey literature usually has not been peer reviewed, but may still be good, reliable information. It can thus be invaluable for your research. It is produced from a variety of sources, and is usually not indexed or organised, often making it difficult to locate.

What is grey literature?

A short video explaining grey literature from Western Universities library

Examples of grey literature

For a comprehensive listing of grey literature types view the greynet.org table .

Common grey literature producers

Why use grey literature.

Grey literature is used in research because:

• It will often be more current than traditionally published sources, with a better coverage of emergent research areas. 
• A literature search that accesses only “black literature” will likely miss information vital to research, especially in health sciences fields.
• It can be a better source of information on policies and programs.
• Using grey literature may help minimise publication bias in systematic reviews.
• Grey literature is often a good source of raw data and data sets to which you wouldn’t otherwise have access.
• << Previous: Home
• Next: Finding grey literature >>

Systematic Reviews and Meta-Analyses: Grey Literature

• Get Started
• Exploratory Search
• Where to Search
• How to Search

Grey Literature

• What about errata and retractions?
• Eligibility Screening
• Critical Appraisal
• Data Extraction
• Synthesis & Discussion
• Assess Certainty
• Share & Archive

This page provides more information about grey literature.

Conference Proceedings

Grey Literature is a broad term that varies across discipline. In very general terms, grey literature is anything other than peer-reviewed, empirical research. Examples of grey literature include: unpublished research, government publications, theses and dissertations, statistics and datasets, news articles, trade and popular magazines, blogs, podcasts, social media posts (e.g., tweets), slide decks, or factsheets. Including grey literature in your systematic review can help mitigate publication bias.

According to  GreyNet, 2013 :

Grey Literature...deals with the production, distribution, and access to multiple document types produced on all levels of government, academics, business, and organization in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body.

According to Paez, 2017 :

Publication bias refers to the propensity for only studies reporting positive findings to be published, and may skew the results of the meta-analysis and systematic review. As gray literature may describe neutral or negative results, including it with commercially published research may provide a more balanced understanding of the evidence and a more accurate effect sizes ...excluding unpublished studies may compromise the validity and reliability of meta-analyses and the specificity of systematic reviews.

Should you include grey literature? 

Can a type (or several types) of grey literature  help to answer your research question ? Do you have time for additional searches and processing additional material? If so, you should consider including grey literature in your review!

However, not all questions will benefit from grey literature and you cannot include  all  kinds of grey literature.

Wilson (2009) states:

"Ignoring the grey literature on the basis that it is of inferior methodological quality is empirically and logically invalid . Furthermore, the use of publication status as a proxy measure for methodological quality abdicates the important role of carefully evaluating the robustness of the evidence to others using unknown criteria (i.e., the journal peer-review process)."

Ćurković M, Košec A. Bubble effect: including internet search engines in systematic reviews introduces selection bias and impedes scientific reproducibility. BMC Med Res Methodol. 2018 Nov 13;18(1):130. doi: 10.1186/s12874-018-0599-2. PMID: 30424741; PMCID: PMC6234590.

Godin, K., Stapleton, J., Kirkpatrick, S.I.  et al.   Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada.   Syst Rev   4 , 138 (2015). https://doi.org/10.1186/s13643-015-0125-0

Gusenbauer M, Haddaway NR. Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources . Res Synth Methods. 2020 Mar;11(2):181-217. doi: 10.1002/jrsm.1378. Epub 2020 Jan 28. PMID: 31614060; PMCID: PMC7079055

Mahood, Q., Van Eerd, D., & Irvin, E. (2014). Searching for grey literature for systematic reviews: Challenges and benefits: MAHOOD ET AL . Research Synthesis Methods , 5 (3), 221–234. https://doi.org/10.1002/jrsm.1106

Paez A.  Grey literature: An important resource in systematic reviews . J Evid Based Med. 2017 Dec 21. doi: 10.1111/jebm.12265. Epub ahead of print. PMID: 29266844.

Check out  Appendix I  for an annotated  list of grey lit sources!

Scherer, R.W., Saldanha, I.J.  How should systematic reviewers handle conference abstracts? A view from the trenches .   Syst Rev   8,  264 (2019). https://doi-org.ezproxy.lib.vt.edu/10.1186/s13643-019-1188-0

Wilson, D.B. Missing a critical piece of the pie: simple document search strategies inadequate for systematic reviews .  J Exp Criminol   5,  429–440 (2009). https://doi-org.ezproxy.lib.vt.edu/10.1007/s11292-009-9085-5

Conference proceedings are some of the most common types of grey literature to consider in systematic reviews. Many of the databases you'll search  will  include conference proceedings . But there are databases that contain only conference proceedings such as  ProceedingsFirst , as well as databases containing the conference proceedings for a specific conference organization (which may require affiliation to access). If you plan to include a specific type of grey literature, like conference proceedings, it is not enough to include what you happen to find through your primary search. You should also search purposefully for this type of literature by targeting focused databases, just as you did when searching for peer-reviewed empirical research.

According to  PRISMA-S, Item 4:

Including conference proceedings in a systematic review search helps  minimize bias ...specify the conference names , the dates of conferences included, and the method used to search the proceedings...

Scherer & Saldanha, 2019  offer a more pragmatic approach , suggesting reviewers consider their capacity and corpus:

If available evidence is  sparse or conflicting , it may be worthwhile to include conference abstracts. If results from conference abstracts are included, then it is necessary to make diligent attempts to  contact the authors  of the abstract and  examine study registers  and  published protocols  to obtain further and confirmatory information on methods and results.

This is important as, in a Cochrane Network Meta-Analysis,  Hackenbroich, Kranke, Meybohm, & Weibel (2022)  found:

Most conference abstracts on PONV were insufficiently reported regarding review's narrow inclusion criteria and could not be included in NMA. The resource-intensive search and evaluation of abstracts did not substantially extent the full-text evidence base of the review, given the few adequately reported abstracts. Conferences should oblige authors to adhere to CONSORT for abstracts.

• << Previous: How to Search
• Next: What about errata and retractions? >>
• Last Updated: Feb 2, 2024 6:24 PM
• URL: https://guides.lib.vt.edu/SRMA

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008-.

Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet].

Finding grey literature evidence and assessing for outcome and analysis reporting biases when comparing medical interventions: ahrq and the effective health care program.

Howard Balshem , MS, Adrienne Stevens , MSc, Mohammed Ansari , MD, MMedSc, MPhil, Susan Norris , MD, MPH, MSc, Devan Kansagara , MD, Tatyana Shamliyan , MD, MS, Roger Chou , MD, MPH, Mei Chung , PhD, MPH, David Moher , PhD, and Kay Dickersin , PhD.

Affiliations

Published: November 18, 2013 .

• Reviews of the literature consistently provide evidence of significant reporting biases.
• Reporting bias should be cautiously assumed to exist even if authors cannot determine its direction and magnitude. As such, all included studies must be assessed for reporting bias.
• When studies do not investigate or report outcomes of interest to the review this may be due to a reporting bias.
• Assessment of outcome and analysis reporting bias should be restricted to those outcomes that will be graded for their strength of evidence, for feasibility.
• Reviewers should always search ClinicalTrials.gov and the International Clinical Trials Registry Platform
• Reviewers should routinely search and request clinical study reports from the European Medical Agency, and should search Drugs@FDA for Medical and Statistical Review documents
• Study protocols should be sought during the literature searching processs
• Reviewers should routinely consider searching conference abstracts and proceedings to identify unpublished or unidentified studies and should consult with their Technical Expert Panels for specific conferences to search
• Reviewers should routinely conduct a search of the Cochrane Central Register of Controlled Trials, a source of handsearching results
• Reviewers should avoid the use of English-only filters when searching standard databases
• Searches of grant and non-English databases and contact with authors may be warranted
• The utility of these sources for identifying or minimizing reporting bias associated with observational studies has not yet been evaluated.
• All sources of evidence, with the exception of conference abstracts, should be collated and used for assessing selective outcome and analysis reporting biases. A framework for assessing selective reporting is detailed. If reviewers decide to use the framework for observational studies, certain considerations or adaptations of the framework may need to be made.
• Introduction

“Search for the truth is the noblest occupation of man; its publication is a duty” [Baronne Anne Louise Germaine de Staël-Holstein (1766–1817)]. 1

Systematic reviews attempt to identify, appraise and synthesize the available empirical evidence in order to minimize bias when representing the results of medical interventions and therapies. However, there is a growing recognition that often evidence is difficult to find because of decisions that are made about where, how, and when to publish the results of studies based on the findings of those studies. Notwithstanding, when unpublished data are actually available (for example as a result of legal action), reporting bias associated with suppression of unfavorable results has been fairly easy to detect. 2 , 3 A review by Song, et al. notes that the results of half of all clinical trials are never published. Other findings were that studies with positive or statistically significant effects tend to report greater treatment effect, tend to be published sooner and in higher impact journals than those with negative or nonsignificant effects, and that exclusion of non-English language literature may bias our understanding of treatment effects, particularly in the area of complementary and alternative medicine. 4

Overview of Guidance

Since evidence syntheses depend on the published literature accurately representing what’s known about medical therapies, reporting biases threaten the veracity of what we know. This document provides guidance on steps that authors of systematic reviews can take to reduce the error in the assessment of the effect of an intervention that arises from biases in the way that studies are published and reported.

The series of steps involved in searching for and identifying eligible studies for the review is lengthy and resource intensive. It involves searches that often turn up no additional studies, despite the searchers’ investment in time that can run into the hundreds of hours. Review teams may be reluctant to take on more searching than absolutely necessary. That said, in recent years it has become clear that the likelihood of finding a critical unpublished study or study data that changes key summary outcomes may be greater than we had once thought. For this reason, we are recommending searching these other sources for studies that might otherwise not be identified. We understand that the number of potential sources for searching is large, and that the task of searching for unreported studies and data can never be considered “complete,” because the “truth” is unknown.

Accordingly, we temper our recommendation for searching other sources with a recommendation to be selective and to choose the sources to be searched where it makes most sense. If a review concerns a drug used off-label, U.S. Food and Drug Administration (FDA) records will not contain effectiveness data for that indication, although they might well contain adverse effect data which could be useful across indications. As another example, if a condition is well-studied in another country (e.g., stroke trials in Japan), it may be a good idea to pay attention to the literature from that country and in that language. As a third example, given the fact that only 60 percent of randomized controlled trials (RCTs) described in conference abstracts reach full publication, 5 and full publication is associated with results favoring the test intervention, then conference abstracts from the meeting(s) most likely to publish trial abstracts are probably worth searching. That said, before conducting their own search, the systematic reviewers should check sources such as the Cochrane Collaboration’s Central Register of Controlled Trials to make sure this task hasn’t already been done by others.

The earlier guidance chapter by Relevo and Balshem 6 (referred to subsequently as Finding Evidence) provides guidance on the standard search for evidence. Here, we expand on that guidance and describe supplementary searches that should be considered as approaches to mitigating the effects of reporting bias. We describe the major data sources that should be considered when searching for unpublished studies, and for published studies that are not likely to be identified through a search of the sources described in Finding Evidence. We discuss when those sources are likely to provide useful evidence and provide guidance on when searches of these sources should be considered.

We do not address the issue of multiple publication bias in this guidance. Multiple publication bias occurs when studies with significant or positive results are reported in multiple publications without citing the other reports of the same study. Nor do we discuss the problem of ghostwriting, which is a question of appropriately and transparently attributing authorship. Instead we focus on providing guidance on identifying studies through the use of special searches, such as contacting authors, use of data from regulatory sites, use of protocols, hand searching, and the inclusion of non-English language literature, to reduce the likelihood of bias in estimates of effects of interventions.

Workgroup Composition

The workgroup for this chapter included 14 investigators and research associates from seven Evidence-based Practice Centers (EPCs) and the Agency for Healthcare Research and Quality (AHRQ). Nearly all workgroup members were authors of multiple systematic reviews with experience in addressing issues of reporting bias, and several have written extensively on the topic. A research librarian with several years of experience in conducting searches for systematic and comparative effectiveness reviews was also a member of the workgroup. The topic was co-led by the Oregon and Ottawa EPCs. Project leadership involved establishing timelines, coordinating and scheduling conference calls, participation in subgroups, contributing to the writing of multiple sections of the guidance, and editing the overall guidance.

Guidance Development

We split the workgroup into two subgroups. A subgroup on comprehensive and special searches focused on issues of finding all relevant published and unpublished literature as well as unpublished data from published studies. The second workgroup focused on how to identify and assess the likelihood of biases arising from selective outcome and selective analysis reporting. Each workgroup member participated in one or more subgroups. While we considered techniques for assessing the likelihood of publication bias outside the scope of this guidance, some approaches for assessing publication bias were addressed by the second workgroup.

The research librarian conducted a search for literature on topics related to reporting biases and compiled an EndNote library of relevant sources. Additional searches for literature were conducted at the request of the workgroups. The search identified more than 500 references spanning the period from 1959 through 2012.

The resulting guidance is based on empiric evidence, where available, and on experience and consensus where evidence was ambiguous or unavailable. Drafts of each subsection were first reviewed by the subgroup responsible for those sections. Subsequently a combined draft of both subsections was reviewed by all workgroup members and revisions made based on that review. The revised draft was then submitted for review by all EPC directors and others at the EPCs interested in providing comments, as well as by an associate editor of the Effective Health Care Program and the project Task Order Officer from AHRQ. We revised the guidance to address the major concerns of these EPC internal reviewers and submitted a revised draft for external peer review and public comment. Comments from reviewers and potential edits were discussed by the workgroup both through conference calls and email. The document was revised again based on peer review and public comment. However, the final guidance reflects the views of the authors and the EPC program, and not those of the peer or public reviewers.

This guidance is divided into four parts. The first part provides an introduction to the guidance, describes the methodology used to develop the guidance, and provides some brief background information on reporting bias. Part 2 describes the major sources of evidence that can be used to minimize the risk of missing information relevant to the review, discusses the available evidence on the value of searching each source, and provides recommended guidance on using each source. Part 3 provides guidance on the process of assessing for selective reporting of outcomes and analyses. Finally, Part 4 offers brief guidance on reporting the search strategy and results.

Definitions and History

The Institute of Medicine has recently described reporting bias as “the greatest obstacle to obtaining a complete collection of relevant information on the effectiveness of health care interventions.” 7 Reporting bias occurs when the dissemination and reporting of research results is influenced by the nature and direction of the findings. The selective publication of results—often those that are statistically significant (“positive”) over nonsignificant (“negative”) or null results—has been recognized for centuries. 8 Despite this, research was not undertaken to describe the size of the problem until about 50 years ago, when Sterling raised concerns that research yielding nonsignificant results was generally not published. 9 He confirmed his findings 35 years later in a second survey, 10 and to this day new research continues to demonstrate the existence of sizable publication bias. 11 – 18 Box 1 describes several types of reporting biases that have been identified in the literature.

Definitions of some types of reporting biases.

Reporting biases result both from the absences of complete studies from the body of literature and from the selective reporting of outcomes and analyses within individual study reports. While all publications necessarily select outcomes and analyses to report, outcome reporting bias and analysis reporting bias occur when outcomes are selectively reported or data selectively analyzed—typically in a post hoc fashion—to favor a hypothesis.

An example of selective outcome reporting might be when a trial protocol indicates the primary outcome is the evaluation of an intervention’s effect on increasing survival, and the publication of the trial’s primary results does not mention survivorship (for which there may have been no effect), but instead indicates that quality of life was the primary outcome, or reports results in a way that implies that quality of life was the primary outcome. Here the trial investigators have provided readers with information about certain outcomes and not others, and misrepresent outcomes as described in the protocol. Chan, et al. compared the contents of 102 trial protocols approved by the scientific ethics committees from Copenhagen and Frederiksberg, Denmark, during 1994 and 1995 with 122 subsequent publications. 20 They reported that in nearly two thirds of the trials there was a change in at least one primary outcome between the protocol and publication. The authors also reported that statistically significant outcomes had a higher likelihood of being reported compared with nonsignificant outcomes.

Selective analysis reporting operates in a similar manner. Here study authors may use selective cutoffs to dichotomize continuous outcomes or report selective time-point analyses when multiple time points were specified for analysis in the protocol.

The selective reporting of outcomes and analyses in published primary reports of individual studies may lead to biased interpretation of findings not only of individual studies but also of systematic reviews that include these studies. 21 Several studies provide empirical evidence of the effect of selective outcome reporting and selective analysis reporting on the pooled estimates of treatment effects. 22 – 25 In addition, the selective reporting of analyses and outcomes may also operate at the systematic review level. 21 – 23 , 26 – 29

Types of Selective Outcome Reporting and Selective Analysis Reporting

Selective outcome reporting and selective analysis reporting can be introduced at several points. At the protocol or conceptual stage of devising a study, investigators may choose outcomes based on whether they will produce favorable results, rather than on their importance for clinical practice or policy decision making. Given the aims, objectives, and duration of a study, a strong suspicion in the minds of reviewers that a key outcome of interest was excluded from the study results, which most investigators would not have excluded, should in itself be taken as a signal for risk of selective outcome reporting bias, despite good agreement between study results reporting and study protocol. In other words, the failure to address clinically important outcomes may introduce a form of outcome reporting bias, if studies with negative results for that outcome are less likely to be published. During results analysis, bias occurs if investigators decide to change their analysis (e.g., change in time point) in order to present favorable results or report the most favorable of the several analyses undertaken. Additionally, results might be selectively reported (or withheld from reporting) to support competing interests. It may not be possible to determine whether some or all of these occur within a given study; this will depend on the extent of information available from other sources, such as the study protocol. Table 1 lists the types of selective outcome reporting and selective analysis reporting that could be identified and determined when assessing studies. Some of these constructs are also listed elsewhere. 30 , 31

Types of selective outcome and analysis reporting, which may affect the direction and/or magnitude of the reported study findings.

• Sources of Evidence

Institute of Medicine (IOM) standard 3.2 requires those conducting systematic reviews to “take action to address potentially biased reporting of research results.” 7 This section discusses the various sources of data discussed in the IOM report, provides empirical evidence of their value as sources of information both for unpublished studies and for unpublished data in published studies, as well as evidence that excluding evidence from these sources can lead to biased effect estimates, and recommends how these sources can be used in the search for evidence.

Grey Literature

The IOM describes grey literature as including trial registries, conference abstracts, books, dissertations, monographs, and reports held by the U.S. Food and Drug Administration (FDA) and other government agencies, academics, business, and industry. Standard 3.2.1 recommends that those conducting a systematic review should “search grey literature databases, clinical trial registries, and other sources of unpublished information about studies.” 7 Our recommendations for incorporating grey literature in the guidance below apply specifically to reviews of conventional drugs and devices ( Table 2 ).

Recommended sources of grey literature for conventional drugs and devices.

Study Registries

Study registries are publicly available databases or platforms, commonly Web-based, in which research studies are catalogued. In the last 5 years, several trial registries have evolved into data repositories of key elements of the trial protocols, including outcomes and/or their summary results. Trial registries can serve as a resource both for identifying unpublished studies and for identifying unreported outcomes in published studies.

The FDA Modernization Act of 1997 32 mandates the registration of clinical trials that evaluate the efficacy of drugs for serious or life-threatening diseases and conducted under an investigational New Drug Application. Beginning in 2005, the International Committee of Medical Journal Editors (ICMJE) required prospective trial registration as a precondition for publication. 28 The FDA Amendments Act of 2007 33 further required that trials already in progress be registered on ClinicalTrials.gov by December 2007 and that researchers post a summary of basic results within a year of completion of data collection or within 30 days after the FDA first approved the drug (see Table 3 ). However, it’s important to note that the FDA Amendments Act does not cover trials initiated and completed before 2007, and so will not cover older drugs unless they are tested in trials that were either initiated or ongoing in 2007. 34 ClinicalTrials.gov, launched in 2000 to comply with FDA Modernization Act, currently contains over 139,000 trials sponsored by the National Institutes of Health, other Federal agencies, and private industry. Studies listed in the database are conducted in all 50 States and in 182 countries. 35 Appendix A describes the data elements available from ClinicalTrials.gov.

Registration and reporting requirements of the U.S. Food and Drug Administration Amendments Act, Section 801 (reprinted with permission from Wood 2009).

The World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP) was established in 2005 as a portal that imports trial registration data from clinical trial registries around the world including ClinicalTrials.gov. It contains more than 180,000 records for nearly 170,000 trials, including records for more than 60,000 trials conducted in the United States. 36 Appendix B describes the data elements available from the ICTRP.

Observational studies, where the assignment of subjects into a treated group versus a control group is outside the control of the investigator, can occasionally be found in study registries. Several trial registries, including ClinicalTrials.gov, ISRCTN/ControlledClinicalTrials, ANZCTR (Australia/New Zealand), Clinical Trials Registry-India, UMIN Clinical Trials Registry (Japan), and the Chinese Clinical Trials Registry, allow registration of observational studies, with observational studies representing 17 percent of all studies registered in ClinicalTrials.gov in the year 2010. 37 However, the utility of these external sources of registry data for identifying or minimizing reporting bias associated with observational studies has not yet been evaluated. There is growing interest in registration of observational studies, especially prospective observational studies, 37 – 39 although some have suggested that requirements to register observational studies might actually impede, rather than advance scientific discovery because serendipity, exploration and chance findings will be lost. 40 , 41

Empirical Findings on the Value of Searching Study Registries

Despite registration requirements more than half of the trials that reported start dates with their registration were registered late 43 and only 12 to 22 percent of trials posted results within one year of completion. 43 , 44 The number of unregistered trials and those with missing results is unknown, as is the accuracy of the data submitted. 27 Compliance with the FDA Amendments Act mandatory reporting requirement of trial results is low: within one year of study completion, only 22 percent of 738 trials were compliant. 44 In a review of a sample of trials registered with the ICTRP between June 2008 and June 2009, Viergever and Ghersi 45 found that over half of the trials were registered after the date of first enrolment and that contact information was available for 94 percent of nonindustry funded and for 54 percent of industry funded trials. Compliance with the requirement to post results for both industry and nonindustry sponsored studies at ClinicalTrials.gov is also poor. 46 The proportion of registries with adequate reporting of trial methodology ranged from 1.4 percent (allocation concealment) to 66 percent (primary outcomes) in a study of ClincialTrials.gov and six other registries supported by the WHO search portal ICTRP. 47

In a study of National Institutes of Health funded trials registered in ClinicalTrials.gov, Ross, et al. 48 found that fewer than half the trials were published in a peer reviewed journal indexed in MEDLINE within 30 months after trial completion. In an earlier study Ross, et al. 49 found that only 46 percent of all completed studies registered in ClinicalTrials.gov had been published, and that even when published, fewer than half of the registrations included a citation to the published report. Wieseler, et al. compared journal publications, clinical study reports submitted to regulatory agencies, and trial registry information and noted that study information was most comprehensively reported in regulatory submissions with registry and publications complementing each other. 50

Although study registration and the reporting of study results remains incomplete and may be delayed, trial registries can still help to identify both unpublished studies and unpublished outcomes in published studies. 21 , 46 , 49 , 51 – 54 Dwan, et al., 21 in their systematic review of the empirical evidence of study publication and outcome reporting bias, included studies of cohorts of trials examining discrepancies between trial registry entries and associated protocols and publications. Several discrepancies were noted–differences in reporting of sample size calculations (84 percent) and methods of allocation concealment (6 percent), handling of missing data (80 percent) blinding (67 percent), and primary outcome analysis (60 percent). Six other studies have shown similar discrepancies between trial registries and subsequent publications in reporting efficacy outcomes and adverse events (e.g., primary outcome omission, upgrading from secondary to primary outcome, new primary outcome introduction, underreporting of recurrent and low grade adverse events, incomplete description of adverse events, and tendency for reporting of statistically significant results favoring test drug). 17 , 46 , 49 , 51 , 52 , 54

Guidance on Using Study Registries

• Reviewers should always search ClinicalTrials.gov and the ICTRP for trials that began recruitment after 2005.
• Match trials with publications found from the standard search, noting (1) trials with existing publication, and (2) trials for which no publication was found.
• Construct a table that provides information on trials found in the registry, their publication status, and whether they are completed or currently active trials, and provide a count of the number of unique trials found along with their status at the time of the search.

Because of its broader coverage, and because that coverage includes trials registered in ClinicalTrials.gov, we recommend that EPCs always consider conducting a search of the ICTRP in addition to ClinicalTrials.gov. However, because ICTRP does not require results reporting, systematic reviewers will always want to directly search ClinicalTrials.gov. Unpublished studies should be identified by matching studies found in the registry search with publications found in the literature search. This is specifically true for trials that began recruitment after 2008 and for which at least one of the participating centers was based in the United States. While mandatory reporting of results in ClinicalTrials.gov came into effect in Dec 2007, the registry was launched in 2000. The ICMJE required prospective trial registration as a precondition for publication in 2005. This latter date coincides with the launch of ICTRP and appears a reasonable cut-off for when the registries should be searched.

Regulatory Documents

Reviews of drugs compared with devices.

Drugs and devices are both regulated by the FDA. However, the regulatory requirements and the approval processes for drugs and devices are quite different. 55 These differences, described below, limit the usefulness of searches of the FDA for information about effectiveness studies on medical devices.

Drug Approval Process

Manufacturers are required to submit a New Drug Application to the FDA for all new drugs for which approval for marketing in the United States is sought. The FDA Center for Drug Evaluation and Research (CDER) reviews the clinical and preclinical data for the proposed indication and makes a determination of approval status. Findings of those reviews are included in a number of FDA documents.

While there are often dozens of documents and tens of thousands of pages produced during the course of the review, the two documents of most relevance to those conducting systematic reviews are the Medical Reviews (sometimes referred to as Clinical Reviews) and the Statistical Reviews. The Medical Review is a comprehensive summary and analysis of the clinical data submitted in support of a marketing application and includes the FDA reviewer’s assessment of and conclusions about: (1) the evidence of effectiveness and safety under the proposed conditions of use; (2) the adequacy of the directions for use; and (3) recommendations on regulatory action based on the clinical data submitted by an applicant. The Statistical Review describes key statistical issues and findings that affect conclusions regarding the demonstration of efficacy/safety. It summarizes and discusses the reviewer’s analyses, the extent of evidence in support of claims, and statistical issues that may affect the conclusion on efficacy and/or safety, and is based on a review of individual studies as well as on the collective evidence. In addition to the primary endpoint analysis, the statistical reviewer may also address secondary or subgroup analyses if these are deemed important. Finally, the FDA officer reports may also provide authors of systematic reviews with a list of potential studies for inclusion that may not have been found through other sources.

Drugs@FDA, ( http://www.accessdata.fda.gov/scripts/cder/drugsatfda/index.cfm ) a Web-based, searchable database of information about FDA-approved brand name and generic prescription and over-the-counter human drugs and biological therapeutic products, while challenging to use, provides access to Medical and Statistical Reviews (see Appendix C ).

Device Approval Process

Medical devices are regulated by the FDA Center for Devices and Radiological Health, and while all devices must comply with regulations regarding good manufacturing practices, proper labeling, adequate packaging, and registration with the FDA, most devices are approved through a process that is much less demanding than that required for drugs and which, for most, does not require trials demonstrating safety and efficacy. 55 Prior to 1976, medical devices were not required to be registered with the FDA or to follow quality control standards prior to marketing, and have come to be known as predicate devices. Since 1976, devices are classified into one of three categories depending on their perceived level of risk. Class 1 devices are those considered to have the lowest level of risk and include devices such as such as tongue depressors and Band-Aids. Class II, which includes devices such as forceps and surgical lasers are considered to pose a greater level of risk. Class III devices are devices that support or sustain life, such as drug-eluding stents and pacemakers, and are considered to have the highest level of risk for injury or illness. Only Class III devices go through a process known as a Premarket Application that is more similar to the process required for drugs, and requires a demonstration of sufficient scientific evidence to demonstrate safety and efficacy for the intended use. However, only about 2 percent of all devices are approved through the Premarket Application process.

While not as useful as Drugs@FDA, a Web-based, searchable database of information about FDA-approved devices (Devices@FDA) is available at http://www.accessdata.fda.gov/scripts/cdrh/devicesatfda/index.cfm .

Empirical Findings on the Value of Searching for Regulatory Documents

Relatively few studies have looked at the impact of including information from regulatory documents on the conclusions of comparative effectiveness reviews. Reviews of the use of FDA documents have found that inclusion of unpublished studies from FDA documents may reduce the estimate of effect found in published studies; 56 that FDA documents suggested an elevated risk of harms not acknowledged in FDA advisory committee recommendations; 57 , 58 that prompt analysis of data available to the FDA can identify harms not identified in the published literature; 59 and that publication is associated with positive outcomes; 25 but also found that the highly selective nature of the populations included in the unpublished trials raise questions about the applicability of those findings to actual clinical practice. 60 Similarly, a review of published and unpublished data provided to the British Medicines and Healthcare products Regulatory Agency found that while published data indicated that benefits of the study drugs outweighed their risks, that the inclusion of unpublished data suggested that risks outweighed benefits for all but one of the drugs reviewed. 61

Rising, et al. 17 compared publications with data submitted to regulatory agencies and found additional and omitted outcomes and reporting of different statistical analyses in the published versions. An updated Cochrane systematic review on oseltamivir for preventing and treating influenza incorporated previously unpublished data obtained from regulators. 62 The authors found evidence of reporting bias in trial publications, and conclusions changed such that the drug could no longer be considered effective. Hart et al. 22 reanalyzed 42 meta-analysis of nine drugs with additional, unpublished data obtained from the FDA. Lower drug efficacy was found in 46 percent of reanalyses, identical efficacy in 7 percent, and greater efficacy in 46 percent. Harms were underestimated when the meta-analysis was restricted to published data. Turner, et al., when comparing the results of unpublished trials of second-generation antipsychotics found in FDA documents with the results of published trials, found that the effect size of the unpublished trials was significantly less than half that for the published trials. 24

Data from the FDA may be obtained by searching the FDA Web site, submitting a Freedom of Information Act request, or both. Over a period of several weeks to months the FDA releases the data in the form of their medical and statistical reviews. However, even when available, FDA reviews can be difficult to use. O’Connor found that the search engine could fail to find a review even when using the application number, and noted that reviews are difficult to navigate, generally being quite long with inadequate or incorrect tables of contents. 63

New policies of the European Medical Agency allow access to regulatory submissions with minimal, commercially sensitive redaction, and will soon provide access to clinical trial data for medications it considers for approval. 64 – 66 A review of documents released under the 2010 policy providing access to all documents held by the Agency, suggests that the European Medical Agency should be considered a valuable and routine source of regulatory documents on drug studies. 64

Guidance on Using Regulatory Documents

• Reviewers should routinely search and request Clinical Study Reports from the European Medical Agency.
• Reviewers should search Drugs@FDA for information on drugs; if a search is not conducted reviewers should provide a rationale explaining why the search was not considered necessary or appropriate.
• When reviewers search for evidence at Drugs@FDA, they should focus their search on the Medical Review and Statistical Review documents.

Reviewers should routinely search and request Clinical Study Reports from the European Medical Agency. Reviewers should also conduct a search of the FDA CDER Drugs@FDA Web site ( www.accessdata.fda.gov/scripts/cder/drugsatfda/ ) for Medical and Statistical Reviews relevant to the review and consider submitting Freedom of Information Act requests for drug and class III device trial data early in the course of their systematic review to allow for FDA response time, which could be several weeks. When a search of these sources is not conducted, the review should provide a rationale for why the authors believed that a search was not necessary. As an example, consider a comparative effectiveness review (CER) on treatment for migraine. Such a review may require consideration of as many as 20 different drug classes. In such a situation a review of FDA documents may, at present, prove impractical because of the challenges of using the FDA site. In this instance reviewers may choose not to search the FDA site, but they should provide a rationale explaining their reason for not doing so and consider factoring in this limitation in their assessment of the risk of reporting bias.

The Drugs@FDA site may be searched by the generic or trade drug name (not drug class) for Statistical and Medical Reviews written by FDA personnel examining information submitted by pharmaceutical companies for drug approval. However, the Web site typically does not have documents related to older drugs and very new drugs. Reviews should be downloaded and hand searched for trials. The CDER site also lists any post-marketing study commitments that are made after the FDA has approved a product for marketing (e.g., studies requiring the sponsor to demonstrate clinical benefit of a product following accelerated approval). 67

Information contained in these reviews may not be not adequate to assess trial quality. However, information included in the reviews can identify unpublished studies and unpublished data from published studies, and can be used to verify data obtained from published manuscripts of these trials or to supplement the published results. Studies identified in FDA documents should be compared with those found in the published literature and unpublished studies submitted by manufacturers to identify any remaining unpublished studies or relevant study data not previously published. In addition, the results of the trials reported in the FDA documents should be compared with those reported in published reports of the same studies to identify variation in outcome reporting. However, comparing data from the FDA Medical and Statistical Review documents can be challenging because it is not always easy to identify whether a particular FDA report pertains to a given included study, and it is important to avoid double counting study data in an evidence synthesis.

Study Protocols

A clinical study protocol is a document that provides details of the study plan and organization and is written prior to the start of subject recruitment and data collection. Protocols include information on study rationale, objectives, methodology (design and statistical approaches), types of participants (i.e., inclusion and exclusion criteria), treatments, clinical procedures, ethical considerations, and the duration of the study. 68 , 69

Study protocols and related information can be located and accessed from several sources such as study authors, industry registries, trial registries, Web sites of relevant agencies (e.g., ClinicalTrials.gov, canadatrials.com, controlled-trials.com, and WHO ICTRP), and through documents made public as a result of litigation. Also, several peer reviewed medical journals including The Lancet , Trials, and others publish study protocols, or summaries of protocols with full protocols available upon request. The Lancet began publishing protocols of randomized trials in 1997 and extended this to observational studies in 2001. 70 , 71 BioMed Central began publishing protocols for a variety of study designs in 2001. 72 In 2006, the journal Trials was launched and has accepted study protocols from the outset. 73

Empirical Findings on the Value of Searching for Protocols

Several empirical studies comparing protocols and published reports of individual trials for consistency and completeness of outcomes and analyses 20 , 54 , 74 , 75 provide evidence of outcome reporting bias in published reports of individual RCTs. Dwan, et al. published two systematic reviews that summarize these findings. 13 , 21 These studies report a high prevalence of unreported or incompletely reported outcomes. Outcomes with a statistically significant difference were more likely to be reported than outcomes associated with a nonsignificant difference (OR [odds ratio] 2.4, 95% CI [confidence interval], 1.4 to 4.0). 20 The primary outcomes specified in the protocols were either changed to secondary (and a new primary outcome was introduced), or omitted from the subsequent publication. 20 , 54 , 74 , 75 In a review of study protocols examined as part of a litigation against Pfizer and Parke-Davis regarding off-label use of gabapentin, published primary outcomes differed from those described in the protocol in 8 of 12 reported trials and all changes between what was specified in the protocol and what was later published led to a more favorable presentation of the efficacy of gabapentin for unapproved indications. 3 However, finding protocols can be challenging. Hartling, et al. in their systematic review attempted to inform their study risk of bias assessments by additionally retrieving protocols for 42 of 107 trials. No restrictions such as on the country in which the trial was conducted, or year of publication were employed. The yield was low (protocols could be obtained for just 12 percent of studies), with protocol retrieval adding 50 percent more time to risk of bias assessment. 76

Guidance on Searching for Study Protocols

• For a priori study methods, grey literature may be a helpful source in the absence of access to full protocol.
• Study protocols that are retrieved in the literature search should be routinely used to identify selective outcome and analysis reporting.

When the protocol for an included study is not found as part of the standard search, reviewers should include other relevant sources such as contacting authors and searching trial registries, industry sites, regulatory submissions, and bibliographic databases not previously searched to attempt to obtain either the protocol or protocol-related details. Since protocols are frequently amended, reviewers should search for later amendments and cross validate the currency of study protocols against Clinical Study Reports submitted to regulatory agencies and using the “history” function of ClinicalTrials.gov.

Conference Abstracts and Proceedings

Authors frequently present, in oral or poster form, interim or full study results at professional meetings. Often, meeting submissions are collated as a catalogue of abstracts.

Empirical Findings on the Value of Searching Conference Abstracts and Proceedings

In a review of findings initially presented as abstracts at European General Practice Research Network meetings from 1999–2002 and 2005–2006, Van Royen, et al. found overall 45 percent of the presentations to have been subsequently published, with abstracts from the 2005 to 2006 meetings having only a slightly higher publication rate (43 percent for the period 1999–2002 and 47 percent for the period 2005–2006). 77 Similarly, Scherer, et al. found that fewer than half of all abstracts were published in full, and that positive results were positively associated with full publication. 5 Tam and Hotte 78 compared a subset of phase III trials presented at the 2000 American Society of Clinical Oncology Annual Meeting with their subsequent full publication (by May 2006). Of 55 abstracts that were subsequently published, the primary endpoint was stated in 34 percent of abstracts compared with 100 percent of publications. Primary and secondary endpoints, primary endpoint results, statistical analysis, and statistical significance of the primary endpoint were frequently not clearly described in the abstract. For abstracts that were clearly described, primary endpoints were identical in 90 percent of cases; statistical significance of the primary endpoint and conclusions were identical in 89 percent and 91 percent of cases, respectively. The primary endpoint results differed by more than 5 percent in 42 percent of abstract-to-publication comparisons. However, abstracts and proceedings frequently report only preliminary results, which may not accurately represent what was found once all data were collected and analyzed. 79 – 81

Guidance on Using Conference Abstracts and Proceedings

• Reviewers should routinely consider conducting a search of conference abstracts and proceedings to identify unpublished or unidentified studies.
• Consult the TEP for suggestions on particular conferences to search and search those conferences specifically.
• Search the full conference abstracts of any meeting identified by reading the references of key articles.
• We do not recommend using conference and meeting abstracts for assessing selective outcome reporting and selective analysis reporting, given the variable evidence of concordance between conference abstracts and their subsequent full-text publications. Abstract and conference proceedings should be searched as a source for identifying trials that may not otherwise be published or which might have been missed in the initial search.

Current guidance 6 stipulates always including search of databases that index meeting reports, such as Conference Papers Index, Scopus, Papers and Proceedings 1 st , BIOSIS previews, et cetera. That guidance notes that because the yield is often in the hundreds rather than in the thousands it does not add appreciably to the burden of the review. Current guidance also recommends searching the reports of specific conferences if any Technical Expert Panel (TEP) member or other key informant suggests that the topic of a particular meeting or conference is highly relevant to the topic of the report and searching the full conference abstracts of any meeting that is found by reading the references of other relevant articles. 6

• Grant Databases

Several grant databases allow for analysis of the registration and publication status of all United States Federally funded studies ( Appendix D ).

The Federal Research Portfolio Online Reporting Tools (RePORT) database, the largest United States based grants database, provides several downloadable and analyzable data elements, including start and end dates, names and affiliations of principal investigators, financial information about the grants, and grant titles and project abstracts. The RePORT database does not include variables indicating study registration or participant recruitment status, rendering it difficult to determine if the study has been completed.

In addition, the current practice of posting all publications that mention a grant complicates attempts to determine a study’s publication status. The RePORT Web site warns that articles posted on the site “are associated with projects, but cannot be identified with any particular year of the project or fiscal year of funding. Some publications will be inadvertently linked to the wrong grant or missing altogether.” Most published articles include several grant numbers, and each grant project includes links to several articles. Published article titles and abstracts often differ from descriptions of the grants.

Empirical Findings on the Value of Searching Grants Databases

Empirical evidence shows low registration rates in clinical trial registries for federally funded trials. 82 , 83 Recent studies that have examined the registration and publication of National Institutes of Health (NIH) funded studies have found poor availability of protocols and study results. 82 , 83 The analysis of NIH funded pediatric trials demonstrated that only 33 percent were registered and only 53 percent were published. 82 The analysis of NIH funded therapeutic studies for female urinary incontinence found that only 6 percent were registered. 83 Published studies (94 percent of all NIH funded) mentioned the NIH grant numbers but did not necessarily report study results. 83

We found no studies comparing the protocols of registered NIH funded studies with published results to evaluate deviations from the protocol and selective outcome reporting.

Guidance on Using Grants Databases

• Searches of grants databases, in general, should only be conducted upon suggestions from the TEP or other key informants.
• Since the process of matching to publications is challenging and the yield likely to be low, when grants databases are searched, we recommend conducting a pilot search first.
• After identifying studies from the grants database, search trial registries using the grant number, title, or name of principal investigator.
• Look for publications of funded grants by searching MEDLINE with the grant number or title.

Since this task is time consuming, we recommend searching grant databases when review authors and Key Informants or the TEP anticipate a significant yield in the number of eligible studies. Review authors should search trial registries using grant titles and numbers for each study to determine registration status of eligible studies. The process of finding exact publications is manual and time consuming. Therefore review authors may conduct a pilot search in grant databases to estimate potential yield in eligible studies. After all funded studies are identified, review authors can compare grant description or posted protocols with publications to judge publication bias and selective outcome reporting.

Contacting Authors

The completeness of reporting of individual studies (and systematic reviews themselves) is often suboptimal. Authors of a study may not have reported all of the outcomes specified in study protocols, may not have completely described the type of participants included in their study, or may have provided published analyses only for the whole study population when analyses were also done for subpopulations. Contacting study authors may be useful for obtaining missing or unreported outcomes, obtaining outcomes in a format suitable for meta-analysis, or to clarify potential errors or unclear results. Contacting authors might also provide additional information regarding study methods that may prove helpful in rating study quality.

Empirical Findings on the Value of Contacting Authors

There are few papers examining the utility of contacting authors in the context of conducting a systematic review. Mullan, et al. reviewed 147 published systematic reviews, of which 54 were Cochrane reviews and 93 were published in high-impact journals. The researchers reported that 46 (50 percent) of the traditionally published reviews and 46 (85 percent) of the Cochrane reviews reported contacting study authors. 84 Missing data was the most common reason for contacting study authors.

In a systematic review of the literature on methods for obtaining unpublished data, Young, et al. found that, in general, requests to authors for clarification about study methods were more likely to be successful than requests for missing data about study results. While contacting authors by email seems to result in the greatest response rate with the fewest number of attempts and the shortest time to respond, they also found that there is no consistent evidence about what approaches work best. 85

Three studies not considered in the Young review assessed whether contacting authors for more information adds substantive information. Kyzas, et al. 86 found that contacting authors (with second attempt at 2 months) and obtaining additional data (11 studies; 996 patients) changed results from statistically significant (RR [relative risk] 1.23, 95% CI, 1.03 to 1.47; 31 studies; 2,392 patients) to not significant (RR 1.16, 95% CI, 0.99 to 1.35, p=0.06; 3,388 patients). Young, et al. noted, however, that response rates do not seem to be influenced by the number of requests. 85

Chan, et al. 74 compared trial protocols with their published versions for 48 relatively large randomized studies funded by the Canadian Institutes of Health Research (1990–1998), the Canadian governmental funding agency. Eighty-eight percent of the 48 trials measuring efficacy and 62 percent of 26 trials measuring harms had at least one unreported outcome. They surveyed authors, and of 43 respondents, 80 percent denied that any outcomes were unreported. When study authors were provided with a list of unreported outcomes at 6 weeks after the initial query, 37 respondents (77 percent) provided some details about the unreported outcomes. Kirkham, 23 in evaluating trials included in a cohort of Cochrane reviews for selective outcome reporting, contacted authors of 167 trials for additional information and received a response from only 39 percent of authors in 3 weeks. They were able to confirm and obtain reasons as to whether outcomes were measured and not analyzed or just not measured. The authors observed similar response rates for trials at high and low risk of suspected outcome reporting bias. It is not known how generalizable the above response rates are, particularly given that some reference older trials when authors were not as aware of such biases. An additional limitation to contacting authors is that they may not have access to full data, or may be contractually obligated to nondisclosure

Guidance on Contacting Authors

• Although likely to occur infrequently, authors should be contacted when in the review team’s judgment clarification regarding study eligibility, study design, or other aspects of study conduct is essential to the conduct of the CER and may affect conclusions.
• When authors are contacted, we recommend that no more than three attempts at contact be made, each attempt separated by a week, and that this be done consistently for all authors from whom information is being sought.
• When contacting authors, be clear and concise in your request and, when possible, provide a table identifying the specific data being requested.
• If bias is suspected based on the study report, adding this to the correspondence may help with obtaining information.
• When reviewers contact authors, they should report the number of authors they attempted to contact, the number of authors actually contacted, and the percentage of authors who responded positively to the request for information.

IOM standard 3.2.2 recommends that authors of systematic reviews “invite researchers to clarify information about study eligibility, study characteristics, and risk of bias.” Although not part of a standard search, and likely to occur infrequently, EPCs should contact researchers and invite them to provide necessary information, when in the review team’s judgment clarification regarding study eligibility, study design, or other aspects of study conduct is essential to the conduct of the CER and may affect the conclusions of the review. This might be the case, for example, when only disaggregated data are reported, and there is a needed to evaluate benefits and/or harms in sub-populations included in the aggregate data.

Contacting study authors can be time intensive, with uncertain yield and effects on review conclusions. An additional limitation to contacting authors is that they may not have access to full data or may be contractually obligated to nondisclosure. When trying to contact a study author, there is little guidance as to how many times this should be attempted. We were unable to locate any papers providing guidance concerning this point, although a survey (n=111 respondents) of systematic reviewers conducted by Mullan, et al. 84 reported that most respondents contacted at least one study author. Anecdotal experience suggests trying to contact study authors up to three times separated by a week interval between each attempt. To avoid potential bias it seems sensible to make a similar number of contacts with all study authors from whom additional information is sought. Trying to contact one study author three times and other study authors once is systematically different and might introduce bias. We are unaware of any reports examining the possible biases associated with contacting or not contacting study authors. Theoretically, a bias might arise if efforts to contact study authors were systematically different. For example, if the review team were examining the comparative effectiveness of two drug eluting devices and ended up only contacting authors of papers that systematically provided nonsignificant effect estimates. Therefore, reviewers should consider the possible biasing effects of strategies for contacting study authors and strive to avoid them when possible.

For specific data, such as a missing standard deviation, the review team may want to provide a brief table depicting the missing information. Whatever information is being requested of study authors it is important that the request is made clearly and concisely. It may be useful to let the study authors know that their help will be acknowledged in the review’s report and any subsequent publication.

Contacting Study Sponsors

Some pharmaceutical companies have started to publicly share their own trial registry data. GlaxoSmithKline has announced that it will release all anonymized patient level data since 2007 in their Clinical Study Register ( www.gsk-clinicalstudyregister.com ). 87 , 88 Novo Nordisk also provides Web access to its trial registry. 89 EPC literature searches for published studies are routinely supplemented with a request to the manufacturer for a scientific information packet (SIP). The SIP includes information about products available from the product label as well as information about published and unpublished trials or studies about the product. To ensure consistency in the way SIPs are requested and to ensure transparency by eliminating contact between the EPC conducting the review and the manufacturers of products being reviewed, the Scientific Resource Center for the AHRQ Effective Health Care Program routinely requests SIPs from manufacturers on behalf of the EPCs for all CERs and technical briefs.

Empirical Findings on the Value of Contacting Study Sponsors

Limited evidence exists on the use of industry documents for identifying selective outcome and analysis reporting, and has been mainly obtained through legal proceedings. Vedula et al. compared 12 of 20 internal pharmaceutical company documents with their published versions (1999–2006) for off-label use of gabapentin. 3 The authors found discrepancies in the primary outcome in the publications of 8 of 12 trials (new primary outcome, no distinction between primary and secondary outcomes, change from primary to secondary outcomes, or outcomes omitted), with statistically significant results presented in five publications. Psaty and Kronmal compared mortality data of two published trials with their respective internal pharmaceutical company documents for rofecoxib given for Alzheimer disease or cognitive impairment 2 In both publications, mortality data were provided in narrative form without accompanying statistical analyses, whereas statistically significant hazard ratios were reported in the internal documents.

Jefferson, et al. recounted their unsuccessful experience trying to obtain unpublished data on oseltamivir from the manufacturer and recommended requesting the full clinical study reports for each trial, but noted there is no guarantee those reports are reliable. 90

Guidance on Contacting Study Sponsors

• When available, EPCs should use industry documents in tandem with published study results for their assessments of risk of outcome and analysis reporting biases.
• The SRC, rather than EPC staff, should be responsible for contacting primary study sponsors for Scientific Information Packets.
• The search for industry documents should include information requested directly from manufactures, as well as industry documents available from the Drug Industry Document Archive.
• Reviewers may also consider searching publicly accessible trial registries maintained by GlaxoSmithKline Inc. and Novo Nordisk Inc.

IOM Standard 3.2.3 states that, in addition to contacting study authors and researchers, authors of systematic reviews should “[i]nvite all study sponsors and researchers to submit unpublished data, including unreported outcomes, for possible inclusion in the systematic review.” The request to manufacturers for product information, including information about published and unpublished studies is part of the standard search conducted by the Scientific Resource Center on behalf of the EPCs, and is described in the guidance on Finding Evidence. 6 Industry documents made public as a result of litigation may also be available from the Drug Industry Document Archive (DIDA). When the review team is aware of litigation regarding a drug under review, they should search DIDA for potentially relevant documents. Additional sources that may be searched include:

• GlaxoSmithKline: www.gsk-clinicalstudyregister.com
• Novo Nordisk: www.novonordisk-trials.com/website/content/trial-results.aspx

However, given that there is little data on the completeness, accuracy, or usefulness of industry-maintained trial registries, as well as the lack of evidence that including such data does not tilt the weight of evidence synthesis in favor of one company over another, we hesitate to make a strong recommendation for searching these additional sources of grey literature.

Handsearching

Handsearching refers to manually scanning print journals to identify relevant studies not retrieved by electronic bibliographic databases. Not included within this definition of handsearching are reviews of reference lists and citation tracking, which are other methods for identifying potentially relevant citations. Handsearching may also be valuable for identifying studies published only as conference abstracts, since these are often published as journal supplements that are not included in electronic databases. Examples of situations in which relevant studies may be included in an electronic database but not well indexed include newer interventions that have not yet been assigned Medical Subject Headings (MeSH), and when systematic reviews address complex interventions, process of care topics, or evaluate topics such as harms or subgroup effects that may not be indexed well.

Empirical Findings on the Value of Handsearching

Less than a third of the world’s medical journals are routinely indexed in the major electronic databases. 91 A Cochrane systematic review found that handsearching identified more relevant randomized trials (92 to 100 percent) than searches based on single electronic databases (range 49 to 77 percent). 92 However, more sensitive search strategies such as the Cochrane Highly Sensitive Search Strategy identified 80 percent of relevant randomized trials, or nearly as many as were found by handsearching. This systematic review did not compare the yield of handsearching with searches based on two or more electronic databases, or handsearching compared with searches of electronic databases, reference list reviews, and other supplemental methods, such as peer review suggestions. It also did not evaluate the yield of handsearching for nonrandomized intervention studies or studies of diagnosis or prognosis. One study found that handsearching for studies of diagnostic test accuracy of 18F-fluorodeoxyglucose positron emission tomography-computed tomography did not yield additional studies compared with database searching. 93

Handsearching is time-consuming and resource intensive. Although no study has evaluated differences in estimates of effects when handsearches are conducted in addition to electronic database searches and other supplemental methods, the value of handsearching probably varies depending on the topic of the systematic review. The yield of handsearching is likely to be higher when relevant studies are published in journals that are not indexed in electronic databases, or in journals that are indexed in electronic databases but indexing is suboptimal, associated with a significant lag time, or published as a journal supplement. 94 Studies that may be less likely to be included in standard English-language electronic databases include older studies, studies of complementary and alternative interventions, and non-English language studies.

Guidance on Handsearching

• Reviewers should routinely conduct a search of the Cochrane Central Register of Controlled Trials.
• If reviewers decide that more comprehensive hand searching is warranted, before conducting the search, work with content experts to identify appropriate journals for hand searching and with a librarian to determine how well those journals are indexed in electronic databases.

IOM Standard 3.2.4 states that authors of systematic reviews should “[h]andsearch selected journals and conference abstracts.” Reviewers should routinely conduct a search of the Cochrane Central Register of Controlled Trials (CENTRAL), since CENTRAL is supplemented with studies gleaned from a hand search of more than 2,000 poorly indexed journals. The Master List, available at http://us.cochrane.org/master-list catalogs the journals and conference abstracts being searched by various Cochrane groups. In addition to routinely searching CENTRAL, reviewers should consider on a case-by-case basis whether to conduct handsearches of selected key journals that are highly relevant to the topic of the report, but not fully indexed, or indexed at all, in the major bibliographic databases, to check the sensitivity of electronic database searches. If the hand search does not identify any relevant studies (or only identifies small and/or lower-quality studies that are unlikely to affect the conclusions of the review) more comprehensive handsearching may be unnecessary. If the reviewers determine that more comprehensive handsearching is necessary, either based on the topic of the systematic review or based on finding missed studies in a selective check of journals, we suggest that they work with content experts to determine which journals may be candidates for handsearches, and with a research librarian to determine which of those journals to hand search, based on how well the journal is indexed in electronic databases and the lag time to indexing.

Searching for Non-English Language Literature

Although most of the more significant medical literature is indexed in the major bibliographic databases such as MEDLINE and EMBASE, there is still a considerable amount of relevant and important literature published in non-English language journals that are not indexed by these databases. Therefore, even when systematic reviewers have not placed language restrictions on searches or inclusion criteria, identifying non-English language articles published in these journals may require a search of additional databases such as Global Index Medicus published under the auspices of the WHO and LILACS (Latin American and Caribbean Literature in Health Sciences).

Empirical Findings on the Value of Searching the Non-English Language Literature

A MEDLINE search of all publications from 2000 to February 3, 2011, conducted by the author of this section, found that of 6,574,939 citations, 90 percent were published in English. Table 4 shows the number and frequency of publications in other languages with at least 1 percent frequency.

Percentage of publications from MEDLINE in various languages (1996–2011).

Based on the author’s review of recent CER reports with final or draft documents downloadable from the AHRQ Web site, most (71 percent) EPC reports restricted literature searches to English language publications. Thus, EPC reports may be at risk of selection bias based on language, and may not be consistently following IOM standards for (Standard 3.2.6).

Empirical evidence, however, has not shown consistent findings regarding language bias. For example, investigators in Germany may be more likely to publish their negative results in German language publications and their positive results in English language publications, 95 , 96 and almost all Chinese acupuncture trials published in Chinese report positive results. 97 Numerous other studies, however, have found that excluding non-English language publications may not have an impact on the conclusions in systematic reviews. 98 – 104

Guidance on Searching for Non-English Language Literature

• Reviewers should avoid the use of English-language only filters when searching standard databases.
• Abstracts and other reports of non-English language studies should be tracked to inform a judgment of the likelihood of bias that might arise from excluding non-English language reports.
• Discuss with the TEP whether excluding non-English language articles might bias the findings of the report.
• Search databases that specifically index reports of studies in languages other than English (1) when a review of English-language abstracts suggests systematic differences between studies reported in English language journals and those reported in non-English language journals, or (2) based on information from TEP members or other key informants.

IOM standard 3.2.6 states that those conducting systematic reviews should search for studies reported in languages other than English if appropriate. Searches of databases that specifically index non-English language literature, however, are likely to be the exception, rather than the rule. On the other hand, a review of English language abstracts of non-English language articles, retrieved during the standard search of the major bibliographic databases, can inform the decision regarding the need for a more comprehensive search for non-English language articles. This is why current guidance recommends against the use of English-only filters when searching major bibliographic databases. 6 If a comparison of the English-language abstracts of non-English articles finds consistent systematic differences in results with articles published in English, the review team should consider expanding the search to include non-English language articles. In addition, the review team should discuss with the TEP whether exclusion of non-English studies might bias the report. When an assessment based on these criteria suggests that non-English language articles be included, we recommend a staged approach. Such an approach might initially include a further review of all English language abstracts of non-English language articles found as part of the standard search. Findings from this review might then suggest expanding the search to include special regional databases.

The review team should always review the English language abstracts of non-English language articles retrieved in the search of the major bibliographic databases. The literature search should be expanded to include databases that specifically index non-English language literature such as LILACs (Literatura Latino Americana e do Caribe em Ciências da Saúde) and Global Index Medicus when a review of the abstracts finds:

• Systematic differences between studies reported in English language journals and those reported in non-English language journals;
• Most of the relevant studies have been reported in a language other than English; or
• Most of the studies have been conducted in non-English language regions.

Information From Searches of the World Wide Web

Nearly all searches for evidence today, including searches for regulatory documents, registries, indexed literature, etc. are conducted on the Web. In this section we take the phrase “search the World Wide Web” to mean using standard Web search engines such as Google or Google Scholar, to supplement searches of specific Web sites, such as the FDA Web site moc.ADF@sgurD or ClinicalTrials.gov, or searches of proprietary databases such as MEDLINE and EMBASE. The World Wide Web is a platform for citation-searching databases as opposed to grey literature searches per se.

Empirical Findings on the Value of Searching the World Wide Web

Several studies have compared the citation counts resulting from searches of Web of Science, Scopus, SciFinder, and Google Scholar. 105 – 108 These studies found that Web of Science, Scopus, and Google Scholar produced quantitatively and qualitatively different citation counts, and that each database missed linking to some references included in other databases. None of these studies provided strong evidence that routinely searching the Web has an important impact on review findings.

Guidance on Conducting Searches of the World Wide Web

• We do not recommend that review authors search the World Wide Web for additional information beyond those sources discussed above, unless there are specific reasons to do so
• If the World Wide Web is used as an information source, the rationale for doing so must be clearly presented, along with the methods for searching.

IOM standard 3.2.5 states those conducting systematic reviews should “[c]onduct a web search.” Current guidance recommends using Web of Science or Scopus if they are available. If subscriptions to these services are not available, however, current guidance recommends using Google Scholar rather than other free search engines such as PubReMiner or PubFocus. 6 However, given the lack of evidence, we are uncertain of the utility of searching the World Wide Web to locate additional studies and do not recommend including such a search as part of the standard or expanded search for evidence unless there is a compelling reason to do so. Because there is no strong evidence showing that routinely searching the Web would have an important impact on review findings, and because of the significant resource burden to do so, when a Web search is conducted, a clear rationale for doing so should be presented, along with specific information about the nature of the search, as well as a description of what was retrieved and how that information was screened and included information selected.

• Guidance on the Process of Assessing for Selective Reporting of Outcomes and Analyses

This section explains how the risk of outcome and analysis reporting biases can be assessed and clarified once information on a study has been retrieved. The proposed assessments of outcome and analysis reporting biases specifically reflect a study level risk (potential) for bias as it applies to the review, not the actual bias in the study (which may or may not be present). For example, authors may be genuinely limited by journal word count restrictions and hence report some outcomes in narrative form or omit them altogether. Such omissions would not necessarily result in biased effect estimates, unlike omissions related to the desirability of certain results. Because the intent of authors cannot be known by systematic reviewers, a thoughtful assessment of the risk of outcome and analysis reporting bias is required.

The review stage when grey literature is used for assessing reporting biases may vary across reviews. For example, when reviewers have searched trial registries, contacted authors, obtained relevant documents from industry, and acquired FDA documents up front as part of their standard review search strategy and used the search output to identify studies for which no published report was found (publication bias), they may have simultaneously identified unpublished study data and protocol details for published studies included in their review. As we recommend below, all information for a study should be examined together for risk of bias assessment and data extraction. In such a situation, the risk of reporting bias may be assessed without further searching or additional clarifications from unpublished sources of study information. Alternatively, when the primary search was restricted to published studies, reviewers might want to search and cross-check against those same sources while conducting reporting bias risk assessments.

Principles for Assessing Reporting Bias

Outcome level assessment.

The risk of selective outcome and analysis reporting bias is an outcome-level assessment, as opposed to a study-level assessment. Reporting bias may differ among outcomes because the decision to selectively present or omit outcomes or their analyses will depend directly on the results that were obtained for a given outcome. Similarly, risk of performance bias (e.g., blinding or masking of participants and providers) and detection bias (e.g., blinding or masking of outcome assessors) entail outcome-level assessments, while selection bias (e.g., allocation concealment) is a study-level assessment.

Assess Important Outcomes Determined a Priori

For outcomes of interest to the review, we suggest restricting reporting bias assessments to those outcomes that will be graded for their strength of evidence according to guidance provided by the EPC Program. 109 Gradable outcomes are those determined a priori during the topic refinement phase and reported in the protocol to be important for health care decisionmaking. We make this recommendation for practical reasons, given the volume of outcomes that can be included in an EPC systematic review. Review authors should evaluate reporting bias for their prespecified gradable review outcomes irrespective of whether those outcomes were designated as primary or secondary in the study.

Assessment of Outcome Reporting Bias and Analysis Reporting Bias for Benefits and Harms

In general, reporting bias in trial publications takes the form in which benefits are over reported and harms under reported. 51 , 110 Reporting biases related to harms can be addressed similarly to beneficial outcomes. However, in rare cases, it is possible that a serious harm was identified during the evidence synthesis process and was not prespecified as an outcome to be included in the assessment of the strength of evidence. In this situation, a post hoc decision may then be made to assess the risk of reporting bias specifically for that outcome.

Composite Outcomes

Reporting of composite outcomes, without reporting on component outcomes, may be a signal of reporting bias. 111 A common example in cardiovascular research is the composite outcome of vascular death plus nonfatal myocardial infarction plus nonfatal stroke. Composite effects could mask the effects corresponding to individual components; we cannot assume the individual components have effects equal to the composite. 112 Studies that report composite outcomes should also provide results for the component outcomes.

Reviewers should be suspicious when unexpected components are included or expected components are excluded. For example, in a trial on the effect of hormone replacement therapy on cardiovascular events in recently postmenopausal women, the authors’ primary endpoint was a composite outcome of death, admission to hospital for heart failure, and myocardial infarction. 113 Neither stroke nor angina were included, raising concerns whether it was a planned outcome. 114

Additional Considerations

Outcome and analysis reporting bias should be assessed comparing treatment effects on outcomes in all available reports of the same study (one or more articles, abstracts, results posted in clinicaltrials.gov, and FDA reviews) including their protocols (published protocols, protocol data elements reported in clinicaltrials.gov, and methods sections in the articles). In general, systematic reviewers should recognize that when studies do not investigate or report outcomes of interest to the review this may be due to a reporting bias. Missing outcomes should, therefore, not be considered as a criteria for excluding otherwise eligible studies from the review.

Because of the potential impact on effect estimates, reporting bias should be cautiously assumed to exist even if authors cannot determine its direction and magnitude.

Identifying Selective Outcome and Analysis Reporting in Included Studies

Above we described the various sources of information on study outcomes and analyses; the empirical evidence on the accuracy, completeness, and feasibility of using those sources to identify and characterize selective reporting; and guidance on using those sources. In this section, we suggest a procedure for using those sources to assess for reporting biases while conducting a systematic review. Our recommendations apply mostly to experimental studies. For observational studies we provide distinct recommendations. Our recommendations are likely to be revisited as new or more robust evidence emerges.

The Initial Search for Evidence

The evaluation of the literature for selective outcome and analysis reporting begins with the search for evidence. The goal of the search is both to find evidence and to reassure readers and reviewers that searches have been thorough. This requires conducting a comprehensive search of all the available sources relevant to the objective of the review in order to establish confidence about the inclusiveness of all relevant evidence. Even then, one may be limited by accessibility of evidence.

Observational Studies

During the process of developing the protocol for a systematic review, systematic reviewers need to make decisions as to what study designs are appropriate for answering their research question(s). Based on the nature of the question, outcome, or methodologic preferences, some reviews may include only studies of experimental design (e.g., randomized and/or nonrandomized controlled trials); other reviews may require the addition of observational studies, for example when examining harms outcomes.

By design, trials are always hypothesis testing and are considered “confirmatory” studies: they are designed to test the null hypothesis of no difference between the compared groups for a given outcome. Observational studies may be either confirmatory (i.e., hypothesis-testing) or exploratory (i.e., hypothesis-generating) in nature. Although the risk of selective reporting of the most favorable of multiple analyses exists for both RCTs and observational studies, the risk is much higher when studies are exploratory. However, based on a publication alone, it is often difficult to distinguish between confirmatory and exploratory studies. There may be more concern about data dredging in exploratory studies, and the risk of reporting biases may be greater than for confirmatory studies. 38

Guidance on Including Observational Studies

• We do not recommend searching for registry information for observational studies, as their study registration is not yet mandated and registration is infrequent.
• Reviewers may limit their search for protocols to specific study designs such as trials and prospective observational studies
• We recommend against routinely searching for protocols of retrospective, observational studies. As with RCTs, systematic reviewers can consider contacting study authors for additional information when practical.
• Searching the World Wide Web may be considered as a last option to find protocols of nonrandomized and observational studies.

Identification of Selective Outcome Reporting and Selective Analysis Reporting Based on the Study Report

• As described below, efforts should routinely be made to identify outcome level selective outcome and analysis reporting for each study included in a systematic review.
• In general, systematic reviewers should recognize that studies that do not investigate or report outcomes of interest to the review may be susceptible to selective outcome or analysis reporting, and so should not exclude such studies from the review.
• We suggest restricting outcome and analysis reporting bias assessments to those outcomes that will be graded for their strength of evidence.
• Collate all companion publications (except conference abstracts) for a given study.
• Compare the planned outcomes and analyses as stated in the Methods section of the report, protocol and other source documents with those reported in the results section, looking for discrepancies.

Comparing Methods Section With Results of Published Reports To Judge the Risk of Outcome Reporting and Analysis Reporting Bias

There are limitations to relying on the study publication for identifying the selective reporting of outcomes or analyses. In particular, discrepancies between the Methods and Results sections cannot be reliably considered as adequate assessment of reporting bias because manuscripts are prepared at a late stage in the research process, generally after authors have reviewed the results and decided which data will be presented. As such, the Methods section of the report may already have been selectively tailored to support favorable findings. It should be noted, however, that our assessments of reporting biases specifically reflect a risk as it applies to our review, as opposed to actual bias in the study (which may or may not be present). For example, authors may be genuinely limited by journal word count restrictions and hence report some outcomes in narrative form or completely omit reporting them altogether.

Dichotomization of outcomes data into published and unpublished is overly simplistic. The risk of reporting bias is largely dependent upon the reviewers’ access, or lack of access, to all study source documents — peer-reviewed journal reports and their published companion reports, trial registries, abstracts and conference proceedings, regulatory submissions, industry maintained registries and databases, and unpublished data with authors and sponsors. Because selective reporting may not be convincingly identified from information contained within the published study report and its published companion papers, systematic reviewers should endeavor to retrieve as much of the recommended grey literature as possible before undertaking an assessment of the risk of reporting bias.

Proposed Steps

Assessment of selective reporting bias for a study is outcome specific. For a given systematic review, study outcomes data are at no risk of reporting bias if all the gradable outcomes that inform a systematic review are fully reported, even if others were concealed. In this case, no further action is needed.

While assessing for reporting bias, we recommend that all companion reports (i.e., published or unpublished data) of a study be linked and examined together ( Figure 1 ). When all the study data from various sources are examined together, concerns about reporting bias provisionally suspected in the study publication might be eliminated because, for example, they were obtained from regulatory submissions or another source of grey literature. On the other hand, reporting bias not otherwise suspected in a trial publication might come to light when compared, for example, with study protocol or trial registry data. Thus, the assessment of reporting bias must be made across all included companion reports—published and retrieved grey literature. EPCs may decide whether cross checking against all recommended external source documents is feasible or relevant based on the guidance reported above for each potential source; if not, this needs to be documented with rationale in the systematic review.

Flow diagram of the risk of outcome reporting bias and analysis reporting bias assessment process. Note: ARB=analysis reporting bias, ORB=outcome reporting bias. a Document exact source of information that clarifies or modifies concern of ORB or ARB.

Reviewers should refer to Table 1 for identifying the types of selective reporting impacting the outcome, and categorize their risk assessment as positive, negative or unclear keeping in mind the four levels of measurement specification that have been described by Zarin et al. 29 These include

• Domain– e.g. anxiety
• Specific measurement–e.g. Hamilton Anxiety Rating Scale
• Specific metric–e.g. change from baseline at a specified time, and
• Method of aggregation–e.g. categorical with proportion of patients with decrease ≥ 50 percent

Following are possible scenarios that may be encountered with respect to a hypothetical outcome X:

Scenario 1—Reporting Bias Ruled Out

When it is clear to the reviewers that outcome X was planned (e.g. from protocol, regulatory submissions, etc.), complete outcome data are available from at least one study document (published or otherwise), and the outcome was appropriately analyzed as planned, then the study is not at risk for reporting bias for this outcome (“ORB risk–” or “ARB risk–”). Here and below “ORB” and “ARB” refer to “outcome reporting bias” and “analysis reporting bias,” respectively. No further assessment is necessary.

Scenario 2—Clear Risk of Reporting Bias

If reviewers determine that an outcome X was planned but the results were not reported, or were only partially reported in study documents, then the study is at risk of reporting bias for that outcome (“ORB risk +”). Also, when reported results are based on a different analysis, effect measure, cut-off, etc. than what was prespecified, then the study is at risk of analysis reporting bias for that outcome (“ARB risk +”). No further assessment is necessary.

Scenario 3—Clear Risk of Reporting Bias

If reviewers determine that an outcome X was not planned but the results were reported, then the study is at risk of reporting bias for that outcome (“ORB risk +”). This study is also at risk of analysis reporting (“ARB risk +”) because there is no way to know whether the reported analysis was planned or post hoc. No further assessment is necessary.

Scenario 4—Reporting Bias Cannot be Ruled Out

If the reviewers are unable to determine whether an outcome X was planned, but data are reported completely or partially, then the study risk of outcome and analysis reporting bias may be categorized as “unclear”. This would also apply to a study that did not report any outcome of review interest across all source documents but was eligible on population, intervention, comparator, and other criteria. Whenever reviewers have categorized their assessment as “unclear risk of ORB,” a final assessment described below is recommended.

For studies for which the risk of reporting bias cannot be ruled out, we suggest that EPCs do one final assessment ( Figure 1 ). Reviewers should ask the question: “ Given the study objectives, duration, and other investigated outcomes, could the study have also likely measured the outcome of interest but not reported it? ” If the answer is “no” the study should be rated as “ORB risk–”. If it still remains unclear whether the outcome of interest may have been assessed, the study should be categorized as “ORB risk unclear.” Alternatively, when the answer is “yes” (e.g., another reported outcome in the study leads the reviewer to believe that outcome X would have been collected), then the study should be rated “ORB risk +” for that outcome. This should be done for all included studies for all gradable outcomes, not just those that reported outcomes data. As such it is important that systematic reviewers should not exclude studies that do not investigate or report outcomes of interest to the review without a sound rationale.

Alternatively, EPCs could also construct a matrix as described by Kirkham et al. 23 and illustrated by Dwan, et al. 115 which uses a multistep process that reviewers can use to determine if potentially eligible trial reports are prone to reporting bias (available at http://www.trialsjournal.com/content/11/1/52/table/T1 ). Briefly, the matrix:

• Includes all included studies (accompanied with all corresponding publications) irrespective of whether or not they report the review-relevant outcomes. Unless justified otherwise, studies should not have been excluded because they did not report any of the review outcomes.
• Arranges outcomes in columns and studies in rows for all included studies. The outcomes tabulated include all the review-relevant outcomes as well as outcomes that are not of review interest but are reported in included studies.
• Should differentiate complete, partial, and nonreporting for each review-relevant outcome for which the risk of reporting bias is being assessed

Combining Studies When Publication Bias or Outcome Reporting Bias is Suspected

The decision regarding whether to combine studies and how to report the result necessarily depends on the level of suspicion of bias. In some cases, the best course is to refrain from combining the available studies if it is known that a substantial amount of data that could influence results is being withheld. For example, the manufacturer Pfizer initially refused to provide data for all of its reboxetine trials for an Institute for Quality and Efficiency in Health Care (IQWiG) review. 116 – 118 Since data on only about 1,600 out of 4,600 patients were analyzed, IQWiG concluded that no statement of benefit or harm could be made. After negative publicity, Pfizer provided the data, and the subsequent IQWiG review reported no benefit of reboxetine for depression.

Assessing for Publication Bias

The funnel plot is a scatter plot of precision versus treatment effect, with a point for each study. The plot is interpreted visually with asymmetric appearance suggesting studies (presumably negative) that may not have been published. Statistical methods based on funnel plot have been proposed to detect and adjust for publication bias. However, for assessing publication bias, an international group of methodologists has recommended a very cautious and judicious approach to statistical testing for the lateral asymmetry of funnel plot. 119 Sensitivity analyses can assess whether a finding of treatment benefit is robust to differing assumptions regarding the extent of potential bias. 120 – 122 However, empirical validation of sensitivity analyses has not been possible, because the true extent of bias in any particular review is unknown. Furthermore, sensitivity methods do not help pin down the size of the effect, which varies depending on the amount of bias assumed. When sensitivity analyses are undertaken, reviewers should discuss how findings influence their confidence on review findings. When there is no avenue for discovering hidden studies and no applicable statistical method for assessing publication bias, sensitivity analyses should be considered and the potential for bias should be noted when reporting combined data.

• Reporting the Search Strategy and Results

General Guidelines

As described more fully in the chapter on Finding Evidence, 6 reviews should provide complete strategies for all indexed databases that were included in the search. Strategies should be included in the appendices of AHRQ publications, and authors should offer to include them as part of the supplementary material offered online for any journal publications. In addition, to the items described in Finding Evidence, the following information should be reported:

• If trial registries or regulatory documents are searched, a count of unpublished studies identified through the trial registries or regulatory documents should be reported.
• If authors of primary studies are contacted, the review should report the authors contacted and the associated study, the number of attempted contacts, and whether the contact was successful.
• Reports of hand searches should include the journals searched and how they were selected, and potentially relevant citations should be recorded and tracked for inclusion in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram.
• In general, whenever recommended guidelines are not followed, the review should include a rationale for that decision.

Reporting of Findings and Investigations of Reporting Bias

Systematic reviews must provide the reader with transparent and reproducible methods and results in regards to efforts to identify the risk of reporting bias. Each review requires a thoughtful, individualized approach to identifying selective outcome and analysis reporting, which must be outlined in the review, along with the rationale for that approach. Most important, the rationale for decisions to explore, or to not explore, information sources outside of the study publication should be clearly presented to the reader.

Some recommendations for avoiding and addressing outcome reporting bias can be gleaned from a tutorial on the assessment of a completed review. 115 A matrix of trials by outcomes reported can be constructed. When this is done, trials should not be excluded because they do not report, or only partially report, outcomes of interest. Instead, evidence that the missing outcomes were measured should be noted, as well as the level of suspicion that suppression was related to the results. Refraining from reporting summary estimates should be reserved for cases with a high level of suspicion of the deliberate withholding of a substantial proportion of data. Although empirical validation of sensitivity analyses has not been possible, a combination of cautious reporting and sensitivity analyses is preferable in cases where there is potential selective reporting. At a minimum, we suggest that the following steps should be described in a systematic review (in evidence tables) for included studies:

• For each gradable outcome, reviewers should report their final study outcome and analysis reporting bias risk assessments similar to their reporting of study risk of bias assessments by outcomes.
• Include the citation to the study protocol with the citations for the main study publications.
• If additional information from a trial protocol, registry, or regulatory submission documents was used to assess selective outcome or analysis reporting, describe what that specific information was and how it contributed to the identification of selective outcome or analysis reporting, and the assessment of reporting bias.
• To help readers assess the extent of outcome reporting bias, systematic reviewers should cross-tabulate trials versus reported outcomes.
• For each included study, reviewers should report the study funder or sponsor and the conflicts of interest of the study authors.
• In reviews where the existence of unobtainable studies has been verified, reviewers should express their opinion concerning the risk of publication bias.
• Finally, it will often happen that systematic reviewers will find themselves with documentation about a trial from various sources, containing varying degrees of conflicting detail. Since we cannot know which source is the more accurate, we recommend that authors of systematic reviews report when such discrepancies occur and report whether the results of sensitivity analyses suggest differences in results depending on which sets of data are included.
• Abbreviations
• Peer Reviewers
• Peter Doshi Johns Hopkins University Baltimore, MD
• Kerry Dwan University of Liverpool Liverpool, United Kingdom
• Eileen Erinoff ECRI Institute Philadelphia, PA
• Jonathan Leo Lincoln Memorial University Harrogate, TN
• Yoon Loke University of East Anglia Norwich, United Kingdom
• Andreas Lundh Nordic Cochrane Center Copenhagen, Denmark
• Natalie McGauren Institute for Quality and Efficiency in Health Care Cologne, Germany

Appendix A. Definitions of the Data Elements From ClinicalTrials.gov ( www.clinicaltrials.gov )

View in own window

Appendix B. Definitions of the Data Elements From the World Health Organization International Clinical Trials Registry Platform ( www.who.int/ictrp/network/trds/en/index.html )

Appendix c. fda web site – drugs@fda.

Abbreviations: FDA = U.S. Food and Drug Administration

Appendix D. Grant Databases

The authors are all members of the AHRQ EPC Reporting Bias Workgroup. Following are additional members of the workgroup: Donna M Dryden, PhD; Jennifer Lin, MD; Rose Relevo, MLIS, MS; Alexander Tsertsvadze, MD, MSc.

This report is based on research led by the Oregon and Ottawa Evidence-based Practice Centers (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract Nos. 290-2007-10057-I and 290-2007-10059-I). The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.

The information in this report is intended to help health care decisionmakers—patients and clinicians, health system leaders, and policymakers, among others—make well informed decisions and thereby improve the quality of health care services. This report is not intended to be a substitute for the application of clinical judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical reference and in conjunction with all other pertinent information, i.e., in the context of available resources and circumstances presented by individual patients.

This report may be used, in whole or in part, as the basis for development of clinical practice guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of such derivative products may not be stated or implied.

None of the investigators have any affiliations or financial involvement that conflicts with the material presented in this report.

Suggested citation: Balshem H, Stevens A, Ansari M, Norris S, Kansagara D, Shamliyan T, Chou R, Chung M, Moher D, Dickersin K. Finding Grey Literature Evidence and Assessing for Outcome and Analysis Reporting Biases When Comparing Medical Interventions: AHRQ and the Effective Health Care Program. Methods Guide for Comparative Effectiveness Reviews. (Prepared by the Oregon Health and Science University and the University of Ottawa Evidence-based Practice Centers under Contract Nos. 290-2007-10057-I and 290-2007-10059-I.) AHRQ Publication No. 13(14)-EHC096-EF. Rockville, MD: Agency for Healthcare Research and Quality. November 2013. www ​.effectivehealthcare ​.ahrq.gov/reports/final.cfm .

• Cite this Page Balshem H, Stevens A, Ansari M, et al. Finding Grey Literature Evidence and Assessing for Outcome and Analysis Reporting Biases When Comparing Medical Interventions: AHRQ and the Effective Health Care Program. 2013 Nov 18. In: Methods Guide for Effectiveness and Comparative Effectiveness Reviews [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008-.
• PDF version of this page (522K)

In this Page

• Definitions of the Data Elements From ClinicalTrials.gov (www.clinicaltrials.gov)
• Definitions of the Data Elements From the World Health Organization International Clinical Trials Registry Platform (www.who.int/ictrp/network/trds/en/index.html)
• FDA Web Site – Drugs@FDA

Other titles in these collections

• AHRQ Methods for Effective Health Care
• Health Services/Technology Assessment Text (HSTAT)

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Similar articles in PubMed

• Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas. [Cochrane Database Syst Rev. 2022] Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas. Crider K, Williams J, Qi YP, Gutman J, Yeung L, Mai C, Finkelstain J, Mehta S, Pons-Duran C, Menéndez C, et al. Cochrane Database Syst Rev. 2022 Feb 1; 2(2022). Epub 2022 Feb 1.
• The future of Cochrane Neonatal. [Early Hum Dev. 2020] The future of Cochrane Neonatal. Soll RF, Ovelman C, McGuire W. Early Hum Dev. 2020 Nov; 150:105191. Epub 2020 Sep 12.
• Review Dissemination and publication of research findings: an updated review of related biases. [Health Technol Assess. 2010] Review Dissemination and publication of research findings: an updated review of related biases. Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, Hing C, Kwok CS, Pang C, Harvey I. Health Technol Assess. 2010 Feb; 14(8):iii, ix-xi, 1-193.
• Assessment of publication bias and outcome reporting bias in systematic reviews of health services and delivery research: A meta-epidemiological study. [PLoS One. 2020] Assessment of publication bias and outcome reporting bias in systematic reviews of health services and delivery research: A meta-epidemiological study. Ayorinde AA, Williams I, Mannion R, Song F, Skrybant M, Lilford RJ, Chen YF. PLoS One. 2020; 15(1):e0227580. Epub 2020 Jan 30.
• Review AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program. [J Clin Epidemiol. 2010] Review AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the effective health-care program. Chou R, Aronson N, Atkins D, Ismaila AS, Santaguida P, Smith DH, Whitlock E, Wilt TJ, Moher D. J Clin Epidemiol. 2010 May; 63(5):502-12. Epub 2008 Sep 26.

Recent Activity

• Finding Grey Literature Evidence and Assessing for Outcome and Analysis Reportin... Finding Grey Literature Evidence and Assessing for Outcome and Analysis Reporting Biases When Comparing Medical Interventions: AHRQ and the Effective Health Care Program - Methods Guide for Effectiveness and Comparative Effectiveness Reviews

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

info This is a space for the teal alert bar.

notifications This is a space for the yellow alert bar.

Research Process

• Brainstorming
• Explore Google This link opens in a new window
• Explore Web Resources
• Explore Background Information
• Explore Books
• Explore Scholarly Articles
• Narrowing a Topic
• Primary and Secondary Resources
• Academic, Popular & Trade Publications
• Scholarly and Peer-Reviewed Journals
• Grey Literature
• Clinical Trials
• Evidence Based Treatment
• Scholarly Research
• Database Research Log
• Search Limits
• Keyword Searching
• Boolean Operators
• Phrase Searching
• Truncation & Wildcard Symbols
• Proximity Searching
• Field Codes
• Subject Terms and Database Thesauri
• Reading a Scientific Article
• Website Evaluation
• Article Keywords and Subject Terms
• Cited References
• Citing Articles
• Related Results
• Search Within Publication
• Database Alerts & RSS Feeds
• Personal Database Accounts
• Persistent URLs
• Literature Gap and Future Research

Web of Knowledge

• Annual Reviews
• Systematic Reviews & Meta-Analyses
• Finding Seminal Works
• Exhausting the Literature
• Finding Dissertations
• Researching Theoretical Frameworks
• Research Methodology & Design
• Tests and Measurements
• Organizing Research & Citations This link opens in a new window
• Scholarly Publication
• Learn the Library This link opens in a new window

Additional Information

• Grey Literature Guides A directory of grey literature research guides, repositories, journals and conference proceedings, and training resources.

Grey Literature Defined

Grey literature is literature produced by government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers. A widely quoted definition of grey literature is "literature which is not readily available through normal book-selling channels, and therefore difficult to identify and obtain" ( Wood, 1982 ). Grey literature includes theses and dissertations, conference papers and proceedings, reports (such as white papers, working papers, internal documentation), government documents, technical notes and specifications, proposals, datasets/statistics, policies/procedures, patents, unpublished trial data, regulatory data, speeches, urban plans, test instruments, pre-prints, company information, social media, and more. A full list of  Document Types in Grey Literature is provided by GreyNet International.

It is crucial to note where the term “grey literature” derives from. Grey literature comes from the uncertainty of the status of this information. However, in cases where there may not be much information on a topic in peer-reviewed research, grey literature may prove a very valuable source of information and introduce alternate viewpoints. Additional benefits to using grey literature include:

• Grey literature can be published much more quickly since it does not have to be subjected to the lengthy peer-review process. Results of studies may appear in gray literature 12 to 18 months before being published via traditional channels.
• Online information from organizations may be updated more frequently than traditional published journals/books.
• Some grey literature may contain more depth—for example, a dissertation may include some raw data not published in a journal article that author goes on to write.
• Grey literature may provide a broader overview of an issue/topic, such as a white paper or fact sheet.
• Minimizes reporting or publication biases; grey literature is more likely to include negative results - e.g. clinical trials.

Locating Grey Literature

Grey literature may present a number of challenges for the researcher, making it difficult to identify and find. Consider the following when searching for grey literature:

• May not be widely disseminated
• May not be published online or not stable online (URL/website may change)
• Older documents may not be archived
• Format and citation information may be inconsistent
• Volume of material may be overwhelming and consuming
• May not have an international standard book number (ISBN) or an international standard serial number (ISSN)
• Typically not peer-reviewed and quality of evidence varies

Review the Evaluating Information Guide to help determine if a resource is appropriate for use in your research. The AACODS checklist is designed to enable evaluation and critical appraisal of grey literature.

The techniques below outline suggestions for locating select types of grey literature, as well as searching more broadly online. Please contact the Library with any questions.

Grey Literature Types

• Conference Proceedings
• Dissertations & Theses
• Government Documents
• Research Reports
• Social Media

Clinical trials are a form of grey literature  and can inform current research conducted by organizations, Federal agencies like the National Institutes of Health, academic institutions, and individual health care providers. These studies investigate the effectiveness of new treatments, interventions, drugs, procedures, and devices in order to improve health outcomes for a specific population. 

Several Library databases and online resources provide access to full-text clinical trials. See our Clinical Trials Guide for detailed instructions.

If you are unable to locate the full text for a particular clinical trial, you may submit an Interlibrary Loan request for the material.

A conference proceeding is the published record of a conference, congress, symposium, or other meeting sponsored by professional association or society. Proceedings typically include abstracts or reports of papers presented by the participants. When the entire  text of the papers presented is included, they are called transactions. For more information, see the FAQ: Are conference proceedings peer reviewed?

Conference papers and proceedings can be challenging to find because they may take several years to be published or may not be published at all. They can be published in various formats including books, abstracts, and journal articles; and, they may be deposited only in an author or institutional repository.

Some Library databases, like ProQuest , make it easier to search for these types of documents by allowing you to limit your search to Source type. Other databases that provide conference proceedings include IEEE Xplore, ACM Digital Library, LearnTechLib, CINAHL, and ERIC. These databases are available from the A-Z Databases page.

Previously published dissertations and theses can be a great source of inspiration for your own dissertation topic. You can access millions of full-text dissertations and theses from the Library. Go to the Library’s Dissertation Resources page to see a list of databases. You can specifically access dissertations from alumni via ProQuest Dissertations and Theses @ Northcentral University. Or you can explore the 3 million full-text dissertations available through ProQuest Dissertations and Theses Global.

For additional information and resources, see the Finding Dissertations Guide .

Government documents are an important primary source of information on a wide range of issues. Most government documents can be found through official government websites.

• Catalog of U.S. Government Publications Finding tool for federal publications that includes descriptive information for historical and current publications as well as direct links to the full document, when available.
• Data.gov The home of the U.S. Government's open data. Here you will find data, tools, and resources to conduct research, develop web and mobile applications, design data visualizations, and more.
• Electronic Code of Federal Regulations The codification of the general and permanent rules published in the Federal Register by the departments and agencies of the Federal Government produced by the Office of the Federal Register (OFR) and the Government Publishing Office.
• Federal Registrar Official journal of the federal government of the United States that contains government agency rules, proposed rules, and public notices.
• govinfo Provides free public access to official publications from all three branches of the Federal Government.
• HathiTrust A collection of U.S. Federal Documents as identified via the HathiTrust Federal Government Documents Registry.
• MetaLib Federated search engine that searches multiple U.S. Federal government databases, retrieving reports, articles, and citations while providing direct links to selected resources available online.
• Pueblo.GPO.gov Operated by the U.S. Government Publishing Office Pueblo Distribution Center (US GPO PDC) to support free federal publication distribution activities.
• U.S. Government Bookstore Allows users to search for and order Government information products currently available from the Information Dissemination area of the U.S. Government Publishing Office.
• U.S. Government Printing Office (GPO) Keeping America Informed as the official, digital, and secure source for producing, preserving, and distributing official Federal Government publications and information products for Congress, Federal agencies, and the American public.
• USA.gov USA.gov is the official U.S. portal to federal, state, local, tribal, and international government information and services.

A preprint is an author's working version of a research manuscript prior to publication. In most cases, they can be considered final drafts. Preprints may also be referred to as working papers or unpublished manuscripts. Preprints have not gone through the peer-review process, nor have they been improved upon by the publisher (e.g., formatting, copy-editing, technical enhancements). It is very important to critically evaluate preprint publications. Because the peer review process can take three to six months, preprints are a way to provide valuable research results and discussion ahead of publication. They are also a great way to locate information on emerging or rapidly changing research topics.

Preprints are published electronically and made publicly available on large databases or preprint repositories. The techniques below describe how to locate preprints in the Libray and online. Note when searching for preprints in Library databases, you do not want to select the peer-reviewed journal limiter.

EBSCO Business Source Complete

In Business Source Complete , select the Advanced Search screen. Scroll down to Publication Type and select Working Paper .

ProQuest Central

In ProQuest Central , select the Advanced Search screen. Scroll down to the Source Type limiter and select Working Papers .

PubMed Central

In PubMed Central, select the Advanced Search Builder . Enter the preprint[filter] command along with your topic keywords.

After conducting a search in Web of Knowledge (Web of Science), check the Early Access box under the Quick Filters on the left-hand side of the screen.

Preprint Repositories

• Advance: a SAGE preprints community Allows researchers within the fields of humanities and social sciences to post their work online and free of charge.
• arXiv Open access to more than a million e-prints in Physics, Mathematics, Computer Science, Quantitative Biology, Quantitative Finance and Statistics.
• Authorea Browse over 20,000 multi-disciplinary research preprints.
• bioRxiv Free online archive and distribution service for unpublished preprints in the life sciences.
• ChemRxiv Free submission, distribution, and archive service for unpublished preprints in chemistry and related areas.
• Figshare Repository where users can make all of their research outputs available in a citable, shareable and discoverable manner.
• medRxiv Free online archive and distribution server for complete but unpublished manuscripts (preprints) in the medical, clinical, and related health sciences.
• OSF Preprints (Open Science Framework) Free, open platform to support research and enable collaboration.
• PLOS Preprints Nonprofit, Open Access publisher empowering researchers to accelerate progress in science and medicine by leading a transformation in research communication.
• Preprints Preprints is a multidisciplinary preprint platform that accepts articles from all fields of science and technology.
• PsyArXiv A free preprint service for the psychological sciences.
• Research Square Research Square is a multidisciplinary preprint and author services platform.
• Rxivist Combines biology preprints from bioRxiv and medRxiv with data from Twitter to help researchers find the papers being discussed in their field.

Research reports contain the results of research projects, investigations, and surveys, and are usually published by the funder or the body undertaking the research. They can be found by searching the websites of subject associations and research organizations in addition to NU Library’s databases.

For example, you may limit your NavigatorSearch results to reports by using the limiters on the left-hand side of your results screen.

Social media and other Web 2.0 applications have become a valuable medium for for experts, governments and academics to publish new information and analysis outside of the traditional publishing arrangement. These sites may be useful as a way to disseminate information/results, follow experts and trending topics, and as a forum for exchange of ideas.

For a thorough overview of Web 2.0 grey literature sources, please see the Grey Literature: Grey 2.0 Guide from New York University.

Grey Literature Repositories

Grey literature repositories are curated collections of grey literature sources. You can use these collections to search or browse for resources for use in your research.

• BASE One of the world's most voluminous search engines especially for academic web resources. Note: Not exclusively grey literature.
• CORE World’s largest collection of open access research papers. Note: Not exclusively grey literature.
• Directory of Open Access Journals (DOAJ) Provides full-text, quality controlled scientific and scholarly, peer-reviewed journals in many subjects and languages. Note: Not exclusively grey literature.
• Grey Literature Publishers List The report is a publication produced by the The New York Academy of Medicine between 1999 - 2016, alerting readers to new grey literature publications in health services research and selected urban health topics.
• Grey Literature Report The report is a bimonthly publication of The New York Academy of Medicine alerting readers to new grey literature publications in health services research and selected urban health topics.
• GreySource Provides examples of grey literature and in so doing profiles organizations responsible for its production and/or processing.
• OAIster Searches across thousands of libraries and scholarly websites for open access content. Note: Not exclusively grey literature.
• OpenGrey Search portal for European grey literature covering Science, Technology, Biomedical Science, Economics, Social Science and the Humanities.
• Science.gov Gateway to U.S. Federal Science. Search over 60 databases and over 2,200 selected websites from 15 federal agencies, for research and development results.
• Google Custom Search
• Domain Limiter
• Document Type Limiter

There are a couple of Google custom searches that narrow your results to a specific type of organizational website. This may be helpful step when searching for grey literature. Enter your search terms into these pre-built basic search boxes to see your filtered results.

• NGO Search Google Custom Search Engine that searches across hundreds non-governmental organization (NGO) websites.
• IGO Search Google Custom Search Engine that searches across IGO websites--International governmental organizations made up of more than one national government. IGO examples include NATO (North Atlantic Treaty Organization) and the WHO (World Health Organization).

Google allows you to limit your search results to a particular domain (e.g., .edu, or .gov). This can be particularly helpful if you are looking for grey literature as you are more likely to find reliable content from organizations rather than commercial content.

Look for the "site or domain" box in Google's Advanced Search options and enter the domain you'd like to search, as shown below.

You can also do this by adding site:.edu (or .org, .gov, etc) to the end of your search terms in any Google search box. For example, to find articles about “ethical leadership” published on government websites, enter the terms "ethical leadership" site:.gov, as shown below.

Google also allows you to search for results with a specific file type (example: .pdf, .doc, .ppt, .xls). This is also helpful when looking for grey literature as government and organizational reports are more likely to be available as PDF or Word documents. Datasets may be provided as Excel files.

Look for the "file type" box in Google's Advanced Search options and select the file type you would like to search, as shown below.

You can also do this by adding filetype:pdf (or .doc, .ppt, .xls, etc.) to the end of your search terms in any Google search box. For example, to find PDF documents about “ethical leadership” enter the terms "ethical leadership" filetype:pdf, as shown below.

Was this resource helpful?

• << Previous: Scholarly and Peer-Reviewed Journals
• Next: Clinical Trials >>
• Last Updated: Mar 4, 2024 3:32 PM
• URL: https://resources.nu.edu/researchprocess

© Copyright 2024 National University. All Rights Reserved.

Privacy Policy | Consumer Information

Grey Literature

• Introduction

About Grey Literature?

Grey literature: pros and cons, some examples of grey literature, introductory video from the university of guelph.

• Finding Grey Literature
• Searching effectively
• Getting Help

The term  grey literature  is used to describe a wide range of different information that is produced outside of traditional publishing and distribution channels, and which is often not well represented in indexing databases.

A widely accepted definition in the scholarly community for grey literature is

"information produced on all levels of government, academia, business and industry in electronic and print formats not controlled by commercial publishing" ie. where publishing is not the primary activity of the producing body." 

From: Third International Conference on Grey Literature in 1997 ( ICGL Luxembourg definition, 1997  - Expanded in New York, 2004).

Uniqueness and currency v quality and longevity

Grey Literature can be a very important research resource.

• It can record findings in niche or emerging research areas and also record research findings that produce null or negative results.  These may not be addressed by the commercial publishers - who may have a more mainstream, profit based publishing strategy.
• It may be more current than formally published research literature which can take time to pass through a sometimes lengthy peer review and editorial publishing process.
• It can connect you with content from a more diverse range of authors and institutions.  Not everyone is able to publish through commercial publication routes

You should be aware of the need to assess and capture potentially useful resources

• Grey Literature sources can vary hugely in terms of quality.
• Scholarly publications such as books and journals pass through a formal publishing process and are subject to in depth editing review. In many cases there is a peer review process where draft versions are subject to a scrutiny process by a panel of experts.  Papers may be modified and improved in light of the expert feedback before reaching final published status. 
• Not all grey literature material is subject to a similarly rigorous pre publication review process, so you should exercise caution and make your own assessment for quality, reliability and potential bias.  See the Evaluating Sources  tutorial for more guidance on this.
• Grey literature is often not formally published. For example, a report may be posted to a government website or a poster presentation may be given at a conference. In these cases you need to consider the longevity of the resource.  Something may be available on the web, or a blog for a short period only and may not be formally archived. 
• Make sure you keep a record of material you wish to use/reference - as it may not be there for discovery later down the line.

A wide, and growing, range of material can be considered as grey literature.  Not all these examples will be relevant to all researchers.    For example, clinical trial information is primarily of interest to health and medical research.  Business researchers will find company and market research information particularly useful.

You should consider which types of information you are interested in before you begin searching as this will help you target and frame your online searching.

Examples include:

• Clinical trials
• Company Information
• Conference papers/proceedings
• Discussion Forums
• Dissertations and theses
• Email discussion lists
• Government documents and reports
• Market reports
• Newsletters
• Policy statements
• Pre print articles
• Press releases
• Research reports
• Statistical Reports
• Survey results
• Working papers

If you prefer an audiovisual introduction, take a look at this short video, produced by the University of Guelph Library.

• << Previous: Introduction
• Next: Finding Grey Literature >>
• Last Updated: Mar 12, 2024 2:36 PM
• URL: https://libguides.exeter.ac.uk/greylit

• University of Michigan Library
• Research Guides

Grey Literature

• Strategies for Finding Grey Literature
• Conferences
• Clinical Trials/Regulatory Data
• Reports & Reviews
• Other Resources

What's on this page

This page provides strategies for finding grey literature. Grey literature is an important part of the evidence base for health services research but can be difficult to search in a systematic way. You'll find three core strategies for identifying this type of content.   See the  Grey Literature Research Guide for many other resources

#1 Going directly to organization   l  #2 Searching Portals Aggregating Grey Literature |  #3 Google Advanced Searches  

Strategy #2: Portals

Below are some resources that can help you identify organizations and/or search directly for grey literature.

• Overton.io This link opens in a new window Provides the world’s largest collection of policy documents, guidelines, think tank publications and working papers.
• Dimensions Plus This link opens in a new window Brings together various research-related data sources, such as grant information, books, journal articles, conference proceedings, and patents.
• Grey Literature Producing Organizations A list of grey literature producing organizations from the New York Academy of Medicine.
• Kennedy School of Government Think Tank Search Searches more than 700 think tanks and research centers. searches the websites of institutions that generate public policy research, analysis, and activity. These sites are affiliated with universities, governments, advocacy groups, foundations, non-governmental organizations, and businesses.
• NLM Digital Collections National Library of Medicine's free online repository of biomedical resources including books, manuscripts, still images, videos, and maps. The content in Digital Collections is freely available worldwide and, unless otherwise indicated, in the public domain.
• Policy Atlas (Harvard Catalyst) Helps identify data sources that can be used for your policy-relevant research or to discover new sources for health data
• Policy Commons Policy Commons provides more than 2.8 million+ policy reports, briefs, analyses, and datasets from IGOs, NGOs, and think tanks.

Strategy #3: Google Advanced Searches

Google offers a great way to track down grey literature resources. Use the advanced search limits below to focus your search.

Limit to a domain:

• site:org  

Find Related Sites:

• related:urban.org

Limit to filetype:

• filetype:pdf (or doc, ppt)

Search for words in a URL:

• inurl:rand.org

Search for sites that link to site of interest:

• link to:academyhealth.org 

Find quick information about a site:

• info:kff.org

Strategy #1: Going Directly to Organization's Website

Below are some top non-governmental organizations & federal agencies working in the health sciences. Since there is a seemingly endless number of organizations that produce grey literature, searching portals & using Google Advanced searches to identify names can be an effective way to find grey literature when you don't know where to begin.

Non-Governmental Organizations

• Academy Health
• Advisory Board
• American Hospital Association
• Center for Advancing Health
• Center for Health Equity Research & Promotion collaborations
• Center for Responsive Politics: Open Secrets
• Commonwealth Fund more... less... A private foundation whose mission is "to promote a high performing health care system." Browse the publications, maps & data and surveys. Search the entire site. Example: "health care" AND (racial OR ethnic).
• Cross Cultural Health Care Program more... less... The mission of the Cross Cultural Health Care Program is to serve as a bridge between communities and health care institutions to ensure full access to quality health care that is culturally and linguistically appropriate.
• Dartmouth Atlas of Health Care more... less... A project of the Dartmouth Institute for Health Policy and Clinical Practice, this site offers data national, regional and some local data on health care issues with mapping tools and publications.
• Henry J. Kaiser Family Foundation
• Joint Center for Political and Economic Studies
• National Center for Cultural Competence more... less... From Georgetown University's Center for Child & Human Development.
• National Quality Forum more... less... The National Quality Forum is a nonprofit organization that works to improve the quality of American healthcare.
• New York Academy of Medicine
• Prevention Institute
• Public Health Law Research
• RAND Corporation more... less... The RAND Corporation is a nonprofit institution that focuses on improving policy surrounding various issues through research and analysis.
• Robert Wood Johnson Foundation more... less... This is the largest philanthropy in the U.S. devoted to issues of public health. The web site offers research reports on issues related to health care and health policy.
• Urban Institute more... less... The Urban Institute works to foster sound public policy and effective government through research and analysis.

Government Agencies (Federal)

• Agency for Healthcare Research & Quality (AHRQ)
• Centers for Disease Control (CDC)
• Department of Energy
• Department of Health & Human Services (DHHS)
• Health Resources & Services Administration (HRSA)
• National Institutes of Health
• National Science Foundation

Reference management. Clean and simple.

What is grey literature? [with examples]

What's the importance of grey literature?

What are some of the drawbacks of grey literature, what are some examples of grey literature, how do you identify grey literature, where can you search for grey literature, are there any other ways to find grey literature, use search engines, go directly to organizations' websites, contact researchers directly, frequently asked questions about grey literature, related articles.

Grey literature refers to materials and research published specifically outside of the traditional commercial, academic publishing, and distribution channels.

It is produced by organizations on all levels of government, academia, business, and industry and either published informally or remains unpublished. Not every grey literature database will have undergone the peer review process due to this informal setup, but the information contained therein can still prove valuable to researchers and academics alike - if they are able to find it.

Since grey literature usually comes from multiple sources, finding the right resource can prove to be challenging. It's not as simple as visiting a library. With no indexes or formal organization, grey literature won't appear in academic sources like books or journals. But the question still remains, what is grey literature and why is it important to academic publishing?

• Grey literature is often the best source for up-to-date research on certain topics. This is because it doesn't have to go through the often lengthy process of publication before it can be reviewed.
• In the same breath, grey literature databases can have a scope that includes recently emerging areas of research.
• Scientific publishing, for instance, has a strong bias toward publishing research that shows a significant positive correlation. Many studies show no effect and can go unpublished, but negative results are just as important to note. Accessing grey literature fills this knowledge gap, and generates a more balanced picture of the available evidence.
• The reports produced in grey literature are often more detailed in nature and can contain raw data sets not available anywhere else - the caveat is that they come in tons of different formats and document types, making the data slightly more difficult to find.
• Grey literature can help researchers to reduce publication bias.
• Most grey literature is free (one exception is materials produced by market research firms, who often charge for access). This makes it a lot more accessible to everyone.
• The information in grey literature hasn't gone through the usual publishing channels, so it isn't rigorously and formally peer-reviewed. This means you'll need to carry out your own checks to assess its reliability.
• When grey literature isn't published, it's free from publication bias. However, it may carry its own set of biases. The organization that distributed the grey literature can have its own political, social, or financial motivations for promoting unreliable research. Make sure to cross-check information from grey literature against studies found elsewhere.
• Grey literature is produced from a great variety of sources, so it's usually not indexed or organized in any way. That can make it very difficult to locate what you're looking for.
• Librarians often have a difficult time acquiring grey literature and making it accessible.

Publication types of grey literature are almost endless, and include:

• Government reports
• Technical reports
• Unpublished clinical trials
• Conference abstracts
• Graduate dissertations and theses
• Newsletters
• White papers
• Working papers
• Dissertations

Websites, manuals, statistics, and datasets are technically also types of grey literature that you may come across during the research process.

Grey literature can be found in so many contexts it can be tough to list them all. Essentially, if you didn't find it in a renowned journal or published book, there's a good chance it's grey literature. Luckily, if you know how to find academic sources, you should be able to identify grey literature in the wild.

Here are some other indicators that you may be looking at a piece of grey literature:

• It was difficult for you to find
• It hasn't been widely distributed
• It contains more information on the subject than you've seen elsewhere, as there weren't publication length stipulations
• It was produced and disseminated quickly
• It hasn't been archived

It can be difficult to develop a systematic approach to finding grey literature for academic purposes. Unlike scholarly journals that are often available in electronic formats, grey literature databases will be all over the map in terms of formats and accessibility.

You should prepare any search terms you're going to use, identify the ones that are most applicable, and record the sources as you search. If you have a research question, use it to guide where you look for the sources you're going to search.

There are also many grey literature databases that specialize in cataloging grey literature:

There are methods that you can make use of to target your search for grey literature.

• Restrict content to only governments or organizations by typing in your search term and then either 'site:.org' or 'site:.gov'.
• You can take it further and limit your search to e.g. UK government sites only by typing 'site:.gov.uk'.
• Restrict content to a specific file type by typing in your search term and then e.g. 'filetype:pdf' or 'filetype:doc'.
• If you're looking for grey literature from a specific country, use their version of Google, for example, Google Australia or Google UK .
• If you'd like a fresh take, try search engine DuckDuckGo , as it doesn't track location or user searches. This means it doesn't filter results based on your user profile.

Identify organizations that might publish technical papers and reports on the subject you're looking for. These could be government agencies, non-profits, research institutes, pressure groups, international organizations, or professional associations.

For example, the World Health Organization (WHO) has several online collections of grey literature and the World Bank makes all of their publications available online for free.

Search your chosen organizations' websites, looking for sections of the site with names such as 'Documents', 'Reports' or 'Library'.

Research known researchers in your field of interest. You can then contact them to kindly ask if they're aware of any unpublished or ongoing studies - more often than not they will be happy to point you in the right direction.

Grey literature is materials and research that have been published specifically outside of the traditional commercial or academic publishing and distribution channels. The term 'grey' refers to the uncertainty of the status of this information.

Yes, preprints are considered grey literature, as they have not undergone the peer-review process.

PubMed is a research database, not a publication type. Material indexed by major databases such as PubMed is generally not considered grey literature.

Books and other scholarly publications have to pass through a formal publishing process and are therefore not considered grey literature.

Yes, dissertations are generally considered grey literature and might be hard to obtain in some cases.

• Research Guides

Grey Literature

• What is Grey Literature? Activities
• Why Use Grey Literature?
• Types of Grey Literature
• Sources of Grey Literature
• Searching for Grey Literature
• Evaluating Grey Literature Activity
• How to Incorporate & Cite Grey Literature

This guide includes content adapted with permission from the University of Illinois Library .

How to Search for Grey Literature

Grey literature, characterized by its rapid production and diverse array of originating entities, offers a valuable resource for accessing contemporary and contextually relevant information. However, the decentralized nature of its dissemination poses challenges to its discovery compared to conventional scholarly works. Unlike scholarly journals and books, which benefit from professionals' systematic cataloging, organization, and preservation efforts, grey literature typically lacks standardized cataloging procedures. Consequently, its accessibility to researchers is hindered by inadequate or inconsistent cataloging practices among producing organizations. This deficiency in cataloging infrastructure is further exacerbated in the realm of digital documentation or materials from defunct organizations, where preservation mechanisms are often absent.

Moreover, grey literature's departure from traditional metadata standards compounds the difficulty in locating cited documents, even when citations are accurate. Compounding these challenges, the absence of a centralized repository for grey literature necessitates researchers to navigate disparate repositories maintained by individual organizations to access pertinent materials. Thus, while grey literature offers a wealth of timely information, its decentralized nature and lack of standardized cataloging mechanisms render it more elusive than its scholarly counterparts.

Tips for Searching for Grey Literature

• Citation Chasing
• Advanced Google Searching: Site Domain
• Advanced Google Searching: File Type
• Directories

During academic inquiry, researchers frequently encounter scholarly works incorporating references to or citations from grey literature. This practice expands the breadth of available literature on a given subject and facilitates the identification of entities responsible for disseminating such materials. Notably, engaging in citation chasing of this nature underscores the perceived authority of the respective grey literature-producing organizations within the subject domain, as evidenced by their incorporation into scholarly discourse. Furthermore, citation tracing aids in discovering pertinent resources, given the close thematic alignment between cited sources and the focal content of the referencing article.

For instance, the article titled " Global Climate Implications for Homelessness: A Scoping Review ," published in the Journal of Urban Health, references various sources, including two studies or reports attributed to the United Nations, illustrate the integration of both scholarly and grey literature. While the primary focus remains on scholarly discourse, such citations enrich the scholarly dialogue by incorporating insights from grey literature sources. Additionally, noting the originating organizations during research endeavors enhances the researcher's awareness and potential utilization of such sources in future research.

The Google Advanced Search differs significantly from the simple default Google Engine Search. There are many different search fields available in this Advanced Search, with one of the most convenient areas for grey literature being the site or domain field. If there is a singular site you are looking to complete a search on, it can be done here. Similarly, if you want to only look for information on U.S. government websites, it is possible to complete a search of only ".gov" websites. Type one of the following into the search bar:

• Site:gov search words​ or phrases
• Site.edu search words​ or phrases
• Site.org search words or phrases

Again,  Google Advanced Search  has many different search fields available in this Advanced Search. Another important area to point out for researching Grey Literature is the file type. You can limit search results to only one file format, such as .pdf or .xls, by choosing a specifier on the Advanced Search page. When searching for a specific kind of resource, this can be extremely helpful. It is also nifty when you just want to parse down your search a little more! Type one of the following into the search bar:

Filetype:pdf​

Filetype:ppt​

Filetype:svg​

Filetype:xls OR xlsx

Directories are lists of individuals or organizations, typically grouped or organized thematically. Directories are beneficial in identifying different organizations that may produce grey literature in the field of research you are interested in. Each directory depends on its coverage, so pick the appropriate directory for your topic. Remember that many directories can become outdated quickly, so check for updated or new web addresses or changes in the organization's name, missions, programs, or goals.  

The Open Think Tank Directory is an excellent example of a general searchable directory. As you can see below, it is also searchable by region, which can help in many areas of Global Studies research.

Tools for Searching for Grey Literature

• Custom Google Search Bars
• General Sources
• FIND POLICY A side project of Transparify, search think tank sites grouped by topic and location.
• Open Think Tank Directory A global collection of 2700 think tanks and related organizations.
• GreySource GreySource provides examples of grey literature to the average net-user and in so doing profiles organizations responsible for its production and/or processing. GreySource identifies the hyperlink directly embedded in a resource, thus allowing immediate and virtual exposure to grey literature.
• Grey Guide portal and repository for resources in the field of grey literature.
• OpenGrey A search portal for European grey literature. It covers Science, Technology, Biomedical Science, Economics, Social Science and the Humanities.
• World Association of Non-Governmental Organizations (WANGO) Worldwide NGO Directory Search for NGO's based on location.
• OpenDOAR OpenDOAR is the quality-assured, global Directory of Open Access Repositories. You can search and browse through thousands of registered repositories based on a range of features, such as location, software or type of material held
• UNESCO Database of Non-Governmental Organizations (NGOs) Contains the profiles of non-governmental organizations that have official relations with UNESCO.
• UN NGO Branch - Integrated Civil Society Organizations System The integrated Civil Society Organizations (iCSO) System, developed by the Department of Economic and Social Affairs (DESA), facilitates interactions between civil society organizations and DESA. The system provides online registration of general profiles for civil society organizations, including address, contacts, activities and meeting participation.
• Harvard Kennedy School of Government Think Tank Search A Google Custom Search of more than 1200 think tanks and research centers.
• Non-Governmental Organizations Custom searches to find information from NGOs created by Government Documents Round Table (GODORT)
• JSTOR Open Research Reports JSTOR hosts a growing curated collection of more than 45,000 open research reports from 175 policy institutes from around the world. These publications are freely accessible to everyone on JSTOR and discoverable as their own content type alongside journals, books, and primary sources. We update research reports on our platform each month as they become available through contributing institutes.Research reports provide current analysis on many of today’s most discussed and debated issues from a diversity of ideological and international perspectives representing 39 countries and 29 languages. A sample of topics would include climate change, border security, fake news, cybersecurity, electric vehicles, artificial intelligence, energy policy, gender issues, terrorism, remote learning, and various public health issues, including COVID-19.
• << Previous: Sources of Grey Literature
• Next: Evaluating Grey Literature >>
• Last Updated: Feb 14, 2024 11:35 AM
• URL: https://researchguides.library.vanderbilt.edu/greyliterature

Systematic Reviews: Searching the Gray Literature

• What Type of Review is Right for You?
• What is in a Systematic Review
• Finding and Appraising Systematic Reviews
• Formulating Your Research Question
• Inclusion and Exclusion Criteria
• Creating a Protocol
• Results and PRISMA Flow Diagram
• Searching the Published Literature
• Searching the Gray Literature
• Methodology and Documentation
• Managing the Process
• Scoping Reviews

How to Manage the Grey Literature Search

• Identify and record the sources you will search. The sources you search will be informed by your research question and where you expect to find information related to your question.
• Document  where you are searching and your search strategies, including document resource name, URL, search terms, and date searched.
• Collect  citation information as you go.
• Adhere  to your established inclusion and exclusion criteria when selecting sources.

Google Scholar

Google Scholar has it's place in research. That being said, GS is not great for doing a systematic review. The filters are not good, you can't build an advance search, and the Google algorithm is unknown. It can be used for gray literature searching through. Do a simplified search in GS and use the first 300 results or the first few pages of results. 

The gray literature: What is it and how do I search it?

What is gray literature? 

Gray literature is defined by the  Cochrane Handbook for Systematic Reviews of Interventions  as "...literature that is not formally published in sources such as books or journal articles." This can include information such as government reports, conference proceedings, graduate dissertations, unpublished clinical trials, and much more. The sources you select will be informed by your research question and field of study, but should likely include, at a minimum, theses and dissertations.

Why search the gray literature if it hasn't been peer-reviewed? 

The intent of an evidence synthesis is to synthesize  all available evidence  that is applicable to your research question.

There is a strong bias in scientific publishing toward publishing studies that show some sort of significant effect.  Meanwhile, many studies and trials that show no effect end up going unpublished.  But knowing that an intervention had no effect is just as important as knowing that it did have an effect when it comes to making decisions for practice and policy-making.  Thus, the gray literature can be critical.

How do I search the gray literature if it's unpublished and not in the scholarly databases?  

Its true--finding gray literature and searching it systematically is challenging.  But there are a few approaches that you can take to add some structure to your search of this type of information:

• Identify and record  the sources you will search, including some indication of search terms used if appropriate.  The sources you search will be informed by your research question and where you expect to find information related to your question.
• Try searching  databases that specialize in gray literature  like  OpenGrey .  See the box below for more information.
• Conference proceedings :  Identify professional organizations that have conferences at which researchers might be presenting work related to your topic.  Search those conference proceedings on the organization's website or by contacting organizational boards for access to past proceedings that may not be online.
• Theses and dissertations:   There are a number of databases dedicated to theses and dissertations, which you can search using your search terms.  See the box below for links to these resources. 
• Identify government agencies, and international and non-governmental organizations , that might publish technical papers and reports on your topic.  Search their websites or any online libraries that they may provide.  For example, the  WHO  has a number of searchable online collections and the  World Bank  now makes all of their publications openly available online.
• Contact known researchers  in the field to determine if there are any ongoing or unpublished studies that s/he may be aware of.
• Grey Literature Reporter The Grey Literature Reporter is a Google Chrome extension that you train on the search results of the current website that you are on. It will then record the results – as JSON, CSV and potentially RIS – as well as the URL of the site and the search string used. With this extension you can easily start your search, click on the extension and tell it where the 1st result item is, tag all the relevant data in that result and automatically scrape the rest of the results and pages. Even if the markup changes for the site the next time you go to it will not matter as the extension is shown each time what elements on the page you’re interested in.

Grey Literature Sources

• Guidance Documents for Gray Literature Searching
• Gray Literature Databases
• Thesis and Dissertations
• Clinical Trials
• Public Policy
• Government Documents
• Other Resources
• Grey Matters: a practical tool for searching health-related grey literature This online manual provides a thorough list of sources for gray literature in medicine and a helpful checklist to help systematize your process.
• Simon Fraser University Searching Grey Literature Guide
• Duke University Medical Center Guide to Resource for Searching the Gray Literature A more thorough guide to gray literature, including resources for trial registries, pharmacological studies, conference abstracts, government document and more.
• Gray Literature Resources for Agriculture Evidence Syntheses
• OpenGrey: System for Information on Grey Literature in Europe The System for Information on Grey Literature in Europe, is an open access database to 700.000 bibliographical references of gray literature produced in Europe and allows you to export records and locate the documents in many research disciplines.
• NY Academy of Medicine Grey Literature Report This report is a bimonthly publication of The New York Academy of Medicine (NYAM) alerting readers to new gray literature publications in health services research and selected public health topics. The database platform is keyword searchable and serves as an archive for the cataloged reports.
• WHO Library Database The WHO library database includes governing documents, reports and technical documentation.
• MedNar MedNar searches across more than 60 medical research sources, including commercial databases, medical societies, NIH resources, and other government resources.
• Center for Research Libraries A resource for institutions outside of the U.S. and Canada.
• Dissertations & Theses @ Louisiana State University This link opens in a new window This database gives access to the dissertations and theses produced by students at your institution.
• EThOS Electronic Theses Online Service contains doctoral theses from candidates in the UK.
• LSU Scholarly Repository This link opens in a new window LSU Scholarly Repository provides access to research, publications, data, and records produced by LSU faculty, students, and units. It includes but is not limited to online versions of various theses and dissertations from LSU graduate students. To learn more about contributing, contact [email protected].
• Networked Digital Library of Theses and Dissertations (NDLT) Free international resource for theses and dissertations.
• Open Access Theses and Dissertations (OATD) OATD.org aims to be the best possible resource for finding open access graduate theses and dissertations published around the world. Metadata (information about the theses) comes from over 1100 colleges, universities, and research institutions. OATD currently indexes 4,444,117 theses and dissertations.
• ProQuest Dissertations & Theses A&I This link opens in a new window Contains citations to 1.2 million dissertations and masters theses. Coverage begins in 1861, with abstracts available since 1980, and thesis abstracts since 1988.
• Australia New Zealand Clinical Trials Registry Online register of clinical trials being undertaken in Australia, New Zealand and elsewhere. The ANZCTR includes trials from the full spectrum of therapeutic areas of pharmaceuticals, surgical procedures, preventive measures, lifestyle, devices, treatment and rehabilitation strategies and complementary therapies.
• ClinicalTrials.gov The US registry for clinical trials. Includes new, ongoing and completed human clinical trials both in the US and in countries around the world.
• Cochrane CENTRAL Register A highly concentrated source of reports of randomised and quasi-randomised controlled trials. The majority of CENTRAL records are taken from bibliographic databases (mainly MEDLINE and Embase), but records are also derived from other published and unpublished sources.
• EU Clinical Trials Register The European Union Clinical Trials Register allows you to search for protocol and results information on interventional clinical trials that are conducted in the European Union (EU) and the European Economic Area (EEA) and clinical trials conducted outside the EU / EEA that are linked to European paediatric-medicine development.
• WHO International Clinical Trials Registry Platform (ICTRP) This Clinical Trials Search Portal provides access to a central database containing the trial registration data sets provided by numerous international registries. It also provides links to the full original records.
• Wikipedia clinical trial registries list This Wikipedia entry contains a list of links to individual clinical trial registries by country.
• Health Research Web A wiki with descriptors of national health research systems, ethics review committees, local and regional policies and research priorities and more.
• Pew Research Center Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping America and the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research.
• Policy Agendas Project The Policy Agendas Project collects and organizes data from various archived sources to trace changes in the national policy agenda and public policy outcomes since the Second World War, including congressional hearing, roll call votes, executive orders, supreme court cases etc.
• Policy Archive Online archive of public policy research. Access open to all Internet users.
• Think Tanks (Harvard Kennedy School Library) Harvard's Kennedy School of Government Library compiles a list of links to public policy think tanks. Access open to all Internet users.
• WHO Institutional Repository for Information Sharing (IRIS) Institutional WHO database of intergovernmental policy documents and technical reports. Can search by IRIS by region (Africa, Americas, Eastern Mediterranean, Europe, South-East Asia, Western Pacific).
• World Bank ​The World Bank now makes all of their publications openly available online.
• Catalog of U.S. Government Publications (GPO Monthly catalog) Provides citations to U.S. federal government publications. Contains references to books, reports, studies, serials, maps and other publications. Topics covered include finance, business, demographics, foreign relations, public health and social issues.
• Congress.gov Offers full-text bills and bill tracking back to the 93rd Congress (1973) and also Congressional committee information.
• Congressional Research Service Reports This collection provides the public with access to research products produced by the Congressional Research Service (CRS) for the United States Congress. By law, CRS works exclusively for Congress, providing timely, objective, and authoritative research and analysis to committees and Members of both the House and Senate, regardless of political party affiliation.
• CQ Researcher Plus Archive This link opens in a new window A database of in-depth, authoritative reports on a full range of political and social-policy issues extending back to 1923. Each report is footnoted and includes an overview, background section, chronology, bibliography and debate-style pro-con feature, plus tools to study the evolution of the topic over time.
• CQ Weekly This link opens in a new window A weekly news magazine featuring in-depth reporting on public policy, politics, congressional legislation, and elections extending back to 1983, including: a complete wrap-up of news on Congress, the status of bills in play, behind-the-scenes maneuvering, committee and floor activity, debates, and all roll-call votes.
• GovInfo.Gov Provides online access to the U.S. Government Printing Office. This service was mandated by Public Law 103-40 (June 8, 1993), "The Government Printing Office Electronic Information Access Enhancement Act of 1993," to make the Federal Register and other Congressional information available to the public.
• Hayley Johnson Head of Government Documents & Microforms
• ProQuest Congressional Publications This link opens in a new window Description: Provides access to Committee Prints & Miscellaneous Publications, Congressional Research Service (CRS) Reports, and Congressional Hearings (1824-2003)

Googling the Greys: Tips for Searching Beyond Health Databases and Turning Information into Insights

A thorough grey literature search should involve a general sweep of the web by using different search engines. Google is an important search engine but other search engines, such as Yahoo and Bing, could also be useful.

While Google is a powerful tool for searching for grey literature it should not be used exclusively - other sources should also be searched in order to find grey literature. 

1) Restrict content to .org or .gov sites

Type in your topic and then either "site:.org" OR "site:.gov"

2) Restrict content to file type

Type in your topic and then "filetype:pdf" OR "filetype:doc"

3) Use Google Australia, Google UK, etc.

4) Use the Duck Duck Go search engine which does not record location or user searches

5) Use Link Klipper (Chrome extension) to pull results into a Microsoft Excel spreadsheet. You can also use the Grey Literature Search Log form linked below to keep track of your searches:

• Grey Lit Search Log
• << Previous: Searching the Published Literature
• Next: Methodology and Documentation >>
• Last Updated: Mar 5, 2024 3:43 PM
• URL: https://guides.lib.lsu.edu/Systematic_Reviews

Provide Website Feedback Accessibility Statement

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 12 March 2024

Mental health and psychosocial interventions in the context of climate change: a scoping review

• Siqi Xue 1 ,
• Alessandro Massazza 2 ,
• Samia C. Akhter-Khan 3 ,
• Britt Wray 4 ,
• M. Ishrat Husain 1 , 5 &
• Emma L. Lawrance 6 , 7 , 8  

npj Mental Health Research volume  3 , Article number:  10 ( 2024 ) Cite this article

301 Accesses

24 Altmetric

Metrics details

• Health care
• Planetary science

The evidence on the impacts of climate change on mental health and wellbeing is growing rapidly. The objective of this scoping review is to understand the extent and type of existing mental health and psychosocial interventions aimed at addressing the mental health and psychosocial impacts of climate change. A scoping review methodology was followed. MEDLINE, PsycINFO, and Web of Science databases were searched from inception to May 2022. Comprehensive gray literature search, including expert consultation, was conducted to identify interventions for which peer-reviewed academic literature may not yet be available. Data on intervention type, setting, climate stressor, mental health outcome, evaluation, and any other available details were extracted, and results were summarized narratively. Academic literature search identified 16 records and gray literature search identified a further 24 records. Altogether, 37 unique interventions or packages of interventions were identified. The interventions act at the levels of microsystem, mesosystem, exosystem, and macrosystem through diverse mechanisms. While most interventions have not been formally evaluated, promising preliminary results support interventions in low- and middle-income-country settings disproportionately affected by climate disasters. Interventions from multidisciplinary fields are emerging to reduce psychological distress and enhance mental health and wellbeing in the context of climate change. This scoping review details existing evidence on the interventions and summarizes intervention gaps and lessons learned to inform continued intervention development and scale-up interventions.

Similar content being viewed by others

The clinical implications of climate change for mental health

Tara J. Crandon, Cybele Dey, … Fiona J. Charlson

A systematic review of the effects of chronic, slow-onset climate change on mental health

Kate Burrows, Christy A. Denckla, … Sarah R. Lowe

Climate change affects multiple dimensions of well-being through impacts, information and policy responses

W. Neil Adger, Jon Barnett, … Sergio Jarillo

Introduction

Climate change is increasingly recognized as a public health emergency 1 , 2 . Beyond its well-recognized physical health consequences, the crisis impacts mental health in profound ways 1 , 3 , 4 , 5 . Climate events have been associated with worsened psychiatric mortality outcomes, including depression, post-traumatic stress disorder (PTSD), and suicide 6 . Various population groups have been identified as being particularly at-risk, including children and youth, older adults, pregnant women, people living with chronic illnesses, and racialized communities 6 , 7 . Indigenous groups worldwide have described feelings of sadness, anger, grief, fear, and helplessness from climate change-related forced migration, disrupted cultural continuity, and historical and ongoing disempowerment 8 .

Researchers in the climate change and mental health space acknowledge the intricate tension between recognizing the detrimental mental health impacts of climate change while not pathologizing culture-specific, expected, and adaptive responses to ongoing and anticipated threats. To encapsulate the nuanced range of experiences, new vocabulary, such as climate emotions, eco-anxiety, and ecological grief, have been introduced to literature 4 , 7 . Academics and third-sector organizations have also begun to identify protective coping mechanisms, including “active hope” and “meaning-focused coping” that emphasize acting in line with values, developing positive framings, and creating hope through action and trust in societal actors 9 , 10 , 11 .

Despite the accumulating evidence on climate-related mental health and/or psychological responses and coping strategies, little is known about evidence-based interventions to mitigate the negative consequences and support individuals and their communities. Of the 120 original studies identified in a scoping review on climate change and mental health research, the vast majority were cross-sectional studies quantifying the impact of climate change exposure on mental health outcomes 6 . The 8 studies related to interventions were primarily proposals and did not contain conclusive evidence. In another review on interventions for eco-anxiety, only 2 studies involved an empirical evaluation component, with the rest being conceptual or reflection papers 12 . The most comprehensive review to date identified 23 studies 13 , but did not differentiate between interventions implemented for a climate-related event and those for other settings (e.g., armed conflicts) theoretically relevant to the climate change context. The review was further limited to academic literature and interventions for preventing or treating known psychiatric disorders. As the World Health Organization (WHO) defines mental health as not the mere absence of a disorder but an overall state of wellbeing 14 , a broader conceptual framework could be helpful to recognize interventions that promote psychological strengths and emotional resilience in the face of climate stressors.

As increasing numbers of mental health interventions are being formally or informally developed and implemented in the context of climate change, there is a strong need to map out this space to facilitate knowledge exchange and identify best practices to scale-up support. Indeed, one of the global research priorities identified in a recent consensus building exercise is to “assess the appropriateness, feasibility, effectiveness, and scalability of mental health and psychosocial interventions (clinical and non-clinical) in the context of climate change” 15 . The current review aims to address this need by (1) focusing on existing interventions aimed at promoting mental health or mitigating the impacts on mental health; (2) considering broad mental health and wellbeing outcomes, which are not limited to predefined psychiatric diagnoses or newly described psychological states; and (3) incorporating a formal gray literature search process to identify interventions that have not been studied academically but are being implemented. The exploratory and flexible nature of a scoping review represents, therefore, an ideal methodology. To our knowledge, this is the first scoping review that aims to identify all such mental health and psychosocial interventions that have been implemented in the context of climate change.

The scoping review was conducted in accordance with the JBI Manual for Evidence Synthesis 16 . The protocol was registered prospectively on March 9, 2022 and is available at: https://osf.io/dya94/ .

Inclusion criteria

The scoping review included records describing existing interventions that promote mental health or mitigate mental health impacts in the context of climate change. No restrictions were imposed on intervention target populations or geographical settings. All study designs were considered, given the a priori understanding that many interventions likely have not been formally evaluated through clinical trials. Further, given the interdisciplinary nature of the topic, records in the humanities and applied social sciences may not be easily categorizable into a study design conventional to medicine or public health.

Exclusion criteria

The scoping review excluded records describing interventions for humanitarian crises not directly related to climate change (e.g., earthquakes, nuclear disasters, acts of terrorism), or natural hazards (e.g., recurrent floods) not explicitly connected with climate change by the authors. Other ineligible records included general resources (e.g., videos, webinars, podcasts, newsletters, therapist directories) that do not have an active interventional or participatory component; private, stand-alone retreats or courses; and books or book chapters. Proposals and expert opinions on psychotherapeutic approaches, while valuable to inform intervention development, are not organized programs that have been expanded beyond the individual therapist’s practice or systematically evaluated or implemented; they were therefore excluded.

Academic literature search strategy

The academic literature search strategy combined three core constructs, “intervention”, “mental health” and “climate change”, the definitions of which are outlined in Table 1 . The search terms were first developed for MEDLINE (Supplementary File 1 ) and modified for other databases as required. MEDLINE, PsycINFO and Web of Science were carried out from inception to May 2, 2022. Reference lists of any included studies and relevant reviews were hand searched to identify additional articles.

Study selection

All identified citations were collated in EndNote X9 with duplicates removed, then uploaded into Rayyan.ai, an online tool for conducting systematic and scoping reviews 17 . Titles and abstracts were screened by one reviewer (SX), with a subset (10%) screened by a second reviewer (SAK). Full texts of potentially relevant studies were acquired and screened by SX, a subset (10%) of which was also screened by SAK. Any discrepancies in reviewer decisions were resolved via discussions with a third reviewer (EL/AM).

Gray literature search strategy

A comprehensive gray literature search was undertaken to capture interventions not published in peer-reviewed journals or evaluated in academic literature. The search strategy was informed by the methodology and best-practice guidelines by Godin et al. 18 and Pollock et al. 18 , 19 , and involved four distinct steps: (1) targeted database search; (2) Google search; (3) targeted website search; and (4) key stakeholder consultation. The first three steps were carried out in May 2022 and the consultations took place between June-July 2022. Details of the gray literature search strategy can be found in Supplementary File 2 .

Data extraction

For academic literature, data were collected concerning the records (authors, publication year, study design) and nature of the interventions (delivery country/location; target population; climate stressor; mental health and wellbeing outcomes; evaluation outcomes; and any other available details, such as theoretical framework, intervention length, cost, facilitator characteristics, delivery methods, and involvement of stakeholders in co-designing the intervention). Best efforts were made to extract the same categories of data from gray literature. As the content from organizations may be trademarked, direct quotations from websites were at times extracted to best reflect the intervention description as intended. Data were presented in tabular form and analyzed narratively in accordance with their relevance to the review objectives.

Data presentation

Results are presented by clustering interventions according to Bronfenbrenner’s ecological theory 20 , 21 . The theory describes one’s ecological environment at four levels: microsystem (individual and immediate home environment); mesosystem (peer groups and social networks); exosystem (institutions, the media, and local government), and the macrosystem (policies, laws, and overarching social structures). Bronfenbrenner’s theory has been widely applied in public mental health research given its embracing of the complex interplay between individual and contextual factors. We believe categorizing the interventions at the different levels would allow for a clear conceptual map of where work has been done and where more work is needed, and the presentation of most pertinent information to relevant stakeholders (e.g., policymakers).

Overview of academic literature findings

In total, 5126 unique records were identified (5107 from databases and 19 from backward citation hand search) (Fig. 1 ). Among the 194 records reviewed in full, 16 met inclusion criteria and were included in the review (Table 2 ). The 16 studies described 13 unique stand-alone interventions or packages of interventions across Asia, Europe, North America and the Caribbean, Oceania, and Sub-Saharan Africa. Among the studies, 7 (44%) were conducted in LMIC settings. All studies were published in or after 2009, with half (50%) having been published within the past 3 years (2019–2022).

PRISMA flow diagram.

Climate stressors included general climatic change, wildfires, droughts, cyclones, typhoons, and floods. Targeted mental health outcomes included psychological distress, psychiatric symptoms (depression, anxiety, PTSD), and broader psychological wellbeing measures such as emotional strength, emotional self-efficacy, confidence in the future, and general mental and spiritual wellness. Nine (56%) studies involved a type of design (e.g., RCT, quasi-experimental, pre-post) aimed to quantitatively evaluate the interventions, and 7 (44%) reported effectiveness results. Eleven (69%) studies mentioned a co-design process, during which local stakeholders were consulted for the needs assessment, intervention design, and/or cultural adaptation of the intervention.

Description of interventions from academic literature

Five microsystem interventions were identified that primarily focus on individual-level emotions, behaviors, and psycho-emotional resilience.

In Nigeria, where increasingly frequent floods are being attributed to climate change, an evaluation of a rational emotive behavior therapy (REBT) program was conducted 22 . REBT is a short-term therapy related to cognitive behavioral therapy (CBT) and encourages participants to actively dispute irrational beliefs after experiencing an undesirable triggering event. REBT was delivered over 20 sessions to 49 flood victims with clinical depression by professional therapists. In comparison to the waitlist control group, the intervention group was found to have significantly decreased depression symptoms as measured by the Hamilton Depression Scale ( F (1,97) = 208.935, p  = 0.001, ηp 2  = 0.69) and by the Goldberg’s Depression Scale ( F (1,97) = 34.842, p  = 0.001, ηp 2  = 0.27) and at 3-month follow-up.

In Haiti, a disaster-prone country susceptible to climate change effects, an integrated community intervention was developed to promote mental health and improve practical disaster preparedness 23 . The intervention was manualized and consisted of activities that promote mental health literacy and coping skills (e.g., progressive muscle relaxation), and increase household-level preparedness (e.g., developing emergency action plan, mapping community risks and resources). The intervention was delivered by local lay workers over 3 days to 144 individuals who recently experienced a hurricane and associated flooding. In comparison to the control group, intervention participants experienced fewer depression ( B  = −0.35, p  < 0.001), anxiety ( B  = 0.27, p  < 0.001), and PTSD ( B  = −0.46, p  < 0.001) symptoms, and increased mental health-focused help-giving intention ( B  = 2.62, p  < 0.001).

Several interventions identified themselves as resilience-building programs. In the Philippines, a six-module intervention Katatagan was developed in the aftermaths of the Super Typhoon Haiyan and evaluated in two settings 24 , 25 . In Tacloban City where trained paraprofessionals delivered the intervention, the 48 intervention participants experienced lower anxiety scores ( F (1,105) = 3.89, p  = 0.05, ηp 2  = 0.036), increased adaptive coping ( F (2.79,192.6) = 5.87, p  = 0.001, ηp 2  = 0.078), and increased individual resilience ( F (1,105) = 4.68, p  = 0.03, ηp 2  = 0.043) 24 in comparison to the control group. In the Samar Province, the intervention was delivered as part of a mission trip by Health Futures Incorporated (HFI). The 163 intervention participants improved in all self-efficacy domains, as measured by a locally constructed scale based on each Katatagan module, in comparison to the participants’ baseline. Reported effect sizes ranged from Cohen’s d  = 0.33 for Engaging in Positive Activities, Cohen’s d  = 0.51 for Managing Thoughts and Emotions, to Cohen’s d  = 0.83 for Seeking Solutions and Support 25 . Benefits were largest immediately post-intervention and decreased at six-month follow-up in both settings.

Skills for Life Adjustment and Resilience (SOLAR) is another resilience-building intervention that was piloted in Tuvalu, a small island developing state (SIDS) vulnerable to sea-level rise 26 . Forty-nine islanders impacted by Cyclone Pam and experiencing mental health symptoms participated in the lay-worker-delivered group intervention over 5 consecutive days. Module content included Skills for Healthy Living, Managing Strong Emotions, Getting Back into Life Following Disaster, Coming to Terms with Disaster, Managing Worry and Rumination, and Maintaining Healthy Relationships. In comparison to controls, participants experienced improvements in psychological distress (Glass’s d  = 1.106), PTSD symptoms (Glass’s d  = 1.575), and functional impairment (Glass’s d  = 1.316). Benefits were retained at six-month follow-up but were reduced compared to immediate post-intervention.

One review study narratively described an Environmental Health Clinic based at New York University 27 , 28 . Individuals concerned about environmental issues received “prescriptions” to participate in environmental projects, with the aim to channel anxiety to specific climate action. This intervention was used as an example of structured problem-based coping and no evaluation was available.

Four mesosystem interventions were identified that involve focuses beyond individual-level changes, and additionally target peer group relationships and local community identity and cohesion in relation to mental health and wellbeing.

Carbon Conversations is a UK-based third-sector initiative which allows individuals to reflect on difficult emotions around climate change, and in turn be better able to engage with carbon footprint reduction 29 . Groups of six to eight participants meet with two facilitators over six sessions and discuss themes pertaining to Climate Change and Low Carbon Futures, Energy in the Home, Travel and Transport, Food and Water, and Consumption and Waste. In an online survey to 113 group participants, 50% agreed or strongly agreed that taking part helped them “face their worries about climate change”. Semi-structured interviews further revealed themes that the intervention allowed participants a unique space to express and share difficult emotions around climate change, feel more empowered and in control, and engage with others with similar experiences.

In various parts of the world vulnerable to extreme weather and regional food security, multiple interventions involving community-level participatory activities have been developed. These include: (1) land restoration work and associated reflection exercises for youth living at the Arizona-Mexico border in the United States; (2) community garden hubs with tree-planting activities for low-income residents and people with chronic and mental health conditions in Australia; and (3) traditional fish camp activities and participatory research for indigenous youth of the Selkirk First Nation in Canada 30 , 31 , 32 . All interventions were designed in the context of climate change to improve psychological and/or spiritual wellbeing, and to promote connectedness among participants through shared identities. Internal program evaluations of the land restoration and community garden interventions both anecdotally suggest improved mental health outcomes among participants, though the specific evaluative methods and outcome measures were not reported.

Two packages of exosystem interventions were identified that involved the implementation of multi-pronged mental health services and mobilization of the media and local institutions.

In Sonoma County, California, the Sonoma Wildfire Mental Health Collaborative was established following the historic 2017 wildfires 33 , 34 . The Collective launched a package of interventions that included a trauma-informed yoga and meditation program, a mental health app targeting adolescent survivors, and a Skills for Psychological Recovery (SPR) training program for counselors and paraprofessionals. The package was coupled with a media campaign to raise post-disaster mental health awareness, destigmatize help-seeking, and promote available resources. Given small sample sizes, the evaluative study was unable to conclude the effects of the app or SPR training; a preliminary survey suggested that participants experienced short-term beneficial effects from the yoga and meditation program.

For the rural farming populations who face climate-related adversity in New South Wales, Australia, a government-funded Drought Mental Health Assistance Package (DMHAP) was implemented 35 , 36 . DMHAP consisted of mental health promotion (e.g., resource booklet development, community mental health forums, mental health first aid trainings) and early intervention (e.g., rural telephone support line, service network planning workshops). Following renewed funding, the extended Rural Adversity Mental Health Program (RAMHP) increased the number of dedicated drought mental health workers and introduced specific activities for priority groups, including women, youth, older farmers, and Aboriginal communities. The authors reported that given funding limitations, no formal outcome evaluation was carried out.

Two macrosystem interventions were identified that involved macro-level interventional components, primarily through poverty reduction to improve mental health and wellbeing outcomes.

In Bangladesh, a Red Cross Red Crescent humanitarian project was delivered to promote financial security and associated psychological benefits among river basin communities amidst the 2017 floods 37 . A forecast-based unconditional cash transfer of BDT 5000 (USD 60 equivalent) was distributed to 1039 poor households prior to a flood peak. Relative to the unassisted households, intervention households were less likely to have always felt anxiety and depression (43% vs 29%, p  = 0.015) and less likely to have always felt miserable or unhappy (61% vs 40%, p  < 0.01) since the flood. While these findings were triangulated with qualitative interviews, the authors noted that the intervention benefits were not sustained after a second flood peak in the same year.

In Ethiopia, a capacity-building intervention was developed for pastoral communities who face increasingly severe droughts, land changes, and food crises. The intervention included components to inspire motivation, build collective-action groups, and improve literacy and numeracy 38 . The macro-economic components specifically involved the promotion of microenterprises and distribution of donor grants for livestock trading. The study reported the creation of 59 primarily women-led collective-action groups, 11 of which received donor grants. Following a major drought, intervention participants who received both capacity building and donor grants in the Liben District scored much higher on study-defined mental health and wellbeing attributes than their peers. The attributes included better ability to recover from crisis (OR = 91.7, p  < 0.001), more confidence in the future (OR = 33.6, p  < 0.001), and better human health (OR = 19.2, p  < 0.001). The positive findings were replicated in the Moyale District with smaller effect sizes.

Overview of gray literature findings

Targeted database search, Google search, and targeted website search identified 14 records describing 14 organizations offering relevant stand-alone and/or packages of interventions. One of the interventions (Carbon Conversations) was already identified through academic literature. A list containing these interventions, along with the inclusion and exclusion criteria, was circulated to 30 international content experts for consultation; through snowballing, we were introduced to and contacted 6 additional experts. Of all individuals contacted, 26 (72%) responded to the gray literature search request, and 8 (31%) of the respondents were based in a LMIC setting. The content experts reviewed and confirmed the list of interventions and identified 11 additional records for inclusion (total records = 25; minus duplicate = 24) (Table 3 ).

All identified interventions are offered by an organization, and all but two were founded or based in a high-income country (HIC), primarily the US (46%) or the UK (33%). For the organizations that cited their founders, a large majority (11/12; 92%) of those founders are women.

The climate stressors addressed by interventions from the gray literature all involve general climatic changes and/or anticipated climate-related threats rather than specific climate-related disasters. Targeted mental health outcomes are independently defined by the organizations rather than by psychiatric diagnoses or standardized instruments, and many used emotion-based wordings such as “overwhelm”, “despair”, and “loneliness”. Five records (21%) described a co-design process involving local stakeholders.

Description of interventions from gray literature

The 24 interventions or packages of interventions acted at the levels of microsystem and mesosystem. No exosystem or macrosystem interventions were identified through gray literature. Of note, none of the interventions reported formal evaluation methods; a minority (12.5%) reported selected positive internal evaluation results.

Four microsystem interventions were identified that take a self-guided approach and support individuals to improve their own mental health and wellbeing in the context of climate change.

In the UK, a group of organizations, namely Climate Cares based at Imperial College London, Force of Nature and Common Vision worked with young people and environmental scientists to create a virtual intervention Hold This Space 39 . The interactive website guides youth to explore their feelings towards climate change, imagine the world they would like to see based on the latest science, and reflect on how to act on environmental issues most concerning to them. In consultation with youth advisors, mental health practitioners, and climate change professionals, Climate Cares also co-designed a 4-week activity-based physical journal 40 . The goal is to reduce the mental health impacts that can be associated with climate-related distress, build coping strategies, envision a desired future, and increase capability to take desired action. The Climate Journal Project based in the US is another intervention involving a journaling approach 41 . The organization created digital and printed journals and worksheets that target eco-anxiety and environmental grief. Individuals have the additional option of participating in virtually guided “journal circles”. Other self-guided activities and worksheets were identified through Eco-Anxious Stories, a Canada-based online platform 42 . In the “Sharing Our Stories” worksheet, individuals are prompted to answer questions such as “Where is eco-anxiety showing up in my life?” and “What does a meaningful response to this crisis look like and feel like?”

The rest of 20 gray literature records all acted at the level of the mesosystem and involve a group-based or outreach approach that harness the power of group dynamics and community building.

Many of the interventions function on the premise of offering a safe space for individuals to gather and make sense of their positive or negative climate emotions. As a quintessential example, Climate Cafés are decentralized, drop-in meetings for discussing the climate crisis and building collective psycho-emotional resilience – often over tea or coffee. The model of Climate Cafés is now adopted by many organizations globally with both in-person and virtual meetings available 43 , 44 , 45 . The Good Grief Network developed an intervention based on the 12-Step approach of Alcoholics Anonymous 46 . Trained peer facilitators deliver the 10-week group program for individuals interested in recognizing and exploring their eco-distress and being supported to move towards meaningful action. The Network’s website reports that over 90% of program participants feel more empowered and less alone, though the exact survey methods or number of surveyed participants are not available publicly. Other examples of facilitated discussions include All We Can Save Circles (10 structured sessions) 47 , Climate Emotions Conversations by Climate Awakening (3 available sessions per month) 48 , and The Rest of Activism (2 available sessions per week) 49 . A few interventions name their specific target audience: Conceivable Future “house parties” are intended for individuals who wish to discuss reproductive decisions and parenthood while facing an uncertain future 50 , and Globe and Psyche conversation meetings are intended for individuals working in psychotherapeutic and psycho-spiritual domains to reflect on professional identities and healings for others 51 .

Five identified organizations offer packages of interventions (e.g., facilitated workshops, events, trainings, online communities) under the same theoretical or philosophical premise. The Work That Reconnects (WTR) is a network based on Joanna Macy’s work, also known as Deep Ecology Work and Active Hope. The work’s philosophical premise follows a spiral sequence of four stages, “gratitude”, “honoring our pain for the world”, “seeing with fresh eyes”, and “going forth”, and is designed to be delivered in an interactive group setting 52 . The Deep Adaptation Forum is based on Jem Bendell’s work 53 , which recognizes the “breakdown” from climate change and aims to support individuals to prepare for and co-create a loving response to what Bendell describes as the “inevitable near term societal collapse”. The Resilient Activist is based on the “Five Essentials” principle (Reconnect to Nature, Respect All Life, Regreen Our Planet, Revamp Our Spending, and Replenish Our Resources) to maintain a healthy mindset and ease the emotional burden from climate change 45 . The Transition Network offers a variety of interventions (e.g., “Heart and Soul” groups) based on the Inner Transition principle, which posits that shifts in emotional and psychological dimensions are needed to make outer systemic changes towards healthier communities 54 . One Earth Sangha is a hub for spiritual-psychological participatory groups and events in response to climate change based on Buddhist teachings 55 .

Seven interventions involve outreach or capacity-building approaches to improve the participants’ mental health and wellbeing and/or that of their wider community. Force of Nature runs training programs for youth affected by eco-anxiety. The trained youth then have the opportunity to run group “anxiety-to-agency” workshops, and become speakers or consultants for businesses and educators on matters related to climate change 56 . The Resilience Project UK also offers a program for young people, who then become leaders of an 8-week Circle to co-design resilience-building programs for other youth 57 . Project InsideOut is an online hub with interactive tools and resources, allowing individuals who experience climate emotions to undergo an inner transformation before becoming “Guides” for others and leading climate action 58 . Eco-Anxious Stories, Climate Psychology Alliance, Circularity, and The Resource Innovation Group (TRIG) all list eco-anxiety outreach services (e.g., speaker hub, trainings, workshops, resource development) for schools, organizations, and communities 42 , 44 , 59 , 60 .

Only two identified interventions at this level are based in a LMIC setting. In Nigeria, The Eco-Anxiety in Africa Program (TEAP) is managed by Sustyvibes - a non-profit climate activism organization. Reported TEAP activities include the creation of virtual and physical spaces to stimulate dialogs on climate change and mental health (e.g., “Sustyparties” that use poetry and open mic settings to facilitate the sharing climate emotions) 61 . In Cameroon, the Ibanikom Climate Mental Health Literacy Project facilitated meetings for flood-affected communities, allowing participants to learn about the effects of climate change on mental health and co-develop local, small-scale culturally relevant integrated health and agriculture projects 62 .

The impact of climate change on mental health and wellbeing is a pressing global challenge. More information is critically needed to plan for the design, implementation, and scaleup of effective interventions that address the dual and interconnected crises of ecological breakdown and mental health and psychosocial wellbeing burden. The present scoping review represents one of the first comprehensive efforts to fulfill this research priority and identified a total of 40 records which describe 37 unique interventions across academic and gray literature. The interventions acted at the levels of individuals, groups, local media and institutions, and larger social structures, and involved diverse mechanisms of action including psychotherapy, resilience-building programs, nature-based activities, community strengthening networks, and climate activism projects (Fig. 2 ). The mental health and wellbeing targets included depression, anxiety, PTSD, emotional strength and resilience, and various climate emotions, stemming from both direct exposure to climate-related extreme events and awareness of climate change.

Identified interventions and their level of action based on Bronfenbrenner’s ecological theory as applied to public mental health research.

The academic literature search identified interventions implemented primarily in geographical areas at risk of extreme weather events, and the target populations were often residents of the areas. Interestingly, all studies that reported a formal evaluation methodology and intervention effectiveness results ( n  = 7) were conducted in a LMIC setting. Among these studies, 5 incorporated a co-design process, and all demonstrated promising initial results in reducing negative psychiatric symptoms and/or promoting positive mental health and wellbeing outcomes. It is well recognized that the climate crisis deepens pre-existing global inequities 63 ; it is therefore encouraging to see that current research efforts have been attentive to supporting the most disproportionately affected populations. However, several studies with a longitudinal evaluation component commented on reduced interventional effects at follow-up 24 , 25 , 26 . Furthermore, it is common for the LMIC-based interventions to be funded or run by international humanitarian organizations (e.g., Red Cross, HFI) rather than being integrated into the local system, and the potential for the sustainable implementation and scaleup of these interventions remain less certain.

In contrast to the academic literature findings, the gray literature search identified mostly interventions delivered by organizations founded or based in HICs. Few interventions commented on a theoretical framework for the mechanism of intervention. None were formally evaluated or have evaluative data publicly available, and therefore it is not possible to determine whether they are effective in addressing their targeted mental health and wellbeing outcomes or have any unintended impacts. It is further difficult to draw the line between non-profit and for-profit organizations, as various workshops and events depend on out-of-pocket payments from participants. This highlights additional sustainability and accessibility issues for individuals who are intersectionally disenfranchised by financial insecurity and mental health burden and may also limit their potential scalability to low-resource settings.

Overall, it appears that conceptual linkage for interventions at the intersection of climate change and mental health remains at a nascent stage, and most interventions are newly designed with scarce or anecdotal evidence. Most of the existing trials involve microsystem-level interventions (i.e., targeting individual emotions and behaviors) implemented in LMIC settings. Even then, the interventions are limited to a single study or country, and the findings have not been replicated. While there are many mesosystem-level interventions that leverage group dynamics, the overwhelming majority are based in HICs and have not been academically evaluated. There is a shortage of exo- and macrosystem interventions that mobilize local government and media or incorporate socioeconomic reforms and policy changes that may influence downstream mental health and wellbeing outcomes, though there is clear evidence for the potential co-benefits of climate policies for mental health and wellbeing 1 . There is, in general, also a lack of publicly available implementation information (e.g., training procedure of facilitators, implementation cost) for existing interventions, which would be key for scaleup.

Strengths and limitations

A strength of this review lies in its comprehensive conceptualization of mental health. To reflect that mental health is “an intrinsic part of our individual and collective health and wellbeing” 14 , the review included search terms such as “wellbeing”, “resilience” and “post-traumatic growth”. This open approach allowed us to identify interventions beyond the realms of clinical psychology or psychiatry and tap into fields such as international development, urban planning, and environmental public health. The resulting diverse findings suggest that climate mental health interventions are likely to require complex, multidisciplinary input. Another strength of this review is its incorporation of a rigorous gray literature review process, which involved a large panel of international content experts. The process allowed for the capturing of emerging interventions and those not formally evaluated. Moreover, the review sought to, beyond identifying interventions, determine which types of interventions are better supported by evidence, and where there are clear gaps for the evaluation of existing interventions and/or the design of new ones.

A limitation of the review is that the search terms were only in English, which likely influenced the outcome that all included academic literature articles were English-language ones. Potential relevant articles published in other languages describing local interventions may be omitted. Further, most of the identified gray literature interventions are based in HICs. While we purposively consulted content experts from LMICs, it is possible that the search was biased towards HIC-based content given the immediate professional network of the authors and given that HIC-based organizations are likely better resourced to host and publicize their interventions on websites. We recognize that there are likely many other local, grassroot initiatives that have not gained international traction, yet also provide safe, accessible, and community-relevant spaces for discussions and actions around climate change and mental health. Many Indigenous communities have historically and continue to implement cultural practices that care for the wellbeing of people and the non-human world. Additionally, without individually contacting the individual organizations, we cannot comment on whether the website information is accurate and up-to-date, or the scale of the intervention (e.g., how frequently the microsystem-level worksheets are being downloaded, how many individuals have participated in the mesosystem group interventions). Therefore, this review cannot claim to be an exhaustive search of all existing and active mental health and psychosocial interventions, but rather a best effort at describing and mapping out the present interventional space.

Recommendations for future research

Most of the existing evidence supports microsystem-level interventions in LMIC settings. For these interventions, studies using more robust study designs and involving more participants, as well as studies involving their adaptation to other geographical regions, would be helpful to better evaluate their larger-scale effectiveness and scalability. Implementation science would need to be applied to better assess the feasibility, acceptability, cost-effectiveness, and sustainability of the interventions. Ideally, the studies would also include a process evaluation component to better elicit why longitudinal effects may be reduced. This review further identified various mesosystem-level interventions in HIC settings, and there is potential for these interventional models to be studied using robust trial methodologies.

Regarding the development and piloting of new interventions, there is much room to explore exo- and macrosystem interventions in both LMICs and HICs. In our literature search, we identified three studies that may offer important insight. All were secondary analyses of the social impacts of climate interventions, which originally did not have a mental health focus (and hence did not meet our inclusion criteria and were not summarized narratively in our results). In one study based in Zimbabwe, it was found retrospectively that a biogas project contributed to community cohesion and empowerment 64 ; in another study based in China, planned relocation and sheltering was found to be protective against depression, anxiety, and PTSD symptoms among flood victims 65 . In contrast, a study conducted in six LMICs (Brazil, Cameroon, Indonesia, Peru, Tanzania, and Vietnam) concluded that the Reducing Emissions from Deforestation and Degradation (REDD+) initiative may have negatively impacted women’s wellbeing 66 . Suggested reasons for the wellbeing decline included unrealized expectations for REDD+ initiatives and limited advanced consideration of addressing gender inequality in REDD+ policies. The studies demonstrated that while macrosystem interventions aiming to improve social welfare and their larger environment likely offer additional psychological benefits, they may also have unintended consequences if mental health and wellbeing, and its relationship to factors such as participatory approaches and gender equity, is not considered at conception. The REDD+ study further highlighted the importance of involving underserved groups, such as women, in the intervention design, implementation, and decision-making stages, thereby empowering them with leadership roles and incorporating their own definitions and experiences of wellbeing. This point applies generally, and it is vital for the appropriateness and ultimately success of both the content and implementation of interventions that they are co-designed with the people for whom they are being created. Understandings of mental health and wellbeing, and the experiences of climate change vary across geographies and cultures, and it is important to not perpetuate colonial practices by imposing Western definitions and understandings universally 67 , 68 .

It has also been increasingly recognized that being in and feeling connected with nature is beneficial for human health, including mental health 69 , 70 , 71 . However, most of the existing studies are limited to cross-sectional assessments that outline the association between time spent in the natural environment and health benefits. Nature-based programs or eco-therapies, such as animal-assisted interventions, therapeutic horticulture, forest bathing, and social prescriptions that bring participants into nature do offer trial evidence, but many were developed or evaluated among specific populations (e.g., cancer patients, children), and have not yet been directly associated with climate change 72 , 73 , 74 , 75 , 76 . It would be of interest to further explore climate-informed nature engagement (e.g. “reciprocal restoration” or social prescribing interventions) 30 and their mental health and wellbeing outcomes.

Finally, disaster psychiatry offers much knowledge of interventions for individuals and communities surviving potentially traumatic events, and many lessons can be drawn regarding what works, where, and for whom. It is likely that climate change has contributed to the development of many natural hazards (e.g., Hurricane Katrina) which have been the object of study in disaster psychiatry 77 . However, as developed interventions typically have not factored in climate change - an unprecedented ongoing and growing crisis - they would likely need to be adapted to support individuals to cope with not only current but future stressors, and incorporate strategies such as proactive, forecast-based interventions, and disaster preparedness. The interventions would also likely benefit from collaborating across multiple bio-psychosocial fields to work preventatively and to address contextual factors.

Implications for policy and practice

Multiple studies identified in this review did not successfully carry out an evaluative component despite having the intent, primarily due to limited funding and/or resources to recruit a larger sample size. This reflects a need to invest in and better support the evaluation of mental health and psychosocial interventions in the context of climate change, such that resources can be channeled into best evidence-based practices. When these practices are identified, a streamlined process is needed for their integration into the local health and social care systems, particularly in low-resource settings using existing infrastructures such as community groups. Consideration of participation cost is crucial, such that individuals from all socioeconomic backgrounds can be included and benefit.

For the work that is being done informally or in a community-led manner, including the gray literature interventions identified here and emerging ones we are aware of anecdotally, should be convened, collated, and showcased in an accessible manner. For example, an “online hub” of currently available interventions and case studies of best practices could promote shared learning and evidence-based investment, while minimizing the likelihood of duplicated efforts 78 .

Conclusions

There is growing awareness of the detrimental effects of climate change on mental health and psychosocial wellbeing. In response to this evidence-base and lived experiences globally, interventions have been designed to promote mental health and wellbeing as well as to manage the detrimental impacts. This scoping review identified interventions acting at the microsystem, mesosystem, exosystem, and macrosystem levels. While most interventions have not been evaluated, existing studies, primarily on protecting mental health in the context of climate-related disasters in LMICs, show preliminary promising results. More evaluative studies using robust trial designs are needed, especially those involving implementation research. Future interventions are recommended to consider at conception the definition of wellbeing, the interests of underserved groups, co-design, equitable access, and sustainability.

Lawrance, E., Thompson, R., Fontana, G. & Jennings, N. The Impact of Climate Change on Mental Health and Emotional Wellbeing - Current Evidence and Implications for Policy and Practice . Briefing Paper No. 36 (Grantham Institute, 2021).

World Health Organization. COP26 Special Report on Climate Change and Health: the Health Argument for Climate Action (WHO, Geneva, 2021).

Berry, H. L., Waite, T. D., Dear, K. B. G., Capon, A. G. & Murray, V. The case for systems thinking about climate change and mental health. Nat. Climate Change 8 , 282–290 (2018).

Article   ADS   Google Scholar  

Cunsolo, A. & Ellis, N. R. Ecological grief as a mental health response to climate change-related loss. Nat. Climate Change 8 , 275–281 (2018).

Hayes, K., Blashki, G., Wiseman, J., Burke, S. & Reifels, L. Climate change and mental health: risks, impacts and priority actions. Int. J. Ment. Health. Syst. 12 , 28 (2018).

Article   PubMed   PubMed Central   Google Scholar  

Charlson, F. et al. Climate change and mental health: a scoping review. Int. J. Environ. Res. Public Health 18 , 4486 (2021).

Cianconi, P., Betro, S. & Janiri, L. The impact of climate change on mental health: a systematic descriptive review. Front. Psychiatry 11 , 74 (2020).

Middleton, J., Cunsolo, A., Jones-Bitton, A., Wright, C. J. & Harper, S. L. Indigenous mental health in a changing climate: a systematic scoping review of the global literature. Environ. Res. Lett. 15 , 053001 (2020).

Macy, J. & Johnstone, C. Active Hope - How to Face the Mess We’re in with Unexpected Resilience and Creative Power (New World Library, Novato, California, 2022).

Ojala, M. How do children cope with global climate change? Coping strategies, engagement, and well-being. J. Environ. Psychol. 32 , 225–233 (2012).

Article   Google Scholar  

Ojala, M. Coping with climate change among adolescents: implications for subjective well-being and environmental engagement. Sustainability 5 , 2191–2209 (2013).

Baudon, P. & Jachens, L. A scoping review of interventions for the treatment of eco-anxiety. Int. J. Environ. Res. Public Health 18 , 9636 (2021).

Palinkas, L. A., O’Donnell, M. L., Lau, W. & Wong, M. Strategies for delivering mental health services in response to global climate change: a narrative review. Int. J. Environ. Res. Public Health 17 , 8562 (2020).

World Health Organization. World Mental Health Report: Transforming Mental Health for All (WHO, Geneva, 2022).

Charlson, F. et al. Global priorities for climate change and mental health research. Environ. Int. 158 , 106984 (2022).

Article   PubMed   Google Scholar  

JBI. JBI Manual for Evidence Synthesis (JBI, 2020). Available from: https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-01 .

Ouzzani, M., Hammady, H., Fedorowicz, Z. & Elmagarmid, A. Rayyan-a web and mobile app for systematic reviews. Syst. Rev. 5 , 210 (2016).

Godin, K., Stapleton, J., Kirkpatrick, S. I., Hanning, R. M. & Leatherdale, S. T. Applying systematic review search methods to grey literature: a case study examining guidelines for school-based breakfast programs in Canada. Syst. Rev. 4 , 138 (2015).

Pollock, D. et al. Undertaking a scoping review: a practical guide for nursing and midwifery students, clinicians, researchers, and academics. J. Adv. Nurs. 77 , 2102–2113 (2021).

Eriksson, M., Ghazinour, M. & Hammarström, A. Different uses of Bronfenbrenner’s ecological theory in public mental health research: what is their value for guiding public mental health policy and practice? Soc. Theory Health 16 , 414–433 (2018).

van Kessel, G., Gibbs, L. & MacDougall, C. Strategies to enhance resilience post-natural disaster: a qualitative study of experiences with Australian floods and fires. J. Public Health 37 , 328–336 (2015).

Ede, M. O. et al. The effect of rational emotive behaviour therapy on post-traumatic depression in flood victims. J. Ration. Emot. Cogn. Behav. Ther. 40 , 124–143 (2021).

James, L. E., Welton-Mitchell, C., Noel, J. R. & James, A. S. Integrating mental health and disaster preparedness in intervention: a randomized controlled trial with earthquake and flood-affected communities in Haiti. Psychol. Med. 50 , 342–352 (2020).

Hechanova, M. R. et al. Evaluation of a resilience intervention for Filipino displaced survivors of Super Typhoon Haiyan. Disaster Prev. Manag. 27 , 346–359 (2018).

Hechanova, M. R. M., Waelde, L. C. & Ramos, P. A. P. Evaluation of a group-based resilience intervention for Typhoon Haiyan survivors. J. Pac. Rim Psychol. 10 , e12 (2016).

Gibson, K. et al. Piloting a scalable, post-trauma psychosocial intervention in Tuvalu: the Skills for Life Adjustment and Resilience (SOLAR) program. Eur. J. Psychotraumatol. 12 , 1948253 (2021).

Koger, S. M., Leslie, K. E. & Hayes, E. D. Climate change: psychological solutions and strategies for change. Ecopsychology 3 , 227–235 (2011).

Schaffer A. Prescriptions for health, the environmental kind. The New York Times https://www.nytimes.com/2008/08/12/health/12clin.html (2008).

Büchs, M., Hinton, E. & Smith, G. ‘It Helped Me Sort of Face the End of the World’: the role of emotions for third sector climate change engagement initiatives. Environ. Values 24 , 621–640 (2015).

Nabhan, G. P., Orlando, L., Smith Monti, L. & Aronson, J. Hands-on ecological restoration as a nature-based health intervention: reciprocal restoration for people and ecosystems. Ecopsychology 12 , 195–202 (2020).

Patrick, R. & Capetola, T. It’s here! Are we ready? Five case studies of health promotion practices that address climate change from within Victorian health care settings. Health Promot. J. Aust. 22 , 7 (2011).

Richards, G., Frehs, J., Myers, E. & Van Bibber, M. Commentary - the climate change and health adaptation program: indigenous climate leaders’ championing adaptation effort. Health Promot. Chronic Dis. Prev. Can. 39 , 127–130 (2019).

Heinz, A. J. et al. Feasibility and preliminary efficacy of a public mobile app to reduce symptoms of postdisaster distress in adolescent wildfire survivors: sonoma rises. Psychol. Serv. https://doi.org/10.1037/ser0000576 (2021).

Heinz, A. J. et al. Rising from the ashes by expanding access to community care after disaster: an origin story of the Wildfire Mental Health Collaborative and preliminary findings. Psychol. Serv. 19 , 58–66 (2022).

Hart, C. R., Berry, H. L. & Tonna, A. M. Improving the mental health of rural New South Wales communities facing drought and other adversities. Aust. J. Rural Health 19 , 231–238 (2011).

Tonna, A. et al. Improving the mental health of drought-affected communities: an Australian model. Rural Soc. 19 , 296–305 (2014).

Gros, C. et al. Household-level effects of providing forecast-based cash in anticipation of extreme weather events: quasi-experimental evidence from humanitarian interventions in the 2017 floods in Bangladesh. Int. J. Disaster Risk Reduct. 41 , 101275 (2019).

Coppock, L., Desta, S., Tezera, S. & Gebru, G. Capacity building helps pastoral women transform impoverished communities in Ethiopia. Science 334 , 6 (2011).

Hold This Space https://holdthis.space/survey-panel/introduction/ (2022).

Diffey, J. & Batey, P. What can we learn from our feelings about climate change. Imperial College London https://blogs.imperial.ac.uk/ighi/2022/03/01/what-can-we-learn-from-our-feelings-about-climate-change/ (2022).

Climate Journal Project https://www.theclimatejournalproject.com (2022).

EcoAnxious Stories https://ecoanxious.ca/ (2023).

Climate Café https://www.climate.cafe (2023).

Alliance CP. Outreach https://www.climatepsychologyalliance.org (2022).

The Resilient Activist https://www.theresilientactivist.org (2022).

Good Grief Network https://www.goodgriefnetwork.org (2022).

All We Can Save https://www.allwecansave.earth/circles (2022).

Climate Awakening https://climateawakening.org (2022).

The Rest of Activism https://www.climateemergence.co.uk/rest-of-activism-membership (2022).

Conceivable Future https://conceivablefuture.org (2022).

Globle and Psyche https://www.globeandpsyche.com (2022).

The Work That Reconnects https://workthatreconnects.org/ (2022).

Deep Adaptation Forum https://www.deepadaptation.info (2022).

Transition Network https://transitionnetwork.org (2022).

One Earth Sangha https://oneearthsangha.org (2022).

Force of Nature https://www.forceofnature.xyz/students (2022).

The Resilience Project UK https://www.theresilienceproject.org.uk/our-work (2022).

Project InsideOut https://projectinsideout.net (2022).

The Resource Innovation Group (TRIG) http://www.theresourceinnovationgroup.org/transformational-resilience/ (2022).

Circularity https://circularitycommunity.com (2022).

The Eco-Anxiety in Africa Project https://www.teap.sustyvibes.org (2022).

Claretian University Centre of Excellence (AIKIT) flags off the Ibanikom Health Literacy Evolution project in Cameroon. https://claretianuniversity.edu.ng/tag/ibanikom-health-literacy-evolution/ .

Romanello, M. et al. The 2021 report of the Lancet Countdown on health and climate change: code red for a healthy future. Lancet 398 , 1619–1662 (2021).

Dhliwayo, N., Chanza, N. & De Wit, A. Beyond climatic intervention: the social dimension of a biogas project in Sogwala village, Zimbabwe. J. Energy South. Afr. 31 , 43–56 (2020).

Zhong, S. et al. Assessing the effectiveness and pathways of planned shelters in protecting mental health of flood victims in China. Environ. Res. Lett. 15 , 125006 (2020).

Article   CAS   ADS   Google Scholar  

Larson, A. M. et al. Gender lessons for climate initiatives: a comparative study of REDD+ impacts on subjective wellbeing. World Dev. 108 , 86–102 (2018).

Pihkala, P. Toward a taxonomy of climate emotions. Front. Climate 3 , 738154 (2022).

Ojala, M., Cunsolo, A., Ogunbode, C. A. & Middleton, J. Anxiety, worry, and grief in a time of environmental and climate crisis: a narrative review. Annu. Rev. Environ. Res. 46 , 35–58 (2021).

Pellens, N., Boelee, E., Veiga, J. M., Fleming, L. E. & Blauw, A. Innovative actions in oceans and human health for Europe. Health Promot. Int. 38 , 1–11 (2021).

Kingsley, M. & EcoHealth, O. Commentary - climate change, health and green space co-benefits. Health Promot. Chronic Dis. Prev. Can. 39 , 131–135 (2019).

White, M. P., Elliott, L. R., Gascon, M., Roberts, B. & Fleming, L. E. Blue space, health and well-being: a narrative overview and synthesis of potential benefits. Environ. Res. 191 , 110169 (2020).

Article   CAS   PubMed   Google Scholar  

Chaudhury, P. & Banerjee, D. “Recovering With Nature”: a review of ecotherapy and implications for the COVID-19 pandemic. Front. Public Health 8 , 604440 (2020).

Hasbach, P. H. Therapy in the face of climate change. Ecopsychology 7 , 205–210 (2015).

Cimprich, B. & Ronis, D. L. An environmental intervention to restore attention in women with newly diagnosed breast cancer. Cancer Nurs. 26 , 9 (2003).

Barrable, A., Booth, D., Adams, D. & Beauchamp, G. Enhancing nature connection and positive affect in children through mindful engagement with natural environments. Int. J. Environ. Res. Public Health 18 , 4785 (2021).

Park Rx: a prescription for nature https://www.parkrx.org (2022).

Bentham, W., Vannoy, S., Badger, K., Wennerstrom, A. & Springgate, B. F. Opportunities and challenges of implementing collaborative mental health care in post-Katrina New Orleans. Ethn. Dis. 21 , 30–37 (2011).

Google Scholar  

Dooley, L., Sheats, J., Hamilton, O., Chapman, D. & Karlin, B. Climate change & youth mental health: Psychological impacts, resilience resources, and future directions. (Los Angeles, CA: See Change Institute, 2021).

Download references

Acknowledgements

We would like to thank the following individuals who acted as expert consultants in the gray literature search stage: Kenneth Yongabi Anchang (Cameroon), John Aruta (Philippines), Helen Berry (Australia), Mike Campbell (Barbados), Fiona Charlson (Australia), Susan Clayton (USA), Julian Eaton (UK), Mohamed Elshazly (Egypt), Brandon Gray (Switzerland), Natalie Greaves (Barbados), Renzo Guinto (Philippines), Katie Hayes (Canada), Jo Hamilton (UK), Asha Hans (India), Zeinab Hijazi (USA), Juan Segundo Peña Loray (Spain), Raj Mariwala (India), Catriona Mellor (UK), Rebecca Nestor (UK), Sena Salcedo (Philippines), Jennifer Uchendu (Nigeria), and Sacha Wright (UK). We would like to thank Jura Augustinavicius for reviewing and providing comments for the protocol.

Author information

Authors and affiliations.

Department of Psychiatry, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada

Siqi Xue & M. Ishrat Husain

Centre for Global Mental Health, The London School of Hygiene & Tropical Medicine, London, UK

Alessandro Massazza

Department of Health Service & Population Research, Institute of Psychiatry, Psychology, & Neuroscience, King’s College London, London, UK

Samia C. Akhter-Khan

Department of Psychiatry and Behavioral Sciences of Stanford Medicine, Stanford, US

Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, ON, Canada

M. Ishrat Husain

Institute of Global Health Innovation, Imperial College London, London, UK

Emma L. Lawrance

Grantham Institute - Climate Change and the Environment, Imperial College London, London, UK

Mental Health Innovations, London, UK

You can also search for this author in PubMed   Google Scholar

Contributions

E.L. and A.M. conceptualized the study in discussion with S.X. S.X. conducted the literature review. S.A.K. acted as a second reviewer for a portion of the abstracts and full texts. B.W. contributed to the gray literature search, and E.L. and A.M. resolved any conflict decisions. S.X. conducted data analysis and wrote the original draft of the manuscript. All authors contributed to the review and editing of subsequent drafts. S.X. received supervision from E.L. (Imperial College London), A.M. (London School of Hygiene & Tropical Medicine) and M.I.H. (University of Toronto).

Corresponding author

Correspondence to Siqi Xue .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Xue, S., Massazza, A., Akhter-Khan, S.C. et al. Mental health and psychosocial interventions in the context of climate change: a scoping review. npj Mental Health Res 3 , 10 (2024). https://doi.org/10.1038/s44184-024-00054-1

Download citation

Received : 08 March 2023

Accepted : 09 January 2024

Published : 12 March 2024

DOI : https://doi.org/10.1038/s44184-024-00054-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Research article
• Open access
• Published: 26 July 2023

Factors contributing to healthcare professionals’ adaptive capacity with hospital standardization: a scoping review

• Foteini Tsandila-Kalakou   ORCID: orcid.org/0000-0001-7638-4643 1 ,
• Siri Wiig   ORCID: orcid.org/0000-0003-0186-038X 1 &
• Karina Aase   ORCID: orcid.org/0000-0002-5363-5152 1  

BMC Health Services Research volume  23 , Article number:  799 ( 2023 ) Cite this article

1595 Accesses

3 Citations

6 Altmetric

Metrics details

Certain factors contribute to healthcare professionals’ adaptive capacities towards risks, challenges, and changes such as attitudes, stress, motivation, cognitive capacity, group norms, and teamwork. However, there is limited evidence as to factors that contribute to healthcare professionals’ adaptive capacity towards hospital standardization. This scoping review aimed to identify and map the factors contributing to healthcare professionals’ adaptive capacity with hospital standardization.

Scoping review methodology was used. We searched six academic databases to September 2021 for peer-reviewed articles in English. We also reviewed grey literature sources and the reference lists of included studies. Quantitative and qualitative studies were included if they focused on factors influencing how healthcare professionals adapted towards hospital standardization such as guidelines, procedures, and strategies linked to clinical practice. Two researchers conducted a three-stage screening process and extracted data on study characteristics, hospital standardization practices and factors contributing to healthcare professionals’ adaptive capacity. Study quality was not assessed.

A total of 57 studies were included. Factors contributing to healthcare professionals’ adaptive capacity were identified in numerous standardization practices ranging from hand hygiene and personal protective equipment to clinical guidelines or protocols on for example asthma, pneumonia, antimicrobial prophylaxis, or cancer. The factors were grouped in eight categories: (1) psychological and emotional, (2) cognitive, (3) motivational, (4) knowledge and experience, (5) professional role, (6) risk management, (7) patient and family, and (8) work relationships. This combination of individual and group/social factors decided whether healthcare professionals complied with or adapted hospital standardization efforts. Contextual factors were identified related to guideline system, cultural norms, leadership support, physical environment, time, and workload.

The literature on healthcare professionals’ adaptive capacity towards hospital standardization is varied and reflect different reasons for compliance or non-compliance to rules, guidelines, and protocols. The knowledge of individual and group/social factors and the role of contextual factors should be used by hospitals to improve standardization practices through educational efforts, individualised training and motivational support. The influence of patient and family factors on healthcare professionals’ adaptive capacity should be investigated.

Trial registration

Open Science Framework ( https://osf.io/ev7az ) https://doi.org/10.17605/OSF.IO/EV7AZ .

Peer Review reports

Studies have shown discrepancies between hospital policies and procedures set to improve quality of care and their implementation by healthcare professionals [ 1 , 2 ]. Why do healthcare professionals not comply with institutional policies, protocols, guidelines, and checklists set to improve quality? Healthcare professionals may be well intentioned and strive to offer quality of care, but they also face challenges such as limited resources, increasing work pressure, and burnout [ 3 , 4 , 5 ]. Non-compliance is multifactorial due to the complexity of the healthcare system and the quantity of information and hospital policies. Some factors are linked to the individual healthcare professionals, e.g. training, beliefs, habits, psychological factors and other factors are contextual such as social norms, staff workload and competing goals between the individual and the institution.

Adaptation or adaptive capacity is seen as a main pillar in resilience across several disciplines [ 6 , 7 , 8 ]. Several studies have aimed at exploring and understanding how resilience contributes to healthcare professionals’ adaptive capacities towards challenging work conditions [ 9 , 10 , 11 ], but still it is poorly understood. According to Smaggus [ 2 ] hospital healthcare professionals proactively adapt to compensate for systemic problems such as protocols and technology poorly aligned with their tasks. They do so through their dedication, expertise, and creativity. However, these adaptations might come at a cost to professionals’ well-being as they often include working longer and more intense hours. In the context of this study healthcare professionals’ adaptive capacity is seen as essential for hospital standardization to be successfully practiced thus contributing to quality of care. Adaptations might come in the forms of compliance or non-compliance to standardized guidelines and protocols, or in the forms of adjustment of the contents of the standardization efforts.

There are certain factors that contribute to healthcare professionals’ adaptive capacity such as habits, stress, anxiety, burnout, coping mechanisms, motivation (internal and external), intention, level of knowledge and education, cognitive capacity, perceptions, attitudes, and beliefs (individual and social) [ 9 , 10 , 11 ] . No evidence was found of literature reviews exploring factors that contribute to healthcare professionals’ adaptive capacity with hospital standardization, except one focusing on hand hygiene guideline adherence [ 12 ] and one on nurses’ non-compliance in infection prevention [ 5 ].

Therefore, the aim of this scoping review is to identify and map the factors contributing to healthcare professionals’ adaptive capacity with hospital standardization. Specific research questions addressed by this review were:

1. In which hospital standardization practices have healthcare professionals’ adaptive capacity been studied?

2. What factors influence healthcare professionals’ adaptive capacity with hospital standardization and how can they be categorized?

A scoping review methodology was chosen because it provides a transparent approach to mapping relevant literature in emerging fields or topics [ 13 , 14 ] and has a broader “scope” and more expansive inclusion criteria than a systematic review [ 15 , 16 ]. It also allows for studies using different designs and methods to be included and synthesized, which was considered necessary for this review. We followed the methodological stages outlined by Arksey and O’Malley [ 13 ] and Levac et al.’s [ 17 ] enhancement to conduct the review. These were: (1) Identifying the research question, (2) Identifying relevant studies, (3) Study selection, (4) Charting the data, (5) Collating, summarizing and reporting the results, and (6) Consulting with relevant stakeholders. A review protocol was developed according to Peters et al. [ 14 ] and registered on October 11 th 2021 on the Open Science Framework ( https://osf.io/ev7az ) https://doi.org/10.17605/OSF.IO/EV7AZ . The reporting of the review follows the PRISMA-ScR Checklist [ 18 ] (Additional file 1 ).

Eligibility criteria

Articles were assessed against the following inclusion criteria: English-language, peer-reviewed research articles of any type published in scholarly journals where the full text was available, as well as grey literature not published in peer-reviewed journals. We chose to focus on healthcare professionals above 25 years indicating that they would have a minimum level of clinical experience including experiences with hospital standardization efforts. Furthermore, we chose the hospital setting to be able to possibly compare different standardization efforts identified. For a full description of inclusion and exclusion criteria, see Additional file 2 .

Information sources

The focus of the review was on peer-reviewed literature and electronic databases from different disciplines such as biomedicine, psychology, health services research, and nursing were searched on 12.10.2021 to identify relevant studies. The electronic databases searched included Scopus, MEDLINE (Ebsco & Ovid), Web of Science, CINAHL, EMBASE and PsycINFO [ 19 ]. Search terms were discussed by the three authors. The preliminary search strategy, search terms and inclusion/exclusion criteria were checked by a research librarian at the University of Stavanger.

The electronic database searches were then conducted by one researcher (FTK) and included all citations published before October 2021. A combination of Title, Subject, Subject headings, MeSH terms, and Keywords/Text words was used. The search strategy was adapted to individual databases. An example of a search strategy is presented in Table 1 .

To reduce the likelihood that relevant articles were overlooked we also hand searched reference lists of included articles and did an additional snowball-search. To further ensure that all relevant information was captured we conducted a targeted search of the grey literature in Google Scholar and in the following grey literature databases: Grey Literature Report ( https://www.greylit.org/ ) and OpenGrey ( http://www.opengrey.eu/ ). Hand searches, snowball-search, and grey literature resulted in an additional 19 records. All the search results were imported into EndNote bibliographic software and merged.

Selection of sources of evidence

The review process consisted of three levels of screening: (1) title, (2) abstract, and (3) full text. For the first level of screening, one researcher (FTK) screened the titles of retrieved citations. Abstract and full text screening involved two researchers (FTK and KA) who shared and independently assessed the articles to determine if they met the inclusion/exclusion criteria. Articles considered relevant by the reviewers were included in the full-text review. Discrepancies about study eligibility at the full-text review stage was solved through discussion for 14 studies. Consensus was achieved between the two researchers making it unnecessary to involve the third researcher (SW) at this stage.

Quality assessment of the included studies did not form part of the current scoping review [ 13 ]. Therefore, all studies were included in the analysis as they would potentially contribute to mapping the knowledge base.

Data charting process and data items

An electronic data charting form was developed in excel to guide data charting from included articles. Data concerning study characteristics, e.g., authors, year of publication, and the methodology, e.g., design, data collection, participants, results were charted in addition to information related to the aim of the review, i.e., hospital standardization type, factors of adaptive capacity, individual level, group/social level, and reviewers’ notes (Table 2 ).

The initial data charting sheet was validated by two reviewers (FTK and KA) with three articles each to corroborate consistency, as recommended by Daudt, van Mossel and Scott [ 20 ]. All data was extracted by two researchers (FTK, KA) independently and then agreed and merged with input and discussion by the third researcher (SW).

Synthesis of results

Results were synthesised and presented using frequency counting as well as summarised in text as categories. The data were compared and synthesised to summarise study characteristics, hospital standardization effort, and factors affecting healthcare professionals’ adaptive capacity. All three authors were involved in the process of synthesizing and describing the results in a suitable format.

Consultation with stakeholders

According to Levac et al. [ 17 ], we conducted the optional stage of consulting with relevant stakeholders once the results were synthesized. The researchers of the Centre for Resilience in Healthcare SHARE, University of Stavanger were identified as relevant and knowledgeable stakeholders on the topic. Hence, an email with draft results was sent to all 79 centre researchers with a request for feedback and input on any additional sources of information relevant to the research questions of the scoping study. Stakeholders’ literature input was sent to author FTK (one book and three studies) who then assessed the information. No additional studies were included.

The search resulted in 1414 unique records of which 180 full-text articles were assessed for eligibility and 57 were included in the review [ 12 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 ]. From the grey literature four articles met the inclusion criteria. The most common reasons for exclusion were no healthcare professional related factors described, mixed sample with unclear reporting of results for healthcare professionals, not in hospital setting or setting unclear, or not a healthcare professional sample.

A total of 27 qualitative studies, 20 quantitative studies, six literature reviews, and four mixed-method studies were included in the review. Figure  1 demonstrates the inclusion and exclusion of records at each stage of the screening process, using the PRISMA flow diagram [ 77 ].

PRISMA flow diagram

Characteristics of studies

The included studies were published between 2000 and 2021, with a tendency towards increased publication frequency over the last five-year period (2017–2021). The studies originated from 26 different countries with all continents represented. Several of the studies were conducted in Australia ( n  = 5), Jordan ( n  = 5), USA ( n  = 5), the Netherlands ( n  = 4), China ( n  = 3), and the UK ( n  = 3). Twenty-nine studies were conducted in developed countries while 19 in developing countries. A total of 9 studies had unspecified country origin, the reason usually being that they were literature reviews.

Hospital standardization practices

The review identified healthcare professionals’ adaptive capacity in numerous standardization practices across hospitals’ specialties. The most common standardization types were clinical or practice guidelines and protocols, precautions, procedures, checklists, policies, forms, and hospital-generic precautions, rules, and regulations. The most common areas of standardization were hand hygiene and infection prevention, personal protective equipment use, surgery, medication administration, cancer, mother and new-born care, falls, asthma, and pneumonia (Table 3 ). Three studies on personal protective equipment use were conducted during pandemics [ 51 , 52 , 54 ] while two studies explored infection prevention after pandemics [ 32 , 55 ].

Factors influencing healthcare professionals’ adaptive capacity

The factors influencing healthcare professionals’ adaptive capacity with hospital standardization practices were grouped in eight categories as described in Table 4 . The eight categories constitute a combination of individual and group/social factors deciding whether hospital healthcare professionals comply with or adapt hospital standardization practices. Below the eight categories are described in more detail.

Psychological and emotional factors

In many instances professionals choose to adhere to infection prevention guidelines due to the psychological pressure or fear of contracting or spreading infections [ 23 , 32 , 41 ]. Adverse incidents with infection prevention equipment are perceived as stressful and worrying [ 51 ], while following standard precautions would decrease their anxiety [ 41 ].

Healthcare professionals’ sense of invulnerability and confidence would lead them to adapt hospital standardization [ 22 , 31 , 55 , 63 , 75 ]. Reasons would differ from physicians considering themselves to be entitled to work independently without protocols to guide them [ 31 ], to professionals feeling minor concerns for infection transmission over time when not acquiring any infection [ 55 ] to nurses expressing psychological gratification about their own ability to creatively solve problems and work around standardization practices  [ 75 ].

Cognitive factors

Various attitudes and beliefs were reported in several studies as influencing healthcare professionals’ adaptive capacity or compliance to hospital standardization [ 28 , 29 , 33 , 37 , 41 , 45 , 47 , 48 , 52 , 55 , 61 , 62 , 69 , 73 , 75 ]. Some of these attitudes and beliefs were patient related [ 41 , 45 , 52 , 62 , 69 ,  74 , 75 ]. For example, in a study on the use of personal protective equipment in the emergency room during the COVID-19 pandemic, there was only a slight difference between healthcare professional beliefs on whether the equipment was protective or not for patients (52% vs. 46%) [ 52 ]. In another study, physicians believed that diarrhoea was a low-risk disease with overrepresentation among poor people but adapted the diarrhoea treatment to patients with higher social status [ 45 ]. Moreover, nurses justified adaptations when they believed that use of gloves or masks was an exaggeration when treating children considered low risk and not contagious [ 41 ].

Attitudes and beliefs were also related to the professional group that healthcare professionals belonged to [ 29 , 61 ]. Physicians and nurses had opposing views on surgical count protocol violations and what constitutes safe clinical practice [ 61 ]. Nurses believed that physicians had lower hand hygiene compliance, while physicians believed that they were role models and leaders of hand hygiene and would warn other staff members [ 29 ].

Healthcare professionals’ agreement or disagreement with specific guidelines also led to adaptations of hospital standardization. Adaptations were made when for example radiologists considered diagnostic imaging guidelines not useful, too rigid, or that they failed to include specific information about changes [ 47 ] or were perceived as inefficient or unnecessary [ 75 ], not relevant for the clinical practice [ 37 , 64 ], or not relevant for certain care systems [ 48 ], or when healthcare professionals doubted the effectiveness of isolation precautions to prevent disease contagion [ 55 ]. Similarly, a systematic review reported adaptations when guidelines and other standardized practices were considered too generic, promoted 'cookbook medicine', oversimplified, difficult or controversial treatment decisions, or when the evidence they were based on was conflicting [ 62 ]. By contrast healthcare professionals complied to standards, protocols and guidelines when believed to be useful tools [ 49 ] in clinical decision making and providing uniform care [ 48 , 62 ], were easy to understand, highly relevant to clinical practice and patient population, and based on credible information sources [ 62 ].

Motivational factors

Motivational factors for healthcare professionals’ adaptive capacity are mainly reported in studies on infection prevention and control. Personal motivational drivers such as moral responsibility, obligation and duty are reported by different professional groups related to infection control practices in the emergency room and on hospital wards in general [ 26 , 33 ], within tuberculosis infection control measures [ 22 ], within hand hygiene obligations [ 29 ], within venous thromboembolism prophylaxis management [ 46 ], within respiratory infectious diseases [ 24 ], and with clinical practice guidelines to prevent falls and injuries [ 71 ]. The moral responsibility would be directed towards themselves as professionals to reduce transmission of pathogens or expressed as a duty of care to their patients. Motivational factors were most often internally driven in the sense of professionals’ own intent and feelings of psychological safety [ 30 , 75 ], while some studies reported on external drivers such as a motivational person in their organization [ 71 ], the intensity of activity in the clinical setting [ 30 ] or organizational neglect of occupational health and safety [ 60 ].

Healthcare professionals’ comfort or discomfort with personal protective equipment would influence their motivation to adapt infection prevention and control standardization practices [ 24 , 33 , 41 , 51 , 53 , 56 ].

Knowledge and experience factors

Knowledge and training of standardization practices was described as important and increased compliance among nurses [ 34 , 42 , 43 ] and younger physicians [ 55 ], but was not seen as sufficient for physicians and nurses in other studies [ 25 , 44 , 47 , 50 , 55 , 58 , 59 ].

Length and type of clinical experience would often lead healthcare professionals to either adapt hospital standardization [ 35 , 41 , 55 , 59 , 61 , 67 , 73 ] or to comply with it [ 43 , 68 , 71 ]. For example, experienced senior nurses had more confidence to adapt protocols in intensive care units [ 59 ] or during fever management [ 73 ], than less experienced nurses with barcode medication administration technology [ 68 ]. Among surgical team members, physicians relied on their experience and tactical knowledge [ 61 ], nurses on their repeated experience of working daily with the same instrument trays [ 59 ] and disregarded surgical count guidelines or made workarounds on surgical safety checklist use [ 58 ]. However, in another study increased length of experience was reported as a contributing factor to compliance with hand hygiene for both physicians and nurses [ 43 ].

Insufficient knowledge and training led healthcare professionals to make adaptations. For example, knowledge deficits about tuberculosis led healthcare professionals to use ineffective measures in preventing transmission [ 33 ]. Similarly, midwives’ limited knowledge of aspects of infection prevention control guidelines [ 23 ] or being unaware that national postnatal care protocols had been updated led them to make adaptations based on inappropriate experiential knowledge [ 60 ].

For some healthcare professionals, negative experiences during clinical practice increased their compliance with clinical guidelines [ 71 ], while experience of lack of consequences led them to continue their adaptations of universal precautions [ 55 ].

Professional role factors

In many cases, the clinical role or profession of healthcare workers influenced their ability and desire to adapt hospital standardization [ 12 , 55 , 57 , 60 , 75 ]. Nurses defined problem solving as part of their job thus contributing to workarounds from standardization practices [ 75 ]. Physicians often defined their role in authoritative ways contributing to lower compliance with hospital standardization than other professions, for example within hand hygiene [ 12 ], the surgical safety checklist [ 57 , 60 ], and MRSA precautions [ 55 ]. In one study, professional status and reputation were identified to influence physicians’ clinical decision-making [ 45 ]. Healthcare professionals’ perception of their own roles also challenged their possibility to intervene in each other’s work tasks and their ability to collectively adapt standardization practices [ 45 , 57 ].

Furthermore, autonomy and clinical and/or professional judgment were seen as vital elements of healthcare professionals’ adaptive capacity [ 39 , 59 , 62 , 75 ]. For example, commitment to infection prevention and control was high in a neonatal unit, however, severely constrained resources made improvisation a vital element of professionals’ clinical judgment and adaptive capacity [ 23 ].

Risk management factors

Healthcare professionals adapted their practices to meet hospital standardization due to individual perceptions of risk and belonging personal costs. They adhered to infection prevention guidelines to protect themselves from being infected or from infecting family and others [ 24 , 27 , 29 , 33 , 41 , 53 , 56 ], or they wanted to avoid reprimands and litigations [ 46 , 62 , 72 ] or negative media attention [ 71 ]. The perceived risk for reprimands or litigations might for example lead to nurses performing fall prevention according to the guideline “just in case” even with non-risk patients [ 71 ] p90. The same goes for physicians sending patients for x-rays “just to be safe” [ 71 ] p90. However, in another study perceived enforcement of rules in the form of monitoring and threats of punishment or sanctions had no direct or indirect effect on physicians’ compliance [ 40 ].

Clinical practice guidelines were adapted or disregarded if healthcare professionals perceived them to constitute a potential risk to patients [ 59 , 67 , 69 , 75 ]. This could involve physicians using more highly concentrated medications than recommended to prevent fatal arrhythmia [ 69 ] or nurses to disregard the protocol for surgical count of instruments in life-threatening emergencies [ 59 ].

Patient and family factors

The main reason for healthcare professionals wanting to adapt hospital standardization was to meet patient needs. In general, this involved deviations from hospital guidelines or policies when they saw them as barriers to patient care and/or patient safety. Patient needs were exemplified as timely care, patient-centred care, quality of patient communication, privacy, improved outcomes [ 36 , 55 , 56 , 67 , 74 , 75 ] and customized care [ 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 ]. Several studies related to infection prevention including three during pandemic situations pointed at adaptations made to personal protective equipment protocols to improve patient communication, reduce patients’ feeling of isolation, and better establish therapeutic relationships [ 22 , 52 , 54 , 55 , 56 ]. This was especially relevant for older patients [ 55 ] and children [ 41 , 56 ]. In emergencies, workarounds of protocols were justified not to jeopardise the patients’ safety [ 75 ], while in surgical settings compliance with the checklist protocol was seen as preserving patient safety [ 60 ].

Family factors were related to presence and expectations, and cultural conflicts. Examples of adaptations span from clinicians not complying with the family witnessed resuscitation protocol as they value it as traumatic for relatives with risk of PTSD [ 72 ] to pressure for antibiotics and intravenous fluids in diarrheal management [ 45 ] to disapproval of pre-operative skin preparation policy due to cultural preferences [ 76 ].

Work relationship factors

Most studies reporting on work relationship factors were related to conditions negatively affecting the adaptive capacity of healthcare professionals such as power issues, group norms, hierarchical relationships, and breakdown in communication [ 21 , 23 , 31 , 41 , 57 , 59 , 61 , 65 ]. This could entail surgeons’ power influencing the practice of the surgical count procedure negatively where nurses felt unable to demand to undertake the count even though it constitutes a crucial safeguard for the outcome of the surgery [ 59 , 61 ]. Hierarchical relationships were shown to negatively affect the use of the safe surgery checklist as surgeons and anaesthetists would disincline to volunteer information and openly communicate with each other and other team members [ 57 ]. Breakdown in communication was identified to negatively influence healthcare professionals’ adaptive capacity within antimicrobial prophylaxis [ 21 ], postnatal care protocols [ 60 ], and infection outbreaks [ 55 ].

A few studies reported on positive effects of work relationship factors such as peer pressure in the forms of healthcare professionals’ reminding each other to wear protective equipment [ 24 ], physicians acting as positive role models to other staff members on hand hygiene [ 29 ], nurse leaders modelling practicing safety rounds to staff [ 38 ] or collegial support from senior medical and nursing staff to junior professionals to improve adherence to standardization practices in the emergency department [ 70 ].

Contextual factors influencing healthcare professionals’ adaptive capacity

Based on our synthesis of studies we identified several contextual factors that influenced healthcare professionals’ adaptive capacity with hospital standardization. These were factors “outside” the individual and group/social level. Even though the review did not focus specifically on the organizational or institutional level, the contextual factors formed parts of healthcare professionals’ explanations for degree of adaptation or compliance with hospital standardization.

Guideline “system”

Some studies described characteristics of the guidelines per se to influence how healthcare professionals adapted to them or not [ 24 , 47 , 48 , 68 , 71 ]. For example, guidelines that were too long and ambiguous or outdated and unclear [ 24 , 47 ] or complex [ 64 ] were considered as barriers as healthcare professionals were confused and unsure how to adhere to them. Moreover, constantly changing guidelines given the time restrictions of daily clinical practice overwhelmed healthcare professionals who could not keep up with the updates or changes [ 24 ]. In addition, insufficient guidelines which lacked specific information were seen as a barrier and practical implementation depended on the healthcare professionals’ expertise [ 48 ].

However, high usability and guidelines that reflected national or international guidelines facilitated healthcare professionals’ compliance [ 24 , 68 ].

Cultural norms

Workplace culture was described to influence adaptation or compliance with hospital standardization [ 24 , 33 , 41 ]. For example, adaptations were made when standard precautions were not the routine practice in the clinical department [ 41 ], when there was complacency to infection prevention control guidelines [ 24 ] or when workplace culture was part of a national culture [ 33 ]. When hospital standardization practices were followed by senior colleagues [ 41 ] or by all staff the compliance was high [ 24 ].

Leadership support

Several studies reported that the level of adaptation or compliance with hospital standardization was influenced by the level of support healthcare professionals received by their clinical leadership [ 24 , 29 , 33 , 60 , 62 , 65 , 71 ]. Leadership support was understood as visibility, encouragement, and modeling compliance with standardization practices [ 24 , 29 , 38 , 65 ].

Physical environment

Healthcare professionals described various factors in the physical environment that led to adaptations of hospital standardization practices [ 24 , 45 , 62 , 67 , 71 ]. For example, limited access to treatment services and facilities [ 62 ], wards being too crowded, noisy, and dirty [ 45 ], lack of adequate ventilation, isolation rooms, and shower facilities to prevent infection transmission [ 24 ], narrow hospital bathrooms [ 71 ], or lack of vital space in examination cubicles [ 60 ].

Time constraints were in several studies described as a reason for adaptations [ 29 , 36 , 48 , 49 , 57 , 58 ]. For example, during emergencies there was no time to either perform proper hand hygiene or proper use of gloves [ 29 ], or to perform time-out procedures or safety checklists during surgical operations [ 57 ]. However, a systematic review suggested that implementation of the surgical safety checklist reduced time delays as miscommunication and confusion were avoided [ 60 ].

Workload issues

Increased workload was mainly reported in studies on infection prevention practices to explain healthcare professionals’ adaptations of hospital standardization [ 12 , 21 , 24 , 29 , 55 ]. Similarly, in a study on perinatal care obstetricians reported that they were more likely to comply with changes in practice if their workloads did not increase [ 34 ]. Staff shortages leading to demanding workloads was also a contributing factor for midwives to collectively decide not to update their knowledge of the new post-natal care protocols, despite training being offered [ 60 ].

In this paper we have reviewed the literature to identify the factors contributing to healthcare professionals’ adaptive capacity with hospital standardization. We have documented that adaptive capacity is multidimensional according to eight factors: psychological and emotional, cognitive, motivational, knowledge and experience, professional role, risk management, patient and family, and work relationships. This multidimensional aspect is supported by Huey and Palaganas’ [ 11 ] emphasizing the influence of individual and workplace cultural factors. Individual traits such as having a higher purpose is in our review specified as motivational, emotional, cognitive, and knowledge-based factors. Our review also adds group/social factors including work relationship, professional role, and physical environment, in line with Toode, Routasalo and Suominen [ 78 ]. New in this study is the establishment of the patient and family factor as a main driver for healthcare professionals’ adaptation of hospital standardization.

The eight factors of adaptive capacity are situated within a contextual setting, described by healthcare professionals as the background for their adaptation. Time and workload issues were most frequently described in studies on infection prevention and control and in surgery, with different reasoning. The time issue in infection prevention and control is related to the time-consuming and resource intensive procedures, while in surgery the time issue is related to urgency and acute situations. Both contexts might lead to a need for healthcare professionals having to adapt protocols and guidelines. Individual factors are indisputable engrained in the contextual surroundings meaning that healthcare professionals’ adaptive capacity needs to be understood in light of the guideline system, cultural norms, leadership support, time and workload issues, and the physical environment. This is in line with previous research on the role of context in healthcare [ 79 , 80 ].

This scoping review covers 26 different countries representing all continents of which 29 studies are from developed countries and 19 from developing countries. We did notice some variation in the extent and type of details reported by healthcare professionals across regions and countries. However, a geographic comparison was not included in our scope and future research comparing continents, regions, or countries based on their economic status and healthcare professionals’ adaptive capacity should be conducted.

In our review infection prevention and control and practices related to hand hygiene and use of personal protective equipment stand out as the most common standardization practices studied. This is an area with clear individual and organizational targets thus requiring a combination of individual and organizational adaptive capacity [ 81 ]. Based on our review and previous research we claim that the field of adaptive capacity and resilience would benefit from incorporating knowledge on individual factors to succeed [ 82 , 83 , 84 ]. Adaptive capacity for healthcare professionals and healthcare organizations seems to depend on each other more than the research has acknowledged so far and should be the topic of future research.

Implications for hospital management and practice

The new knowledge from this review on individual, group/social factors and contextual factors influencing healthcare professionals’ adaptive capacity should be used by hospitals to improve and tailor make current standardization practices. Efforts should be made to construct educational efforts, individualise training and motivational support, and to address the role of patients and families as the main driver for healthcare professionals’ adaptation of hospital standardization.

Based on the findings of this scoping review, complex standardisation practices should be revised to be easily understandable, as short as possible, and relevant to the professional practice. Healthcare professionals should be involved in standardization development and/or revisions to achieve this. Educational efforts to inform healthcare professionals on new or revised standardisation practices should integrate knowledge building not only on the standardisation measure in itself, but also on how individual, group/social, and contextual factors promote or hamper their compliance to or adaptation of it. This is especially important within the areas of infection control and personal protective equipment as the evidence for healthcare professionals’ adaptation is strong.

Strengths and limitations

The review was conducted in accordance with an acknowledged framework for scoping reviews and the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines. The scoping methodology allowed information from a broad range of studies, using different designs and methods, to be included and synthesized. The findings highlight individual and group/social factors for healthcare professionals to comply with or adapt hospital standardization practices.

The review was limited to English-speaking literature and the included studies were not assessed for quality. The review does not provide a definitive account of the successful outcomes of healthcare professionals’ adaptive capacity towards standardization practices. Moreover, this scoping review focused on individual factors and group/social factors and did not include a full review of organizational factors as this is done in other studies. Finally, it was not possible to draw any conclusions on how pandemics influence healthcare professionals’ adaptive capacity, as only five studies met the inclusion criteria. This should be followed-up with further research.

Conclusions

Our study identified the following hospital standardization practices where healthcare professionals’ adaptive capacity has been studied: clinical guidelines and protocols, precautions, procedures, checklists, policies, forms, and hospital-generic precautions, rules, and regulations. These are typically studied within infection prevention and control, followed by more disease-specific areas such as for example cancer.

There has been lack of knowledge on factors impacting on healthcare professionals’ adaptation or compliance with hospital standardization. This scoping review stands out by identifying a multidimensional set of eight factors at the individual and group/social level. The main factor influencing healthcare professionals’ adaptation of hospital standardization was patient and family needs. The review also identified surrounding contextual factors influencing healthcare professionals’ adaptive capacity including the guideline system, cultural norms, leadership support, physical environment, time, and workload issues.

Future research needs to investigate the relationship between individual factors for adaptive capacity and their contextual setting, as well as the relationship between individual, group/social, and organizational factors.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews

Medical Subject Headings

Methicillin-resistant Staphylococcus aureus

Post traumatic stress disorder

Arts DL, Voncken AG, Medlock S, Abu-Hanna A, van Weert HC. Reasons for intentional guideline non-adherence: a systematic review. Int J Med Inform. 2016;89:55–62. https://doi.org/10.1016/j.ijmedinf.2016.02.009 .

Smaggus A. Safety-I, safety-II and burnout: how complexity science can help clinician wellness. BMJ Qual Saf. 2019;28(8):667. https://doi.org/10.1136/bmjqs-2018-009147 .

Article   PubMed   Google Scholar  

Ralefala T, Mokokwe L, Jammalamadugu S, Legobere D, Motlhwa WS, Oyekunle AA, et al. Provider Barriers and Facilitators of Breast Cancer Guideline-Concordant Therapy Delivery in Botswana: A Consolidated Framework for Implementation Research Analysis. Oncologist. 2021:26(12).  https://doi.org/10.1002/onco.13935 .

Pastores S, Kvetan V, Coopersmith C, Farmer JC, Sessler C, Christman J, et al. Workforce, Workload, and Burnout Among Intensivists and Advanced Practice Providers. Crit Care Med. 2019;47(4):550–7.

McCauley L, Kirwan M, Matthews A. The factors contributing to missed care and non-compliance in infection prevention and control practices of nurses: A scoping review. IJNS Advances 2021;3. https://doi.org/10.1016/j.ijnsa.2021.100039 .

Atkinson PA, Martin CR, Rankin J. Resilience revisited. J Psychiatr Ment Health Nurs. 2009;16:137–45. https://doi.org/10.1111/j.1365-2850.2008.01341.x .

Article   CAS   PubMed   Google Scholar  

Southwick SM, Bonanno GA, Masten AS, Panter-Brick C, Yehuda R. Resilience definitions, theory, and challenges: interdisciplinary perspectives. Eur J Psychotraumatol. 2014;1(5). https://doi.org/10.3402/ejpt.v5.25338 .

Wiig S, Aase K, Billett S, Canfield C, Roise O, Nja O, et al. Defining the boundaries and operational concepts of resilience in the resilience in healthcare research program. BMC Health Serv Res. 2020;20(1):330. https://doi.org/10.1186/s12913-020-05224-3 .

Article   PubMed   PubMed Central   Google Scholar  

Arrogante O, Aparicio-Zaldivar E. Burnout and health among critical care professionals: the mediational role of resilience. Intensive Crit Care Nurs. 2017;42:110–5. https://doi.org/10.1016/j.iccn.2017.04.010 .

McCann CM, Beddoe E, McCormick K, Huggard P, Kedge S, Adamson C, et al. Resilience in the health professions: a review of recent literature. International Journal of Wellbeing. 2013;3(1):60–81.

Article   Google Scholar  

Huey CWT, Palaganas JC. What are the factors affecting resilience in health professionals? a synthesis of systematic reviews. Med Teach. 2020;42(5):550–60. https://doi.org/10.1080/0142159X.2020.1714020 .

Erasmus V, Daha TJ, Brug H, Richardus JH, Behrendt MD, Vos MC, et al. Systematic review of studies on compliance with hand hygiene guidelines in hospital care. Infect Control Hosp Epidemiol. 2010;31(3):283–94.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32. https://doi.org/10.1080/1364557032000119616 .

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13(3):141–6.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143. https://doi.org/10.1186/s12874-018-0611-x .

Tricco AC, Zarin W, Ghassemi M, Nincic V, Lillie E, Page MJ, et al. Same family, different species: methodological conduct and quality varies according to purpose for five types of knowledge synthesis. J Clin Epidemiol. 2018;96:133–42.

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69. https://doi.org/10.1186/1748-5908-5-69 .

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73. https://doi.org/10.7326/M18-0850 .

Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6(1):245. https://doi.org/10.1186/s13643-017-0644-y .

Daudt H, van Mossel C, Scott S. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. BMC Med Res Methodol. 2013;13:48. https://doi.org/10.1186/1471-2288-13-48 .

Bonfait H, Delaunay C, de Thomasson E, Charrois O. Near-miss event assessment in orthopedic surgery: Antimicrobial prophylaxis noncompliance. Orthop Traumatol Surg Res. 2010;96(5):493–9. https://doi.org/10.1016/j.otsr.2010.03.014 .

Chapman HJ, Veras-Estevez BA, Pomeranz JL, Perez-Then EN, Marcelino B, Lauzardo M. The Role of Powerlessness Among Health Care Workers in Tuberculosis Infection Control. Qual Health Res. 2017;27(14):2116–27. https://doi.org/10.1177/1049732317731317 .

Herbec A, Chimhini G, Rosenberg-Pacareu J, Sithole K, Rickli F, Chimhuya S, et al. Barriers and facilitators to infection prevention and control in a neonatal unit in Zimbabwe - a theory-driven qualitative study to inform design of a behaviour change intervention. J Hosp Infect. 2020;106(4):804–11. https://doi.org/10.1016/j.jhin.2020.09.020 .

Houghton C, Meskell P, Delaney H, Smalle M, Glenton C, Booth A, et al. Barriers and facilitators to healthcare workers' adherence with infection prevention and control (IPC) guidelines for respiratory infectious diseases: A rapid qualitative evidence synthesis. Cochrane Database Syst Rev. 2020;4:1–55. https://doi.org/10.1002/14651858.CD013582 .

Suliman M, Aloush S, Aljezawi M, AlBashtawy M. Knowledge and practices of isolation precautions among nurses in Jordan. Am J Infect Control. 2018;46(6):680–4. https://doi.org/10.1016/j.ajic.2017.09.023 .

Chapman HJ, Veras-Estévez BA, Pomeranz JL, Pérez-Then EN, Marcelino B, Lauzardo M. Perceived Barriers to Adherence to Tuberculosis Infection Control Measures among Health Care Workers in the Dominican Republic. MEDICC Rev. 2017;19(1):16–22. https://doi.org/10.37757/MR2017.V19.N1.4 .

Darawad MW, Al-Hussami M, Almhairat, II, Al-Sutari M. Investigating Jordanian nurses' handwashing beliefs, attitudes, and compliance. Am J Infect Control. 2012;40(7):643–7. https://doi.org/10.1016/j.ajic.2011.08.018 .

Jenner EA, Fletcher BC, Watson P, Jones FA, Miller L, Scott GM. Discrepancy between self-reported and observed hand hygiene behaviour in healthcare professionals. J Hosp Infect. 2006;63(4):418–22. https://doi.org/10.1016/j.jhin.2006.03.012 .

McLaws ML, Farahangiz S, Palenik CJ, Askarian M. Iranian healthcare workers' perspective on hand hygiene: A qualitative study. J Infect Public Health. 2015;8(1):72–9. https://doi.org/10.1016/j.jiph.2014.05.004 .

O'Boyle CA, Henly SJ, Larson E. Understanding adherence to hand hygiene recommendations: the theory of planned behavior. Am J Infect Control. 2001 Dec;29(6):352–60. https://doi.org/10.1067/mic.2001.18405 .

Shah N, Castro-Sanchez E, Charani E, Drumright LN, Holmes AH. Towards changing healthcare workers’ behaviour: a qualitative study exploring non-compliance through appraisals of infection prevention and control practices. J Hosp Infect. 2015;90(2):126–34.

Wang Y, Yang J, Qiao F, Feng B, Hu F, Xi ZA, et al. Compared hand hygiene compliance among healthcare providers before and after the COVID-19 pandemic: A rapid review and meta-analysis. Am J Infect Control. 2021. 50(5):563–571. https://doi.org/10.1016/j.ajic.2021.11.030 .

Woith W, Volchenkov G, Larson J. Barriers and motivators affecting tuberculosis infection control practices of Russian health care workers. Int J Tuberc Lung Dis. 2012;16(8):1092–6. https://doi.org/10.5588/ijtld.10.0779 .

Al-Faouri I, Okour SH, Alakour NA, Alrabadi N. Knowledge and compliance with standard precautions among registered nurses: A cross-sectional study. Ann Med Surg. 2021;62:419–24. https://doi.org/10.1016/j.amsu.2021.01.058 .

Flynn AV, Sinclair M. Exploring the relationship between nursing protocols and nursing practice in an Irish intensive care unit. Int J Nurs Pract. 2005;11(4):142–9. https://doi.org/10.1111/j.1440-172X.2005.00517.x .

Kakeeto M, Lundmark R, Hasson H, von Thiele Schwarz U. Meeting patient needs trumps adherence. A cross-sectional study of adherence and adaptations when national guidelines are used in practice. J Eval Clin Pract. 2017;23(4):830–8. https://doi.org/10.1111/jep.12726 .

Kotzeva A, Guillamon I, Gracia J, del Campo PD, Gich I, Calderon E, et al. Use of clinical practice guidelines and factors related to their uptake: a survey of health professionals in Spain. J Eval Clin Pract. 2014;20(3):216–24.

Meyer P, Hill C, Baker D. Standardizing Nurse Leader Safety Rounds to Promote Highly Reliable Care. J Nurs Care Qual. 2020;35(3):252–7.

Nissinboim N, Naveh E. Process standardization and error reduction: A revisit from a choice approach. Saf Sci. 2018;103:43–50.

Weske U, Boselie P, van Rensen E, Schneider M. Physician compliance with quality and patient safety regulations: The role of perceived enforcement approaches and commitment. Health Serv Manage Res. 2019;32(2):103–12.

Efstathiou G, Papastavrou E, Raftopoulos V, Merkouris A. Factors influencing nurses' compliance with Standard Precautions in order to avoid occupational exposure to microorganisms: a focus group study. BMC Nurs. 2011;10(1). https://doi.org/10.1186/1472-6955-10-1 .

Luo Y, He GP, Zhou JW, Luo Y. Factors impacting compliance with standard precautions in nursing, China. Int J Infect Dis. 2010;14(12):E1106–E14. https://doi.org/10.1016/j.ijid.2009.03.037 .

Nofal M, Subih M, Al-Kalaldeh M. Factors influencing compliance to the infection control precautions among nurses and physicians in Jordan: A cross-sectional study. J Infect Prev. 2017;18(4):182–8.

Adeniyi B, Ilesanmi O, Obaseki D, Desalu O, Betiku B, Erhabor G. Relationship between knowledge and quality of asthma care among physicians in South-West Nigeria. Niger J Clin Pract. 2017;20(5):566–72.

Biswas D, Hossin R, Rahman M, Bardosh KL, Watt MH, Zion MI, et al. An ethnographic exploration of diarrheal disease management in public hospitals in Bangladesh: From problems to solutions. Soc Sci Med. 2020;260:12. https://doi.org/10.1016/j.socscimed.2020.113185 .

Chapman NH, Lazar SP, Fry M, Lassere MN, Chong BH. Clinicians adopting evidence based guidelines: a case study with thromboprophylaxis. BMC Health Serv Res. 2011;11:240. https://doi.org/10.1186/1472-6963-11-240 .

Gransjøen AM, Wiig S, Lysdahl KB, Hofmann BM. Barriers and facilitators for guideline adherence in diagnostic imaging: an explorative study of GPs' and radiologists' perspectives. BMC Health Serv Res. 2018;18(1):556. https://doi.org/10.1186/s12913-018-3372-7 .

Heutinck L, van Gameren M, Verschuuren J, Geurts ACH, Jansen M, de Groot IJM. Clinical Management of Duchenne Muscular Dystrophy in the Netherlands: Barriers to and Proposals for the Implementation of the International Clinical Practice Guidelines. J NeuromusculDis. 2021;8(4):503–12.

Google Scholar  

Talarico R, Marinello D, Bombardieri S, Burmester G, Fonseca J, Frank C, et al. Clinical practice guidelines adherence, knowledge and awareness in rare and complex connective tissue diseases across Europe: results from the first ERN ReCONNET survey. RMD Open. 2020;6(2):12.

Halm EA, Atlas SJ, Borowsky LH, Benzer TI, Metlay JP, Chang YC, et al. Understanding physician adherence with a pneumonia practice guideline: effects of patient, system, and physician factors. Arch Intern Med. 2000;160(1):98–104.

Chen FF, Zang YL, Liu Y, Wang XM, Lin XF. Dispatched nurses' experience of wearing full gear personal protective equipment to care for COVID-19 patients in China-A descriptive qualitative study. J Clin Nurs. 2021;30(13-14):2001–14. https://doi.org/10.1111/jocn.15753 .

Seitz RM, Yaffee AQ, Peacock E, Moran TP, Pendley A, Rupp JD. Self-Reported Use of Personal Protective Equipment among Emergency Department Nurses, Physicians and Advanced Practice Providers during the 2020 COVID-19 Pandemic. Int J Environ Res Public Health. 2021;18(13):6. https://doi.org/10.3390/ijerph18137076 .

Chughtai AA, Seale H, Dung TC, Hayen A, Rahman B, Raina MacIntyre C. Compliance with the Use of Medical and Cloth Masks Among Healthcare Workers in Vietnam. Ann Occup Hyg. 2016;60(5):619–30. https://doi.org/10.1093/annhyg/mew008 .

Hu X, Zhang Z, Li N, Liu D, Zhang L, He W, et al. Self-reported use of personal protective equipment among Chinese critical care clinicians during 2009 H1N1 influenza pandemic. PLoS One. 2012;7(9):e44723. https://doi.org/10.1371/journal.pone.0044723 .

Moore D, Gamage B, Bryce E, Copes R, Yassi A, BC Interdisciplinary Respiratory Protection Study Group. Protecting health care workers from SARS and other respiratory pathogens: organizational and individual factors that affect adherence to infection control guidelines. Am J Infect Control. 2005;33(2):88–96. https://doi.org/10.1016/j.ajic.2004.11.003 .

Seale H LJ, Gallard J, Kaur R, Chughtai AA, Tashani M, et, al. “The cookie monster muffler”: perceptions and behaviours of hospital healthcare workers around the use of masks and respirators in the hospital setting. Int J Infect Control. 2014;11(1). https://doi.org/10.3396/ijic.v11i1.13704 .

Braaf S, Manias E, Riley R. The "time-out' procedure: an institutional ethnography of how it is conducted in actual clinical practice. BMJ Qual Saf. 2013;22(8):647–55. https://doi.org/10.1136/bmjqs-2012-001702 .

Gillespie BM, Marshall AP, Gardiner T, Lavin J, Withers TK. Impact of workflow on the use of the Surgical Safety Checklist: a qualitative study. ANZ J Surg. 2016;86(11):864–7. https://doi.org/10.1111/ans.13433 .

Riley R, Manias E, Polglase A. Governing the surgical count through communication interactions: implications for patient safety. Qual Saf Health Care. 2006;15(5):369–74. https://doi.org/10.1136/qshc.2005.017293 .

Wangoo L, Ray RA, Ho YH. Compliance and Surgical Team Perceptions of WHO Surgical Safety Checklist; Systematic Review. Int Surg. 2016;101(1-2):35–49.

Warwick VR, Gillespie BM, McMurray A, Clark-Burg KG. The patient, case, individual and environmental factors that impact on the surgical count process: An integrative review. J Perioper Nurs. 2019;32(3):9–19. https://doi.org/10.26550/2209-1092.1057 .

Bierbaum M, Rapport F, Arnolda G, Nic Giolla Easpaig B, Lamprell K, Hutchinson K, et al. Clinicians' attitudes and perceived barriers and facilitators to cancer treatment clinical practice guideline adherence: A systematic review of qualitative and quantitative literature. Implement Sci. 2020;15(1). https://doi.org/10.1186/s13012-020-00991-3 .

te Boveldt N, Vernooij-Dassen M, Besse K, Vissers K, Engels Y. Adaptation of an evidence-based clinical practice guideline in cancer pain management by medical oncologists: a case vignette study. Support Care Cancer. 2015;23(5):1409–20. https://doi.org/10.1007/s00520-014-2472-0 .

Ismaila N, Salako O, Mutiu J, Adebayo O. Oncology guidelines usage in a low-and middle-income country. J Glob Oncol. 2018;4:1–6.

PubMed   Google Scholar  

Belizan M, Meier A, Althabe F, Codazzi A, Colomar M, Buekens P, et al. Facilitators and barriers to adoption of evidence-based perinatal care in Latin American hospitals: a qualitative study. Health Educ Res. 2007;22(6):839–53.

Muhindo M, Bress J, Kalanda R, Armas J, Danziger E, Kamya MR, et al. Implementation of a Newborn Clinical Decision Support Software (NoviGuide) in a Rural District Hospital in Eastern Uganda: Feasibility and Acceptability Study. JMIR Mhealth Uhealth. 2021;9(2):16.

Yevoo LL, Agyepong IA, Gerrits T, van Dijk H. Improvisation and harm avoidance: An ethnographic study of adherence to postnatal care protocols in two hospitals in Southern Ghana. Midwifery. 2020;82:11. https://doi.org/10.1016/j.midw.2019.102576 .

Othman EH, Darawad MW. Nurses’ Compliance With Bar-code Medication Administration Technology: Results of Direct Observation of Jordanian Nurses’ Practice. Comput Inform Nurs. 2020;38(5):256–62.

Uema A, Kitamura H, Nakajima K. Adaptive behavior of clinicians in response to an over-constrained patient safety policy on the administration of concentrated potassium chloride solutions. Saf Sci. 2020;121:529–41. https://doi.org/10.1016/j.ssci.2019.09.023 .

McEwan H, Baker R, Armstrong N, Banerjee J. A qualitative study of the determinants of adherence to NICE falls guideline in managing older fallers attending an emergency department. Int J Emerg Med. 2018;11(1):33 https://doi.org/10.1186/s12245-018-0192-9 .

Stenberg M, Wann-Hansson C. Health care professionals' attitudes and compliance to clinical practice guidelines to prevent falls and fall injuries. Worldviews Evid Based Nurs. 2011;8(2):87–95. https://doi.org/10.1111/j.1741-6787.2010.00196.x .

Johnson C. A literature review examining the barriers to the implementation of family witnessed resuscitation in the Emergency Department. Int Emerg Nurs. 2017;30:31-5. https://doi.org/10.1016/j.ienj.2016.11.001 .

Thompson HJ, Kagan SH. Clinical management of fever by nurses: doing what works. J Adv Nurs. 2011;67(2):359–70. https://doi.org/10.1111/j.1365-2648.2010.05506.x .

Toxopeus DCM, Pell CL, Westrhenen NB-v, Smit C, Wit FWNM, Ondoa P, et al. Compliance with laboratory monitoring guidelines in outpatient HIV care: a qualitative study in the Netherlands. AIDS Care. 2019;31(7):840–7. https://doi.org/10.1080/09540121.2018.1563280 .

Debono DS, Greenfield D, Travaglia JF, Long JC, Black D, Johnsonet J. et al. Nurses' workarounds in acute healthcare settings: a scoping review. BMC Health Serv Res. 2013;13(175). https://doi.org/10.1186/1472-6963-13-175 .

Yami AAM. Using the Theory of Planned Behaviour to explore the intentions of a multicultural nursing workforce to comply with policies and procedures in the Prince Sultan Military Medical City (PSMMC) [Doctoral Thesis]: University of Southampton Institutional Repository; 2015. Available from: http://eprints.soton.ac.uk/id/eprint/381574 .

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097 .

Toode K, Routasalo P, Suominen T. Work motivation of nurses: A literature review. Int J Nurs Stud. 2011;48(2):246–57. https://doi.org/10.1016/j.ijnurstu.2010.09.013 .

Coles E, Anderson J, Maxwell M, Harris FM, Gray NM, Milner G, et al. The influence of contextual factors on healthcare quality improvement initiatives: a realist review. Syst Rev. 2020;9(1):94. https://doi.org/10.1186/s13643-020-01344-3 .

Lyng HB, Macrae C, Guise V, Haraldseid-Driftland C, Fagerdal B, Schibevaag L, et al. Exploring the nature of adaptive capacity for resilience in healthcare across different healthcare contexts; a metasynthesis of narratives. Appl Ergon. 2022;104:103810. https://doi.org/10.1016/j.apergo.2022.103810 .

Wiig S, O'Hara JK. Resilient and responsive healthcare services and systems: challenges and opportunities in a changing world. BMC Health Serv Res. 2021;21(1):1037. https://doi.org/10.1186/s12913-021-07087-8 .

Iflaifel M, Lim RH, Ryan K, Crowley C. Resilient Health Care: a systematic review of conceptualisations, study methods and factors that develop resilience. BMC Health Serv Res. 2020;20(1):324. https://doi.org/10.1186/s12913-020-05208-3 .

Berg SH, Akerjordet K, Ekstedt M, Aase K. Methodological strategies in resilient health care studies: An integrative review. Saf Sci. 2018;110:300–12. https://doi.org/10.1016/j.ssci.2018.08.025 .

Ellis LA, Churruca K, Clay-Williams R, Pomare C, Austin EE, Long JC, et al. Patterns of resilience: A scoping review and bibliometric analysis of resilient health care. Saf Sci. 2019;118:241–57. https://doi.org/10.1016/j.ssci.2019.04.044 .

Download references

Acknowledgements

We would like to thank Eline Ree and Adriana Rosenberg of the Centre for Resilience in Healthcare (SHARE) at the University of Stavanger for their contributions and feedback in this scoping review.

This research was funded by the Centre for Resilience in Healthcare (SHARE) at the University of Stavanger. The funder has played no role in the design and completion of the study.

Author information

Authors and affiliations.

Centre for Resilience in Healthcare SHARE, Faculty of Health Sciences, University of Stavanger, N-4036, Stavanger, Norway

Foteini Tsandila-Kalakou, Siri Wiig & Karina Aase

You can also search for this author in PubMed   Google Scholar

Contributions

FTK: study inception and study design, database searches and title screening. KA & FTK: abstract and full text screening, data extraction, data charting, thematic analysis, synthesis, and description of results. FTK, KA, SW agreed on study design, search strategy, data extraction, thematic analysis, synthesis, and description of the results. All three authors contributed, edited, and approved the final manuscript.

Corresponding author

Correspondence to Foteini Tsandila-Kalakou .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The author SW is a member of the editorial board (Associate Editor and Guest

Editor of this special collection). SW had no role in the peer review or in the handling of this manuscript. The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist.

Additional file 2.

Inclusion Exclusion criteria.

Additional file 3.

Table of all studies included in the review.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Tsandila-Kalakou, F., Wiig, S. & Aase, K. Factors contributing to healthcare professionals’ adaptive capacity with hospital standardization: a scoping review. BMC Health Serv Res 23 , 799 (2023). https://doi.org/10.1186/s12913-023-09698-9

Download citation

Received : 30 October 2022

Accepted : 13 June 2023

Published : 26 July 2023

DOI : https://doi.org/10.1186/s12913-023-09698-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Adaptive capacity
• Healthcare professionals
• Hospital standardization

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Open access
• Published: 19 September 2022

Understanding injecting drug use in Afghanistan: A scoping review

• Frishta Nafeh 1 ,
• Sufiat Fusigboye 1 &
• Bundit Sornpaisarn 2 , 3 , 4  

Substance Abuse Treatment, Prevention, and Policy volume  17 , Article number:  65 ( 2022 ) Cite this article

5176 Accesses

8 Citations

4 Altmetric

Metrics details

Several reports have described a growing prevalence of illicit drug use in Afghanistan, with recognition of a recent shift from traditional modes of consumption involving inhalation and oral ingestion to injecting drug use.

Conduct a comprehensive review of existing literature to map the injecting drug use situation in Afghanistan. The review intends to describe risk factors and impacts of injecting drug use, drug use characteristics and risk behaviours among people who inject drugs (PWID), and access to harm reduction and treatment.

We searched Embase, Global Health, Medline, PsycINFO, Web of Science, and grey literature to identify English language publications up to March 26 th , 2022. Studies were eligible for inclusion if they explicitly targeted PWID or injecting drug use in Afghanistan and provided information relevant to the review questions. Two reviewers independently screened titles and abstracts for inclusion and extracted information based on the review objectives.

A total of 25 articles were identified representing 15 studies (11 quantitative, 2 qualitative, 2 mixed methods). All but one studies were cross-sectional. In majority of the studies, over 95% of the participants were male and most were conducted over a decade ago, in urban settings, mainly Kabul. Only one study examined risk factors of injecting drug use. Eleven studies described drug use characteristics and 9 reported risk behaviours among PWID. Health and social burden of injecting drug use were reported by 8 and 5 studies, respectively. Nine studies described access to harm reduction and treatment. Afghan PWID had high levels of injecting and sexual risk behaviours compared to global estimates. They reported high prevalence of incarceration and displacement. Access to harm reduction and treatment was very limited. This scoping review revealed important knowledge gaps including a gender gap in research with serious implications for drug policy and substance use care.

Conclusions

Development of a national public health-oriented drug policy and substance use care programme is warranted along with efforts to develop health research capacity to address the need for epidemiological data. The current humanitarian crisis necessitates continued access to evidence-based harm reduction and treatment in Afghanistan.

Introduction

Since the mid 1990s, Afghanistan has come to be known as the world’s largest producer of opium with an estimated 85% share of the total global supply in 2020 [ 1 , 2 ]. Opium poppy cultivation has been steadily increasing in recent decades, with an average annual increase of 4,000 hectares since the start of systematic monitoring in 1994 [ 3 ]. Furthermore, preliminary evidence indicates emergence of a growing illicit methamphetamine market in Afghanistan, after manufacturing rose in 2016 [ 4 ]. For instance, data on drug seizures showed an increasing percentage of methamphetamine seized in neighbouring countries originated in Afghanistan between the periods 2014–2018 and 2019–2021 [ 5 , 6 ]. Given the burgeoning illicit drug economy in Afghanistan with opiates making up to 14% of the national gross domestic product (GDP) in 2021 [ 5 ], it is inevitable that drug use is a considerable public health problem in Afghanistan. The availability of cheap illicit drugs [ 5 ] coupled with rampant poverty [ 7 ] and decades of war – contributing to large-scale population displacement [ 8 ] and widespread psychological distress [ 9 , 10 ] – puts many Afghans at risk of problematic drug use and subsequent health and social consequences [ 11 , 12 ].

The extant literature – albeit limited – points to a rapidly growing drug use problem of potentially epidemic proportions. According to Afghanistan's first ever population-based drug use survey conducted in 2005, 3.8% of the population reported illicit drug use with the most common drugs being cannabis followed by opiates [ 13 ]. In 2009, a follow-up survey reported a drug use prevalence of 8% among adults 15–64 years of age, along with a 53% increase in the number of regular opium users and an increase of 140% in the number of regular heroin users since 2005 [ 14 ]. In 2015, a new survey (Afghanistan National Drug Use Survey) that used confirmed biological measures found a national drug use prevalence of 12.8% among those 15 years and older [ 15 ] compared to a global rate of 5.3% among adults of the same age in the same year [ 16 ]. According to Afghanistan National Drug Use Survey in 2015, opioids became the most used illicit drugs in Afghanistan with a prevalence of 8.5%, exceeding the opioids prevalence in North America (4.42%) [ 15 , 16 ].

Among illicit drugs, opioids are the most harmful, causing the highest burden of morbidity and mortality attributable to drug use disorders [ 17 ]. Between 2016–2019, over 70% of disability adjusted life years (DALYs) attributable to drug use disorders were due to opioids alone [ 2 , 17 ]. Opioids are the most injected drugs with a global prevalence of 83% among people who inject drugs (PWID) [ 18 ]. Route of drug administration has important implications for health with injection being the riskiest route. Compared to non-injection drug users, PWID have elevated risk of drug dependence [ 19 , 20 ], frequent overdoses [ 21 , 22 ], and all-cause mortality [ 23 ]. While inhalation and oral ingestion was historically the most common routes of drug administration in Afghanistan, emerging evidence suggests that displacement and migration to and return from neighbouring countries of Pakistan and Iran have contributed to a growing trend in injecting among drug users in Afghanistan [ 24 ].

The high prevalence of illicit opioid use in a nation with one of the world’s worst public health and socioeconomic indicators [ 25 , 26 ] and punitive laws often dictated by religion around substance use [ 27 , 28 ] may lead to devasting health and social outcomes for people who use drugs in general and PWID in particular. Afghanistan has high rates of communicable diseases and is endemic for malaria, viral hepatitis, and emerging concentrated human immunodeficiency virus (HIV) among drug users [ 29 , 30 ]. Health indicators are further challenged by a severe lack of social and health infrastructures due to decades of war and poverty [ 31 ]. Therefore, it is critical to understand the nature and extent of injecting drug use (IDU) in Afghanistan and identify priority areas for future research.

Several global literature reviews have described the epidemiology of IDU, some of which have provided numerical information for Afghanistan; however, these lack context and interpretation [ 32 , 33 , 34 , 35 , 36 , 37 ]. Therefore, the present review was undertaken to identify and map the existing literature on IDU in Afghanistan by adopting a scoping review design, which is more suitable for broad objectives that aim to identify, describe, and/or map the current body of evidence on a topic [ 38 , 39 ]. Consistent with the Joanna Briggs Institute (JBI) Manual for Evidence Synthesis , the Population, Concept, and Context Framework was used to guide the review questions [ 40 ]. The present scoping review intends to answer the following review questions:

What are the risk factors of IDU in Afghanistan?

What are the common drug use characteristics and risk behaviours among PWID in Afghanistan?

What is the health and social burden of IDU among PWID in Afghanistan (e.g., comorbidity, overdose, mortality, incarceration, stigma, etc.)?

What is the evidence on access to harm reduction and treatment service among PWID in Afghanistan?

This scoping review followed a framework proposed by the JBI [ 40 ], which builds on previous guidance developed by Arksey and O-Malley (2005) [ 39 ] and Levac et al. (2010) [ 41 ]. In addition to the JBI framework, the “Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for Scoping Reviews” (PRISMA-ScR) [ 42 ] was also used to guide the reporting of this review (see Supplementary Appendix B ). A scoping review protocol was developed a priori, to ensure the review methods were transparent and reproduceable. The protocol is available upon request from the corresponding author.

Literature review and data sources

A comprehensive search algorithm was developed and pretested to capture relevant literature. The final search algorithm included two components, one pertaining to “IDU” and the other pertaining to “Afghanistan”. See Supplementary Appendix A for additional information on the search strategies. Searches were executed on March 26 th , 2022, using five bibliographic databases: Embase, Global Health, Medline, PsycINFO, and Web of Science. To provide a more comprehensive and timely view of available evidence [ 43 , 44 ], grey literature was searched using a systematic approach consistent with technical guidelines developed by the National Drug and Alcohol Research Centre [ 45 ]. We also searched the reference lists of all the selected articles to identify additional relevant documents. See Fig.  1 for the PRISMA flow diagram.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Flow Diagram of Articles Selection Process. The bibliographic database searches yield 3375 unique articles. Based on title and abstract screening, 3084 articles were excluded and 291 underwent full-text screening. After full-text screening based on the inclusion criteria, 20 articles were included in the scoping review. Grey literature and citation searching of included studies yielded another 5 articles. PRISMA Flow diagram adapted from Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., et al. (2020). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ , 372, n71

Inclusion and exclusion criteria

The inclusion criteria consisted of studies or reports (written in English) with primary data generated by study authors or reliable government and/or research organizations. In addition, studies or reports were eligible for review if they explicitly targeted PWID or IDU in Afghanistan and provided information relevant to the review questions. Furthermore, conference proceedings were considered for inclusion if they provided information relevant to the review questions. There was no restriction on publication date due to the dearth of published literature on drug use in Afghanistan; we searched for all available articles until March 26 th , 2022. In terms of study design, we included both quantitative and qualitative studies. Exclusion criteria included unit of analysis not being individuals (PWID) or country (Afghanistan), aggregated information or data (e.g., regional estimates), study population being Afghans residing outside Afghanistan (refugees and immigrants), and study setting not being Afghanistan. In addition, publications about PWID that did not stem from primary research (e.g., reviews, commentary, opinions, or editorials) as well as study protocols were also excluded.

Data extraction

Publication details from the bibliographic databases were imported into the reference management software, Endnote desktop, where duplicates were removed manually. All articles underwent a two-step screening process, whereby titles and abstracts were initially screened for full-text review if they addressed the target population ‘people in Afghanistan’ and/or ‘IDU/PWID’. In the second step, the full text of selected titles and abstracts were examined and selected for review based on the inclusion and exclusion criteria. The screening process was carried out independently by FN and SF. Discrepancies in the screening process were discussed and resolved by the review team.

A data charting form (using Excel spreadsheet) was developed by FN a priori and pilot tested. The data charting form included identification information (author(s); publication date), methodological information (study year, location, design; sampling method; sample size), main findings, limitations as well as information pertaining to the review questions. Data charting was performed independently by FN and SF. Discrepancies in the data charting process was discussed and resolved by the review team.

Collating, summarizing, and reporting results

The results were organized as themes guided by the review questions. The thematic categories included risk factors of IDU, drug use characteristics and risk behaviours among PWID, health and social burden of IDU, and access to harm reduction and treatment services. Risk factors included any personal and/or structural determinants of IDU. Drug use characteristics included any description of drug use patterns (e.g., type of drugs used, age of drug use initiation, frequency and duration of drug use, place of drug consumption, etc.) and risk behaviours included any sexual and drug use related activity that put PWID at risk of health harms such as infectious disease transmission, overdose, and mortality. Health burden included any comorbidity and mortality among PWID, while social burden included incarceration, criminalization, financial issues, and experiences of stigma. Themes were presented based on how they emerged in the selected studies.

Characteristics of included studies

The flow diagram of eligible studies is shown in Fig.  1 . In total, 25 articles (20 peer-reviewed papers and 5 reports) met the inclusion criteria and covered 15 studies (see Table 1 ). The study dates ranged from 2005–2019, with majority (9 out of 15) conducted over a decade ago, prior to 2012 (Table 1 ). Eleven studies were quantitative, 2 were qualitative, and 2 were mixed-methods design (Table 1 ). All but one of the quantitative studies were cross-sectional with one observational cohort study (Table 1 ). One of the mixed-methods studies included in-country assessments involving key informant interviews, focus groups, and field observations, supplemented by desk reviews [ 46 ]; unit of analysis was country. The other mixed methods study utilized structured, interviewer-administered questionnaire and focus groups [ 47 ]. In most studies ( n  = 8; representing 17 publications), participants were convenience samples of PWID recruited through harm reduction services or by harm reduction and/or outreach workers known to them [ 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 ]. In terms of geographic representation, only one study included participants from rural areas [ 14 ]. Over half (8 out of 15) of the studies were conducted in the capital city of Afghanistan, Kabul alone (Table 1 ). Majority of the studies ( n  = 11) included male PWID participants only; in 2 studies, over 95% of the sample were male (Table 1 ). Only one study was exclusively about women who inject drugs [ 53 ]. Table 1 outlines key characteristics of the included studies.

Sociodemographic characteristics of PWID

A total of n  = 4622 PWID were assessed across 14 studies; in one study unit of analysis was country (Table 1 ). Over 95% of the sample were male. Most PWID were below the age of 35 and around half reported being married [ 49 , 54 , 58 , 59 , 60 , 61 , 67 , 68 , 69 ]. Low levels of education were common, which was typically below 6 years of formal education [ 49 , 54 , 60 , 61 , 68 , 69 ]. Five out of the 15 studies reported average monthly income which ranged from 3000–6900 Afghanis: equivalent to approximately US $60–138 around the study periods [ 49 , 54 , 60 , 68 , 69 ]. Having been refugee in the last 10 years was reported by 65%-92% of the participants in 6 studies [ 49 , 54 , 60 , 61 , 68 , 69 ]. Only two studies reported homelessness among participants; in one study with a convenience sample of 463 male participants, only 4 reported homelessness [ 52 ], while another study with a convenience sample of 483 reported 23% of participant as homeless [ 61 ]. In the qualitative study with women who inject drugs, mean age was 42 years, all participants were ever married and had children [ 53 ]. Ninety percent had never been to school and the majority identified themselves as economically disadvantaged [ 53 ].

Risk factors of IDU use (Q1)

Only one quantitative study using a cross-sectional design and convenience sample of 55 PWID examined the risk factors of IDU [ 66 ]. Joblessness (OR = 2.92; 95%CI: 1.20–7.11), starting drug use in another country (OR = 7.46;95%CI: 1.99–28.03), and previous history of incarceration (OR = 3.75;95%CI: 1.85–6.86) were significantly associated with IDU in multivariate binary logistic regression analysis [ 66 ]. According to two qualitative studies, participants reported physical effects (stronger kick) and cost as reasons for injecting drugs as they reported injecting to have the lowest daily cost compared to other routes of drug administration [ 53 , 59 ]. Male PWID also reported fear of arrest as a reason for switching to injecting as they perceived it to be less publicly noticeable than smoking [ 59 ].

Drug use characteristics and risk behaviours (Q2)

Eleven studies (10 quantitative and 2 qualitative) described the drug use characteristics of PWID [ 14 , 49 , 53 , 54 , 58 , 59 , 61 , 64 , 67 , 68 , 69 ]. Heroin was the most common drug used [ 14 , 51 , 54 , 61 , 67 , 68 , 69 ]. Some studies reported opium as the second most common drug used among PWID [ 14 , 67 , 68 ]. Polydrug use was common [ 49 , 54 , 61 , 68 , 69 ] with 20–77% of participants reporting mixing heroin with pheniramine maleate (Avil) [ 49 , 54 , 61 , 69 ]. On average, age at which PWID initiated injecting drugs was 26 years based on four studies [ 14 , 49 , 54 , 61 ]. In terms of place of drug consumption, male PWID reported public settings as common place of injecting and using drugs [ 51 , 69 ] while for women, drug use usually took place at home with either spouse, friends, or neighbours [ 53 ].

Risk behaviours among PWID was reported by nine studies and common risk behaviours included needle/syringe sharing, re-using needles/syringes, paying for sex with a female, men who have sex with men, low condom use and re-injecting blood [ 14 , 47 , 49 , 53 , 54 , 60 , 61 , 68 , 69 ]. The proportion of PWID that reported ever needle/syringe sharing ranged from 17% in a convenience sample of 483 IDU recruited from harm reduction sites and areas where drug users congregate in Kabul [ 61 ] to 87% in a population-based survey [ 14 ]. In a qualitative study of IDU with 10 women from Kabul, sharing needles/syringes was reporting to be common, especially with a spouse [ 53 ]. Furthermore, three studies reported that 70–83% of PWID drew and re-injected their own blood [ 51 , 54 , 61 ]. The proportion of male participants who reported ever patronizing a female sex worker ranged from 40 to 76% and the proportion of male PWID ever having sex with a male was between 11 to 28% [ 49 , 54 , 60 , 61 , 68 , 69 ]. Five studies reported condom use behaviour among PWID, of which four reported fewer than one-third of PWID ever using a condom [ 49 , 54 , 60 , 61 , 68 , 69 ].

Health and social burden of IDU (Q3)

Eight quantitative studies (6 cross-sectional, 1 observational cohort) examined the burden of infectious diseases among PWID with HIV and hepatitis C as the most common examined diseases [ 49 , 54 , 58 , 60 , 61 , 67 , 68 , 69 ]. Point prevalence of HIV in studies that used confirmed biological measures ranged from 1.8% to 7.1% [ 49 , 54 , 58 , 61 , 68 ]. One observational cohort study reported an incidence rate of 1.5 cases per 100 person-year and median time to infection of 18.3 months [ 62 ]. Furthermore, a large ( n  = 1378) cross-sectional study in 2019 reported an HIV prevalence of 20.7% based on self-report [ 67 ]. Among those who tested positive for HIV, co-infection with hepatitis C virus (HCV) was very common [ 54 , 58 , 61 ]. Point prevalence of HCV based on biological measures ranged from 36.0% to 60.2% [ 49 , 54 , 58 , 61 , 68 ]. An observational cohort study reported an incidence rate of 35.1 cases per 100 person-year and median time to HCV infection of 9.3 months [ 62 ]. Hepatitis B prevalence ranged from 4.6% to 7.1% [ 49 , 54 , 61 , 68 ], while syphilis prevalence ranged from 1.2% to 5.5% [ 60 , 61 , 68 ]. One study reported crude mortality rate among PWID (93.4/1000 person-year), 43.9% of which was due to overdose during an 18-month follow-up period [ 62 ]. Burden of mental health among PWID was only examined in one study with n  = 83 participants recruited from an OST pilot programme in Kabul [ 58 ]; participants reported a mean number of 6.5 mental health symptoms. The two qualitative studies reported perceived psychological distress, infections, various physical health symptoms, and overdose among PWID [ 53 , 59 ].

Only five studies examined the social burden of IDU among PWID, by reporting the burden of incarceration, criminalization, and perceived stigma. According to four quantitative studies, burden of incarceration among PWID ranged from 51.8% to 63.1% [ 49 , 54 , 58 , 61 ]. The qualitative study conducted with 10 women who inject drugs reported experiences of community and intimate partner violence, economic hardships as well as marginalization [ 53 ]. Criminalization and fear of it were reported to be common among male PWID based on two studies [ 58 , 59 ]. Furthermore, one qualitative study with male PWID reported perceived material deprivation and stigma at the family and community levels [ 59 ].

Access to harm reduction and treatment services (Q4)

Nine studies (6 quantitative, 2 qualitative, 1 mixed methods) reported on access to harm reduction and treatment services [ 14 , 46 , 52 , 53 , 58 , 61 , 67 , 68 , 69 ]. National coverage of needle and syringe programmes (NSP) and opioid substitution therapy (OST) in Afghanistan was reported to be low based on an assessment of a programme funded by The Global Fund [ 65 ]. According to this study, the programme only covered 28% of the PWID population and average number of sterile needles/syringes provided per PWID per year was 157 compared to the 300 needles/syringes per PWID per year standard recommended by the World Health Organization [ 65 ]. Furthermore, harm reduction services provided by the Afghanistan National Program for Control of AIDS, STI and Hepatitis , only reached 14% of the PWID population, provided OST for 2%, and 50 sterile needles/syringes and 5 condoms were provided per PWID in 2019 [ 67 ]. Access to addiction treatment was reported to be low [ 52 , 53 ] and difficulty accessing sterile needles/syringes was common [ 49 , 53 ]. Pharmacies were reported as the usual place for obtaining sterile needles [ 51 , 69 ] and in one study that examined service utilization among PWID, only 1 participant reported ever receiving syringes from a NSP [ 52 ]. Barriers to accessing harm reduction and treatment services included financial limitations, service capacity issues, fear of arrest, as well as stigma and discrimination [ 52 , 53 , 59 ].

Three studies reported HIV testing among PWID; the proportion of study participants reporting ever being tested ranged from high (82%) to low (22%) [ 14 , 53 , 67 ]. Knowledge about HIV was assessed by 4 studies, 3 of which reported low knowledge among PWID [ 50 , 68 , 69 ], while 1 study reported 60.2% of OST clients having correct HIV knowledge [ 58 ]. Only one study assessed OST programme retention rate with a follow-up period of 18 months [ 58 ]. The study reported a retention rate of 54.2% at 18 months, with reasons for lost to follow-up including imprisonment and migration among other reasons [ 58 ].

Results of this scoping review indicate that PWID in Afghanistan tend to be young (below the age of 35), married, and have low education. A high percentage of them reported being incarcerated, similar to trends among PWID in North America and elsewhere [ 18 ]. What is unique about PWID in Afghanistan is a large proportion reported previously being refugees in neighbouring countries of Iran and Pakistan, some of whom initiated drug use [ 14 ] and injecting behaviour in these countries [ 61 , 64 , 68 ]. In a study of Pakistani and Afghan refugee drug users, the authors found the latter group nearly three times less likely to have heard of HIV/AIDS and none of those ever sexually active reported ever using a condom compared to their Pakistani counterparts [ 70 ]. Thus, our findings have important implications for HIV and harm reduction programmes in countries hosting large numbers of vulnerable Afghan refugees. Our results demonstrated a sever lack of knowledge regarding risk factors of IDU in Afghanistan as only one study examined this with a small ( n  = 55) sample of male PWID. Previous incarceration was among one of the risk factors. This and a high prevalence of incarceration among PWID in Afghanistan suggests prisons may be an important setting for delivering harm reduction and addiction treatment services for frequently displaced and hard-to-reach PWID populations in Afghanistan.

Heroin first and opium second were the most common drugs consumed among PWID. Use of synthetic opioids and stimulants was relatively rare in comparison to global drug use trends [ 18 ]. Although this trend has remained constant according to a recent study of PWID in Afghanistan [ 67 ], there is need for ongoing research to monitor trends as synthetic drug markets have emerged in recent years in Afghanistan and in the neighbouring countries of Iran and Pakistan [ 4 , 6 ]. Furthermore, polydrug use appears to be common among PWID in Afghanistan further warranting a need for ongoing monitoring of drug use patterns to inform evidence-based harm reduction and treatment programmes.

Engagement in risky sexual behaviours such as patronizing female sex workers, sex with a male, and low condom use was very common among male PWID in Afghanistan. Although comparison with other settings is difficult due to varying measurements and definitions, PWID in Afghanistan appear to have higher prevalence of sexual risk as 67–83% [ 60 , 69 ] reported unprotected sex with a casual partner while prevalence of sexual risk was 36.7% in South Asia, 40.1% in the Middle East and North Africa, and 37.4% globally according to a 2017 systematic review [ 18 ]. Evidence of a network of high-risk groups (PWID, female sex workers, men who have sex with men) coupled with low condom use have important implications for widespread transmission of HIV, which is currently concentrated among drug users [ 30 ]. Therefore, it is important for future studies to assess risk behaviours among high-risk networks in Afghanistan. Ever sharing needles/syringes was very common among PWID in Afghanistan. Past 12-month prevalence of needle/syringe sharing was 87% among current injecting drug users according to a population-based survey [ 14 ], which is more than double the past 6–12-month global estimate of receptive needle/syringe sharing among PWID (32.8%) [ 37 ]. A nuanced risk behaviour reported among male PWID in Afghanistan was drawing and reinjecting one’s own blood [ 51 , 54 , 61 ]. Similar risk behaviours have been reported among PWID in Pakistan [ 71 ], Sub-Saharan Africa [ 72 ], and North America [ 73 , 74 ], which is shown to be a risk factor for HIV transmission [ 73 ] and soft-tissue abscess [ 74 ]. This merits further examination to understand the patterns and impacts of this behaviour in terms of disease transmission.

There is limited information regarding the health burden of IDU among PWID in Afghanistan. The most common diseases studied were HIV and hepatitis with prevalence based on biological measures ranging from 1.8–7.1% for HIV, 36.0–60.2% for HCV, and 4.6–7.1% for hepatitis B virus (HBV). In comparison, the global prevalence of HIV, HCV, and HBV according to a 2017 systematic review was 17.8%, 52.3% and 9.1%, respectively [ 18 ]. The estimates for Afghanistan likely underestimate the true extent of HIV and hepatitis among PWID as they were based on small convenience samples that did not include women and rural PWID. According to a large study conducted with 1378 PWID across 8 cities, self-reported prevalence of HIV was 27% [ 67 ], further suggesting underestimation of the true burden of infectious diseases in this population. Information about mortality rates and comorbidity among PWID is limited as psychological disorders and other communicable diseases of concern (e.g., malaria and tuberculosis) [ 29 ] are not assessed to date. Moreover, except one observational cohort study conducted a decade ago [ 62 ], studies on fatal and non-fatal overdoses among PWID are non-existent. Information on the social burden of IDU is also limited in Afghanistan, particularly with regards to important indicators such as homelessness, financial problems, and systemic discrimination.

Access to harm reduction and addiction treatment is extremely limited among PWID in Afghanistan. Similar access barriers as reported by other PWID worldwide included financial limitations, capacity issues, as well as stigma and discrimination [ 75 ]. Of important note is fear of police harassment as it was both reported as a reason why PWID chose injecting as route of drug administration and as a barrier to accessing harm reduction programmes [ 53 , 59 ]. This coupled with high incarceration rates among PWID indicate an opportunity to collaborate with law enforcement to expand provision of critical harm reduction programmes such as HIV testing and counselling, OST, NSP, and treatment referrals, which have been successfully implemented in the neighbour country, Iran [ 76 , 77 , 78 ] and have led to positive outcomes in different countries [ 79 ]. In addition to limited access to HIV testing and counselling, knowledge about HIV/AIDS is low among PWID in Afghanistan, which has important implications for HIV prevention and treatment programmes, particularly in the context of widespread needle/syringe sharing and sexual risk behaviours. While harm reduction programmes namely NSP, OST, and HIV testing and counselling have been implemented in Afghanistan and drug treatment programmes have been associated with positive outcomes (reduced drug use and criminal activity) [ 80 , 81 ], our findings indicate lack of evaluation of treatment and harm reduction programmes for PWID.

Moving forward with IDU research in Afghanistan

Illicit drug use patterns are highly gendered in Afghanistan, in terms of drug types, route of administration, and place of consumption [ 14 , 47 ]. Women’s drug use is hidden and often happens at home whereas men commonly use drugs in public places. This has led to a gender gap in research as it has reportedly been difficult to recruit women PWID in research. This warrants the need to conduct research with women who inject drugs to better understand gender-based risk factors and barriers to treatment and harm reduction services. Furthermore, majority of the studies did not include rural populations, which according to a population-based survey in 2015, had drug use rates nearly 3 times higher than the urban populations [ 15 ]. As such, the extant literature does not highlight the true extent, nature, and impacts of IDU in Afghanistan. Lack of data on IDU has serious implications for implementation of evidence-based prevention and treatment interventions. Inadequate information on determinants and impacts of IDU and limited knowledge about service needs and barriers to care among PWID contributes to difficulties in garnering support for resource allocation towards uptake and expansion of evidence-based solutions and improvement of existing programmes serving these populations.

Research on IDU has dramatically decreased during the last decade in Afghanistan (Table 1 ), despite a growing trend in illicit drug availability and use [ 5 , 13 , 14 , 15 ]. This warrants the need for continued research and surveillance of illicit drug use patterns and impact as well as evaluation of substance use care programmes. While conducting research in Afghanistan is difficult due to the complex environments as has been reported by studies included in this review [ 15 , 53 , 67 , 69 ], it is critical for generating necessary evidence to identify and address the health care needs of vulnerable populations such as PWID. Potential solutions to overcome research barriers – to some extent – include greater resource mobilization towards health research and local capacity building through greater international collaborations [ 82 , 83 ].

The present scoping review had some limitations. We restricted the search results to English language publications, which may have resulted in potential loss of relevant information. In addition, we missed reports that are not posted online as we did not seek additional information via personal communication with staff from relevant agencies. Nonetheless, there are several strengths to note. We developed a comprehensive and highly sensitive search strategy that was pretested. We searched several relevant grey literature databases following expert technical guidelines in a systematic fashion to allow replicability. Additionally, no restrictions were placed on publication date.

This scoping review identified and mapped the extant literature on IDU in Afghanistan and revealed several important knowledge gaps. Overall, Afghan PWID have high levels of injecting and sexual risk behaviours compared to global estimates. High incarceration rates, displacement, and limited availability of harm reduction and treatment resources puts this population at great risk of carrying the burden of infectious diseases and other harms. A national, public health-oriented drug policy and substance use care programme should be a key long-term development goal for Afghanistan to promote uptake and expansion of evidence-based harm reduction and addiction treatment strategies. Development goals for Afghanistan should also include resource allocation for health research and local capacity building in order to address the need for ongoing and scientifically rigorous research, necessary for guiding priority setting for drug policy and substance use care. Future research should also address the current gender gap in IDU research.

The current humanitarian crisis due to Taliban occupation has worsened pre-existing risk environments including widespread poverty, displacement, and fragile healthcare systems. Several news reports have described rising illicit drug use and a shift towards punitive drug prevention and treatment strategies consisting of imprisonment and forced, unmedicated detoxification. There is an urgent need for international aid agencies and their partners to prioritize provision of harm reduction and evidence-based addiction treatment in Afghanistan.

Availability of data and materials

Not applicable.

Abbreviations

Injecting Drug Use

People Who Inject Drugs

Joanna Briggs Institute

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews

Human Immunodeficiency Virus

Hepatitis C Virus

Hepatitis B Virus

Opioid Substitution Therapy

Needle and Syringe Programme

Ward C, Byrd W. Afghanistan’s opium drug economy. 2004. Available from:  https://documents1.worldbank.org/curated/en/158651468767124612/pdf/311490PAPER0AF100SASPR0no051Dec0171.pdf .

Google Scholar  

World Drug Report 2021 (United Nations publication, Sales No. E.21.XI.8). Available from: https://www.unodc.org/res/wdr2021/field/WDR21_Booklet_2.pdf

United Nations Office on Drugs and Crime (UNDOC). Afghanistan opium survey 2021: Cultivation and production. 2021. Available from:  https://www.unodc.org/documents/crop-monitoring/Afghanistan/Afghanistan_Opium_Survey_2021.pdf .

European Monitoring Centre for Drugs and Drug Addiction (EMCDDA). Emerging evidence of Afghanistan’s role as a producer and supplier of ephedrine and methamphetamine. 2020. Available from:  https://www.emcdda.europa.eu/system/files/publications/13410/emcdda-methamphetamine-in-Afghanistan-report.pdf .

United Nations Office on Drugs and Crime (UNDOC). Drug situation in Afghanistan 2021: Latest findings and emerging threats. 2021. Available from:  https://www.unodc.org/documents/data-and-analysis/Afghanistan/Afghanistan_brief_Nov_2021.pdf .

United Nations Office on Drugs and Crime (UNDOC). Afghanistan synthetic drugs situation assessment. 2017. Available from:  https://www.unodc.org/documents/scientific/Afghanistan_Synthetic_Drugs_Assessment-2017.pdf .

National Statistics and Information Authority. Afghanistan Multidimensional Poverty Index 2016–2017. NSIA, Kabul; 2019. Available from: https://www.mppn.org/wp-content/uploads/2019/03/AFG_2019_vs9_online.pdf

Crawford NC. Cost of war: Update on the human costs of war for Afghanistan and Pakistan, 2001 to mid-2016. 2016. Available from:  https://watson.brown.edu/costsofwar/files/cow/imce/papers/2015/War%20in%20Afghanistan%20and%20Pakistan%20UPDATE_FINAL.pdf .

Panter-Brick C, Goodman A, Tol W, Eggerman M. Mental health and childhood adversities: a longitudinal study in Kabul, Afghanistan. J Am Acad Child Adolesc Psychiatry. 2011;50(4):349–63.

Article   PubMed   PubMed Central   Google Scholar  

Scholte WF, Olff M, Ventevogel P, de Vries GJ, Jansveld E, Cardozo BL, Crawford CA. Mental health symptoms following war and repression in eastern Afghanistan. JAMA. 2004;292(5):585–93.

Article   CAS   PubMed   Google Scholar  

Rhodes T. The ‘risk environment’: a framework for understanding and reducing drug-related harm. Int J Drug Policy. 2002;13(2):85–94.

Article   Google Scholar  

Collins AB, Boyd J, Cooper HL, McNeil R. The intersectional risk environment of people who use drugs. Soc Sci Med. 2019;234:112384.

United Nations Office on Drugs and Crime (UNDOC). Afghanistan drug use survey 2005. 2005. Available from:  https://www.unodc.org/pdf/afg/2005AfghanistanDrugUseSurvey.pdf .

United Nations Office on Drugs and Crime (UNDOC). Afghanistan drug use survey 2009. 2009. Available from:  https://www.unodc.org/documents/data-and-analysis/statistics/Drugs/Drug%20use/Drug_use_in_Afghanistan_2009_survey.pdf .

SGI-Global. Afghanistan National Drug Use Survey 2015. 2015. Available from:  https://colombo-plan.org/wp-content/uploads/2020/03/Afghanistan-National-Drug-Use-Survey-2015-compressed.pdf .

United Nations Office on Drugs and Crime, World Drug Report 2017 (ISBN: 978–92–1–148291–1, eISBN: 978–92–1–060623–3, United Nations publication, Sales No. E.17.XI.6). Available from: https://www.unodc.org/wdr2017/field/Booklet_2_HEALTH.pdf

Degenhardt L, Charlson F, Ferrari A, Santomauro D, Erskine H, Mantilla-Herrara A, Vos T. The global burden of disease attributable to alcohol and drug use in 195 countries and territories, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Psychiatry. 2018;5(12):987–1012.

Degenhardt L, Peacock A, Colledge S, Leung J, Grebely J, Vickerman P, Larney S. Global prevalence of injecting drug use and sociodemographic characteristics and prevalence of HIV, HBV, and HCV in people who inject drugs: a multistage systematic review. Lancet Glob Health. 2017;5(12):e1192–207.

Novak SP, Kral AH. Comparing injection and non-injection routes of administration for heroin, methamphetamine, and cocaine users in the United States. J Addict Dis. 2011;30(3):248–57.

Strang J, Bearn J, Farrell M, Finch E, Gossop M, Griffiths P, Wolff K. Route of drug use and its implications for drug effect, risk of dependence and health consequences. Drug Alcohol Rev. 1998;17(2):197–211.

Darke S, Hall W. Heroin overdose: research and evidence-based intervention. J Urban Health. 2003;80(2):189–200.

Shrestha S, Stopka TJ, Hughto JM, Case P, Palacios WR, Reilly B, Green TC. Prevalence and correlates of non-fatal overdose among people who use drugs: findings from rapid assessments in Massachusetts, 2017–2019. Harm Reduct J. 2021;18(1):1–12.

Onyeka IN, Basnet S, Beynon CM, Tiihonen J, Föhr J, Kauhanen J. Association between routes of drug administration and all-cause mortality among drug users. J Substance Use. 2016;21(6):559–65.

Griffin N, Khoshnood K. Opium trade, insurgency, and HIV/AIDS in Afghanistan: relationships and regional consequences. Asia Pacific J Public Health. 2010;22(3_suppl):159S-167S.

World Health Organization. The global health observatory: Afghanistan. 2019. Available from:  https://www.who.int/data/gho/data/countries/country-details/GHO/afghanistan?countryProfileId=5f0ca8ad-037c-4241-a792-b3f06bddd40e .

The World Bank Group in Afghanistan. Country update. 2021. Available from:  https://documents1.worldbank.org/curated/en/451181617171930719/pdf/The-World-Bank-Group-in-Afghanistan-Country-Update.pdf .

Xinwei L. The drug situation and the practice of drug control in Afghanistan. Himalayan Central Asian Stud. 2019;23(3/4):194-0_4.

Todd CS, Nassiramanesh B, Stanekzai MR, Kamarulzaman A. Emerging HIV epidemics in Muslim countries: assessment of different cultural responses to harm reduction and implications for HIV control. Curr HIV/AIDS Rep. 2007;4(4):151–7.

Article   PubMed   Google Scholar  

Wallace MR, Hale BR, Utz GC, Olson PE, Earhart KC, Thornton SA, Hyams KC. Endemic infectious diseases of Afghanistan. Clin Infect Dis. 2002;34(Supplement_5):S171–207.

Mumtaz GR, Riedner G, Abu-Raddad LJ. The emerging face of the HIV epidemic in the Middle East and North Africa. Curr Opin HIV AIDS. 2014;9(2):183–91.

Gutlove P, Thompson G, Russell JH. Health, Human Security, and Social Reconstruction in Afghanistan. In: Beyond Reconstruction in Afghanistan. New York: Palgrave Macmillan; 2004. p. 188–209.

Chapter   Google Scholar  

Grebely J, Larney S, Peacock A, Colledge S, Leung J, Hickman M, Degenhardt L. Global, regional, and country-level estimates of hepatitis C infection among people who have recently injected drugs. Addiction. 2019;114(1):150–66.

Rashti R, Sharafi H, Alavian SM, Moradi Y, Mohamadi Bolbanabad A, Moradi G. Systematic Review and Meta-Analysis of Global Prevalence of HBsAg and HIV and HCV Antibodies among People Who Inject Drugs and Female Sex Workers. Pathogens (Basel, Switzerland). 2020;9(6):432.

CAS   Google Scholar  

Larney S, Leung J, Grebely J, Hickman M, Vickerman P, Peacock A, Degenhardt L. Global systematic review and ecological analysis of HIV in people who inject drugs: National population sizes and factors associated with HIV prevalence. Int J Drug Policy. 2020;77:102656.

Arum C, Fraser H, Artenie AA, Bivegete S, Trickey A, Alary M, Strathdee SA. Homelessness, unstable housing, and risk of HIV and hepatitis C virus acquisition among people who inject drugs: a systematic review and meta-analysis. Lancet Public Health. 2021;6(5):e309–23.

Mathers BM, Degenhardt L, Ali H, Wiessing L, Hickman M, Mattick RP, Strathdee SA. HIV prevention, treatment, and care services for people who inject drugs: a systematic review of global, regional, and national coverage. Lancet. 2010;375(9719):1014–28.

Tran LT, Peacock A, Colledge S, Memedovic S, Grebely J, Leung J, Larney S, Trickey A, Stone J, Vickerman P, Hickman M, Degenhardt L. Injecting risk behaviours amongst people who inject drugs: A global multi-stage systematic review and meta-analysis. Int J Drug Policy. 2020;84:102866.

Munn Z, Peters MD, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):1–7.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Peters MD, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, Khalil H. Updated methodological guidance for the conduct of scoping reviews. JBI Evidence Synthesis. 2020;18(10):2119–26.

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5(1):1–9.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, Straus SE. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Paez A. Gray literature: An important resource in systematic reviews. J Evid Based Med. 2017;10(3):233–40.

Mahood Q, Van Eerd D, Irvin E. Searching for grey literature for systematic reviews: challenges and benefits. Res Synthesis Methods. 2014;5(3):221–34.

Degenhardt L, Gibson G, Leung J, Kumvaj M, Lareney S. Searching the grey literature to access research on illicit drug use, HIV, and viral hepatitis. 2016. https://ndarc.med.unsw.edu.au/sites/default/files/ndarc/resources/Technical%20report%20number%20334.pdf .

Burrows D, McCallum L, Parsons D Falkenberry H. Global Summary of Findings of an Assessment of HIV Service Packages for Key Populations in Six Regions. APMG Health, Washington, DC; 2019. Available from: https://www.theglobalfund.org/media/9753/core_hivservicesforkeypopulationssixregions_review_en.pdf

UNODC, Impacts of Drug Use on Users and Their Families in Afghanistan, 2014. Available from: https://www.unodc.org/documents/data-and-analysis/Studies/Impacts_Study_2014_web.pdf

Bautista CT, Todd CS, Abed AM, Botros BA, Strathdee SA, Earhart KC, Scott PT. Effects of duration of injection drug use and age at first injection on HCV among IDU in Kabul. Afghanistan J Public Health. 2010;32(3):336–41.

Todd CS, Abed AM, Strathdee SA, Scott PT, Botros BA, Safi N, Earhart KC. HIV, hepatitis C, and hepatitis B infections and associated risk behavior in injection drug users, Kabul. Afghanistan Emerg Infect Dis. 2007;13(9):1327.

Todd CS, Abed A, Strathdee SA, Scott PT, Botros BA, Safi N, Earhart KC. Association between expatriation and HIV awareness and knowledge among injecting drug users in Kabul, Afghanistan: A cross-sectional comparison of former refugees to those remaining during conflict. Confl Heal. 2007;1(1):1–8.

Todd CS, Abed AM, Scott PT, Botros BA, Safi N, Earhart KC, Strathdee SA. Correlates of receptive and distributive needle sharing among injection drug users in Kabul, Afghanistan. Am J Drug Alcohol Abuse. 2008;34(1):91–100.

Todd CS, Abed AM, Scott PT, Safi N, Earhart KC, Strathdee SA. A cross-sectional assessment of utilization of addiction treatment among injection drug users in Kabul. Afghanistan Subst Use Misuse. 2009;44(3):416–30.

Middle East and North Africa Harm Reduction Association (MENAHRA). Women injecting drug users in the Middle East and North Africa Region (MENA). 2013. Available from:  http://fileserver.idpc.net/library/MENAHRA_Women_Injecting_Drug_Users_2013_.pdf .

Nasir A, Todd CS, Stanekzai MR, Bautista CT, Botros BA, Scott PT, Tjaden J. Prevalence of HIV, hepatitis B and hepatitis C and associated risk behaviours amongst injecting drug users in three Afghan cities. Int J Drug Policy. 2011;22(2):145–52.

Nasir A, Todd CS, Stanekzai MR, Bautista CT, Botros BA, Scott PT, Tjaden J. Implications of hepatitis C viremia vs, antibody alone on transmission among male injecting drug users in three Afghan cities. Int J Infect Dis. 2011;15(3):e201–5.

Sanders-Buell E, Bose M, Nasir A, Todd CS, Stanekzai MR, Tovanabutra S, McCutchan FE. Distinct circulating recombinant HIV-1 strains among injecting drug users and sex workers in Afghanistan. AIDS Res Hum Retroviruses. 2010;26(5):605–8.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Sanders-Buell E, Rutvisuttinunt W, Todd CS, Nasir A, Bradfield A, Lei E, Tovanabutra S. Hepatitis C genotype distribution and homology among geographically disparate injecting drug users in Afghanistan. J Med Virol. 2013;85(7):1170–9.

Ruiseñor-Escudero H, Vu A, Wirtz AL, Familiar-Lopez I, Berry M, Mfochive I, Burnham G. Cross-sectional assessments of participants’ characteristics and loss to follow-up in the first Opioid Substitution Therapy Pilot Program in Kabul. Afghanistan Harm Reduc J. 2015;12(1):1–6.

Todd CS, Stibich MA, Stanekzai MR, Rasuli MZ, Bayan S, Wardak SR, Strathdee SA. A qualitative assessment of injection drug use and harm reduction programmes in Kabul, Afghanistan: 2006–2007. Int J Drug Policy. 2009;20(2):111–20.

Todd CS, Nasir A, Stanekzai MR, Abed AM, Strathdee SA, Bautista CT, Tjaden J. Prevalence and correlates of syphilis and condom use among male injection drug users in four Afghan cities. Sex Transm Dis. 2010;37(11):719.

Todd CS, Nasir A, Stanekzai MR, Fiekert K, Rasuli MZ, Vlahov D, Strathdee SA. Prevalence and correlates of HIV, syphilis, and hepatitis B and C infection and harm reduction program use among male injecting drug users in Kabul, Afghanistan: A cross-sectional assessment. Harm Reduct J. 2011;8(1):1–8.

Todd CS, Nasir A, Stanekzai MR, Fiekert K, Sipsma HL, Vlahov D, Strathdee SA. Hepatitis C and HIV incidence and harm reduction program use in a conflict setting: an observational cohort of injecting drug users in Kabul. Afghanistan Harm Reduc J. 2015;12(1):1–9.

Todd CS, Nasir A, Stanekzai MR, Fiekert K, Sipsma HL, Strathdee SA, Vlahov D. Impact of conflict and displacement on risk behaviours amongst people who inject drugs in Kabul, Afghanistan. Int J Drug Policy. 2016;27:173.

Vogel M, Tschakarjan S, Maguet O, Vandecasteele O, Kinkel T, Dˇrsteler-MacFarland K. Mental health among opiate users in Kabul: a pilot study from the Medecins du Monde Harm Reduction Programme. Intervention. 2012;10(2):146–55.

Burrows D, Falkenberry H, McCallum L, Parsons D, Ngoksin E, Zhao J, Kunii O. Design, implementation, and monitoring of HIV service packages for people who inject drugs: An assessment of programs supported by the Global Fund in 46 countries. Int J Drug Policy. 2021;88:103036.

Rasekh H, Naimi HM, Mousavi SH. Prevalence and risk factors of hepatitis B, hepatitis C and HIV viruses among people who use drugs (PWUD) in Kabul, health care facilities. Hepat Mon. 2019;19(7):e84298.

Rasheed A, Sharifi H, Wesson P, Pashtoon SJ, Tavakoli F, Ghalekhani N, Mirzazadeh A. Mapping and population size estimates of people who inject drugs in Afghanistan in 2019: Synthesis of multiple methods. PLoS ONE. 2022;17(1):e0262405.

Ruiseñor-Escudero H, Wirtz AL, Berry M, Mfochive-Njindan I, Paikan F, Yousufi HA, Yadav RS, Burnham G, Vu A. Risky behavior and correlates of HIV and Hepatitis C virus infection among people who inject drugs in three cities in Afghanistan. Drug Alcohol Depend. 2014;143:127–33.

World Bank. 2008. Mapping and Situation Assessment of Key Populations at High Risk of HIV in Three Cities of Afghanistan. South Asia human development sector series;no. 23. Washington, DC. © World Bank. https://openknowledge.worldbank.org/handle/10986/17938 License: CC BY 3.0 IGO. Available from: https://openknowledge.worldbank.org/bitstream/handle/10986/17938/437330NWP0Box31gAfghanistanApril708.pdf?sequence=1&isAllowed=y

Zafar T, Brahmbhatt H, Imam G, Hassan SU, Strathdee SA. HIV knowledge and risk behaviors among Pakistani and Afghani drug users in Quetta, Pakistan. JAIDS J Acquir Immune Defic Syndr. 2003;32(4):394–8.

Kuo I, Galai N, Thomas DL, Zafar T, Ahmed MA, Strathdee SA. High HCV seroprevalence and HIV drug use risk behaviors among injection drug users in Pakistan. Harm Reduct J. 2006;3(1):1–10.

Asher AK, Hahn JA, Couture MC, Maher K, Page K. People who inject drugs, HIV risk, and HIV testing uptake in sub-Saharan Africa. J Assoc Nurses AIDS Care JANAC. 2013;24(6):e35–44.

Bruneau J, Daniel M, Abrahamowicz M, Zang G, Lamothe F, Vincelette J. Trends in human immunodeficiency virus incidence and risk behavior among injection drug users in Montreal, Canada: a 16-year longitudinal study. Am J Epidemiol. 2011;173(9):1049–58.

Murphy EL, DeVita D, Liu H, Vittinghoff E, Leung P, Ciccarone DH, Edlin BR. Risk factors for skin and soft-tissue abscesses among injection drug users: a case-control study. Clin Infect Dis. 2001;33(1):35–40.

Harm Reduction International. The global state of harm reduction: Towards an integrated response. 2012. Available from:  https://www.hri.global/files/2012/07/24/GlobalState2012_Web.pdf .

Ohiri K, Claeson M, Rassaghi E, Nassirimanesh B, Afshar P, Power R. HIV/AIDS prevention among injection drug users: Learning from harm reduction in Iran. Iran: HIV Prevention Consultation; 2006. p. 17–20.

Saberi Zafarghandi MB, Jadidi M, Khalili N. Iran’s Activities on Prevention, Treatment and Harm Reduction of Drug Abuse. Int J High Risk Behav Addict. 2015;4(4):e22863.

Shahbazi M, Farnia M, Rahmani K, Moradi G. Trend of HIV/AIDS Prevalence and Related Interventions Administered in Prisons of Iran -13 Years’ Experience. Iran J Public Health. 2014;43(4):471–9.

PubMed   PubMed Central   Google Scholar  

Kamarulzaman A, Reid SE, Schwitters A, Wiessing L, El-Bassel N, Dolan K, Moazen B, Wirtz AL, Verster A, Altice FL. Prevention of transmission of HIV, hepatitis B virus, hepatitis C virus, and tuberculosis in prisoners. Lancet (London Engl). 2016;388(10049):1115–26.

Courser M, Johnson K, Abadi MH, Shamblen SR, Young L, Thompson K, Browne T. Building an evidence base for drug abuse treatment in Afghanistan: Lessons learned and implications for future research. Int J Prev Treatment Subs Use Dis. 2013;1(1):12–27.

Shamblen SR, Courser M, Young L, Schweinhart A, Shepherd C, Morales B, Redpath B. The Efficacy of Drug Treatment in Afghanistan: Overall Results and Implications From a New Evaluation. Int J Mental Health Addiction. 2022;20:541–52. https://doi.org/10.1007/s11469-020-00382-1 .

El Achi N, Papamichail A, Rizk A, Lindsay H, Menassa M, Abdul-Khalek RA, Ekzayez A, Dewachi O, Patel P. A conceptual framework for capacity strengthening of health research in conflict: the case of the Middle East and North Africa region. Glob Health. 2019;15(1):81.

Mansour R, Naal H, Kishawi T, Achi NE, Hneiny L, Saleh S. Health research capacity building of health workers in fragile and conflict-affected settings: a scoping review of challenges, strengths, and recommendations. Health Res Policy Syst. 2021;19(1):84.

Download references

Acknowledgements

This study did not receive any funding.

BS acknowledges the funding support from the Canadian Institutes of Health Research’s Institute of Neurosciences, Mental Health and Addiction (Canadian Research Initiative on Substance Misuse Ontario Node Grant SMN-13950).

Author information

Authors and affiliations.

Department of Public Health Sciences, Queen’s University, 62 Fifth Field Company Lane, Carruthers Hall, Kingston, Ontario, K7L 3N6, Canada

Frishta Nafeh & Sufiat Fusigboye

Centre for Addiction and Mental Health, Institute for Mental Health Policy Research, Toronto, ON, Canada

Bundit Sornpaisarn

Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada

Faculty of Public Health, Mahidol University, Bangkok, Thailand

You can also search for this author in PubMed   Google Scholar

Contributions

FN conceptualized and designed the scoping review with input from SF. FN executed the search strategies and compiled the Endnote file. FN and SF screened titles and abstracts, performed full-text assessments, and carried out data extraction. FN wrote the first draft of the manuscript, which was critically revised by SF and BN. All authors approved the final manuscript.

Corresponding author

Correspondence to Frishta Nafeh .

Ethics declarations

Ethics approval and consent to participant, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Understanding injecting drug use in Afghanistan: A scoping review.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Nafeh, F., Fusigboye, S. & Sornpaisarn, B. Understanding injecting drug use in Afghanistan: A scoping review. Subst Abuse Treat Prev Policy 17 , 65 (2022). https://doi.org/10.1186/s13011-022-00491-1

Download citation

Accepted : 21 August 2022

Published : 19 September 2022

DOI : https://doi.org/10.1186/s13011-022-00491-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• People who inject drugs
• Injecting drug use
• Harm reduction
• Global health
• Afghanistan

Substance Abuse Treatment, Prevention, and Policy

ISSN: 1747-597X

• General enquiries: [email protected]

Evidence Synthesis: Finding, appraising and synthesizing research

• Getting Started
• Steps in Evidence Synthesis
• Methodology and Documentation
• Searching the published literature
• Searching the Gray Literature
• Managing the process

The gray literature: What is it and how do I search it?

Why search the gray literature if it hasn't been peer-reviewed?   There is a strong bias in scientific publishing toward publishing studies that show some sort of significant effect.  Meanwhile, many studies and trials that show no effect end up going unpublished.  But knowing that an intervention had no effect is just as important as knowing that it did have an effect when it comes to making decisions for practice and policy-making.  Thus, the grey literature can be critical.

How do I search the gray literature if it's unpublished and not in the scholarly databases?  Its true--finding gray literature and searching it systematically is challenging.  But there are a few approaches that you can take to add some structure to your search of this type of information:

• Identify and record the sources you will search, including some indication of search terms used if appropriate.  The sources you search will be informed by your research question and where you expect to find information related to your question.
• Try searching databases that specialize in gray literature like OpenMD , MedNar and BASE . OpenGrey , a previously available resource for gray literature, is also searchable from a locally maintained SQL database.  For more resources, see the box below.
• Conference proceedings :  Identify professional organizations that have conferences at which researchers might be presenting work related to your topic.  Search those conference proceedings on the organization's website or by contacting organizational boards for access to past proceedings that may not be online.
• Theses and dissertations :  There are a number of databases dedicated to theses and dissertations, which you can search using your search terms.  See the box below for links to these resources. 
• Identify government agencies, and international and non-governmental organizations , that might publish technical papers and reports on your topic.  Search their websites or any online libraries that they may provide.  For example, the WHO has a number of searchable online collections and the World Bank now makes all of their publications openly available online.
• Contact known researchers in the field to determine if there are any ongoing or unpublished studies that s/he may be aware of.

Image credit:  Stack 'em high, by Antony Theobald via Flickr.com, https://flic.kr/p/6WW7E2, CC BY-NC-ND 2.0

Guidance documents for gray literature searching

• Grey Matters: A practical search tool for evidence-based medicine This online manual provides a thorough list of sources for gray literature in medicine and a helpful checklist to help systematize your process.
• Googling the Greys: Tips for Searching Beyond Health Databases and Turning Information into Insights This presentation by Sarah Bonato, a health science librarian at the Centre for Addiction and Mental Health in Toronto, ON, provides an excellent guide to searching Google effectively to find gray literature.
• Duke University Medical Center Guide to Resource for Searching the Gray Literature A more thorough guide to gray literature, including resources for trial registries, pharmacological studies, conference abstracts, government document and more.

Databases of grey literature

• BASE BASE is one of the world's most voluminous search engines especially for academic web resources. BASE provides more than 240 million documents from more than 8,000 content providers. You can access the full texts of about 60% of the indexed documents for free (Open Access). BASE is operated by Bielefeld University Library.
• MedNar Mednar is a free, medically-focused deep web search engine that uses Explorit Everywhere!, an advanced search technology by Deep Web Technologies. As an alternative to Google, Mednar accelerates your research with a search of authoritative public and deep web resources, returning the most relevant results to one easily navigable page.
• NY Academy of Medicine Grey Literature Report This report was a bimonthly publication of The New York Academy of Medicine (NYAM) alerting readers to new grey literature publications in health services research and selected public health topics. The report is no longer being maintained but some of the resources are still accessible.
• OpenMD OpenMD.com is a search engine that makes high-quality medical information easily accessible to everyone. OpenMD searches billions of documents from government agencies, global health organizations, medical journals, and reference sites.

Sources for searching theses and dissertations

• ProQuest Dissertations and Theses With more than 2 million entries, PQD&T is the single, central, authoritative resource for information about doctoral dissertations and master's theses.

Resources for Clinical Trials

• Australia New Zealand Clinical Trials Registry Online register of clinical trials being undertaken in Australia, New Zealand and elsewhere. The ANZCTR includes trials from the full spectrum of therapeutic areas of pharmaceuticals, surgical procedures, preventive measures, lifestyle, devices, treatment and rehabilitation strategies and complementary therapies.
• ClinicalTrials.gov The US registry for clinical trials. Includes new, ongoing and completed human clinical trials both in the US and in countries around the world.
• Cochrane CENTRAL Register A highly concentrated source of reports of randomised and quasi-randomised controlled trials. The majority of CENTRAL records are taken from bibliographic databases (mainly MEDLINE and Embase), but records are also derived from other published and unpublished sources.
• EU Clinical Trials Register The European Union Clinical Trials Register allows you to search for protocol and results information on interventional clinical trials that are conducted in the European Union (EU) and the European Economic Area (EEA) and clinical trials conducted outside the EU / EEA that are linked to European paediatric-medicine development.
• WHO International Clinical Trials Registry Platform (ICTRP) This Clinical Trials Search Portal provides access to a central database containing the trial registration data sets provided by numerous international registries. It also provides links to the full original records.
• Wikipedia clinical trial registries list This Wikipedia entry contains a list of links to individual clinical trial registries by country.
• << Previous: Searching the published literature
• Next: Managing the process >>
• Last Updated: Feb 26, 2024 3:48 PM
• URL: https://guides.library.cmu.edu/systematic_reviews