quality improvement project dissertation

Advertisement

Evidence-Based Quality Improvement: a Scoping Review of the Literature

• Open access
• Published: 29 September 2022
• Volume 37 , pages 4257–4267, ( 2022 )

Cite this article

You have full access to this open access article

• Susanne Hempel PhD 1 , 2 , 6 ,
• Maria Bolshakova BS 1 ,
• Barbara J. Turner MD 2 ,
• Jennifer Dinalo PhD 5 ,
• Danielle Rose PhD 3 ,
• Aneesa Motala BA 1 , 2 , 6 ,
• Ning Fu PhD 1 , 4 ,
• Chase G. Clemesha MD 5 ,
• Lisa Rubenstein MD 6 &
• Susan Stockdale PhD 3  

4858 Accesses

11 Citations

1 Altmetric

Explore all metrics

Quality improvement (QI) initiatives often reflect approaches based on anecdotal evidence, but it is unclear how initiatives can best incorporate scientific literature and methods into the QI process. Review of studies of QI initiatives that aim to systematically incorporate evidence review (termed evidence-based quality improvement (EBQI)) may provide a basis for further methodological development.

In this scoping review (registration: https://osf.io/hr5bj ) of EBQI, we searched the databases PubMed, CINAHL, and SCOPUS. The review addressed three central questions: How is EBQI defined? How is evidence used to inform evidence-informed QI initiatives? What is the effectiveness of EBQI?

We identified 211 publications meeting inclusion criteria. In total, 170 publications explicitly used the term “EBQI.” Published definitions emphasized relying on evidence throughout the QI process. We reviewed a subset of 67 evaluations of QI initiatives in primary care, including both studies that used the term “EBQI” with those that described an evidence-based initiative without using EBQI terminology. The most frequently reported EBQI components included use of evidence to identify previously tested effective QI interventions; engaging stakeholders; iterative intervention development; partnering with frontline clinicians; and data-driven evaluation of the QI intervention. Effectiveness estimates were positive but varied in size in ten studies that provided data on patient health outcomes.

Conclusions

EBQI is a promising strategy for integrating relevant prior scientific findings and methods systematically in the QI process, from the initial developmental phase of the IQ initiative through to its evaluation. Future QI researchers and practitioners can use these findings as the basis for further development of QI initiatives.

Similar content being viewed by others

Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach

The ABC of systematic literature review: the basic methodological guidance for beginners

A 24-step guide on how to design, conduct, and successfully publish a systematic review and meta-analysis in medical research

Avoid common mistakes on your manuscript.

Evidence-based quality improvement (EBQI) is one of a growing number of strategies used to enhance quality improvement (QI) initiative impacts in clinical practice. EBQI aims to integrate scientific evidence and methods into the QI process while maintaining focus on team-based innovation and problem-solving within real-world settings. Standard healthcare QI approaches focus powerfully on the need for measurement to determine innovation effects, and teams are advised to consult subject matter experts to strengthen their work. 1 , 2 , 3 , 4 , 5 There currently is no standard approach, however, for integrating evidence from relevant pre-existing scientific literature into QI innovation and evaluation. Comprehensive review and critical appraisal of relevant research, for example, is not typically emphasized or conducted. 6 In practice, QI teams often use anecdotal evidence alone to shape innovations, and low-validity methods to evaluate them. 7 EBQI initiatives, as a subset of all QI initiatives, aim to systematically incorporate pre-existing scientific evidence and methods into the QI process as a core focus.

Given its foundation in applying best evidence and distinct focus on collaboration with the practice, EBQI is increasingly recognized as a valuable approach to structure implementation of advances in healthcare delivery. 7 Among other factors, the rapid evolution of partnership improvement initiatives between healthcare organizations and researchers, and the increasing availability of embedded healthcare researchers within healthcare organizations have made EBQI more accessible and attractive to healthcare organizations. 3 , 4 , 5

To date, core elements of EBQI have not been well documented, leaving a critical knowledge gap about components of EBQI and how it differs from other QI approaches. In addition, evidence of the effects of employing EBQI has yet to be synthesized. We found no prior systematic reviews of EBQI, and while individual studies have shown promising results 8 to our knowledge, EBQI has not been evaluated in an evidence synthesis across studies.

This scoping review explores the EBQI literature. We document how EBQI is defined in publications and aimed to identify key components that characterize this methodology across studies. The review catalogues definitions and characteristics of EBQI as currently used in practice. Particular emphasis was on the definition, scope, and use of evidence, i.e., the core aspect of EBQI. We also examined evidence of effectiveness of EBQI. The scoping review was guided by these review questions 10 :

Review question 1: How is EBQI defined?

Review question 2: How is evidence used to inform evidence-informed quality improvement initiatives?

Review question 2a: How is evidence defined in these initiatives?

Review question 2b: What are the components of EBQI?

Review question 3: What is the effectiveness of EBQI to promote uptake of evidence-based practices?

Our objective was to conduct a systematic search to identify the available knowledge, provide a clear description of the methodology, and inform further development of methods for incorporating research evidence into QI initiatives.

The scoping review followed a detailed review protocol. We followed the steps outlined by Arksey and Malloy: (1) determining the research question; (2) identifying relevant studies; (3) selecting studies; (4) charting the data; and (5) collating, summarizing, and reporting the results. 9 In addition, we conducted a consultation exercise to inform and validate findings. The project was deemed exempt by our institutional Human Subject Committee. The protocol was registered in the Open Science Framework and is publicly available. 10 The reporting follows PRISMA-ScR, a PRISMA adaptation for scoping reviews. 11 , 12

Search Strategy

The literature searches are documented in the supplemental digital content ( SDC ). First, a search using the exact terms (“evidence based quality improvement,” “evidence-based quality improvement,” or “EBQI”) was employed to identify publications published to March 2020 that explicitly refer to EBQI in the title, abstract, or keyword of the publication (i.e., the elements that are searchable in research databases). All retrieved publications that used the terminology were included.

Second, we used a broader search strategy aimed at identifying QI initiative evaluations that were not explicitly labeled as EBQI. We assumed that some authors may not use the term “EBQI” even when they have used an evidence-based QI strategy and describe a similar approach in the full-text publication. We applied a string of exclusion criteria to arrive at a manageable sample (see eligibility section), and given the large literature on QI interventions, 19 we searched only for studies published between 2017 and 2020.

We searched PubMed (biomedical literature), CINAHL (nursing and allied health profession literature), and SCOPUS (social sciences). We searched for EBQI publications without date restriction, other QI studies were limited to three years of QI publications as described below in more detail.

Eligibility Criteria

Eligibility criteria were organized in a SPIOS (study design, participants, intervention, outcome, setting) framework; full details are shown in the SDC. Briefly, we applied the following:

EBQI–labeled publications: All publications using EBQI terminology were included in the data abstraction.

Primary care effectiveness subsample: Among EBQI publications, we identified studies reporting effectiveness results for the evaluation of an EBQI initiative. Studies had to report on patient health, and we restricted to primary care to identify a more homogenous sample of research studies.

EBQI–compatible studies: Empirical studies involving U.S. healthcare professionals, reporting on an evaluation of a QI initiative in primary care, and documenting evidence review as part of their methodology to select, design, or implement a QI intervention. Evidence review was defined as a literature review undertaken at the beginning of the project, documentation of locally generated data to determine the need for the intervention (practice-based evidence), and/or utilizing of authoritative sources such as evidence-based clinical practice guidelines. Two independent literature reviewers screened citations and full-text publications; discrepancies were resolved through discussion in the team. Reviewers first excluded all citations that did not indicate an empirical evaluation of a QI initiative. The remaining citations were screened as full-text publication, applying all eligibility criteria described in the SDC (e.g., U.S.-based).

Data Abstraction and Synthesis

Data abstraction was tailored to the review questions. We used ten features in total to characterize the included studies (described in more detail in the SDC ):

Evidence to identify target : using evidence (data) to identify the target of the QI initiative

Iterative : conducting an interactive process for selecting the QI intervention

Engagement of stakeholders : reaching out within the organization to ensure a collaborative process

Evidence to identify intervention : reviewing evidence (research literature or local data) to select effective QI interventions

QI facilitation : use of facilitation of the QI process

Leadership involvement : involving clinical operations leadership in the QI initiative

Priority setting with leadership : setting priorities for the QI initiative together with clinical operations leadership

Frontline engagement : engaging frontline personnel early in the QI initiative

Evidence to determine success : using data to determine the success of the QI initiative

Analytic support : using analytic support to help QI teams

The abstraction domains had been developed by the QI content expert team members drawing on practical and research expertise (SH, ST, BT). The information was collated in evidence tables and component tables allow a concise overview. Effectiveness outcomes were summarized in a random-effects meta-analysis.

Expert Consultation

The preliminary scoping review results were presented to Dr. Lisa Rubenstein, a proponent and conceptual originator of EBQI. The formal consultation step aimed to ensure that the review addresses the right questions, identified all relevant literature, and synthesized the included material appropriately. Dr. Rubenstein was not involved in the planning of the review and assessed methods and results de novo. The consultation exercise resulted in one additional domain (priority setting with leadership) that was added to the data abstraction (see SDC ).

The literature searches identified 2001 citations. Of these, we obtained 496 for full-text inclusion screening. Figure 1 shows the flow diagram.

Flow diagram.

We included 211 publications, detailed in the evidence tables in the SDC. In total, we identified 170 diverse publications that used the term EBQI. SDC Figure 1 plots the number of EBQI publications over time and shows the rapid increase in frequency and popularity of EBQI. Two peaks emerged, one around 2006–2008, the other after 2016. The 170 identified publications are described in detail in an evidence table in the appendix (see SDC Table 1 ) and were used to address review question 1.

Review Question 1 Synthesis: How Is EBQI Defined?

The majority of EBQI–labeled publications did not define EBQI; only 23 of the 170 studies provided a definition or detailed description of the EBQI process. Studies highlighted different aspects of EBQI such as stakeholder engagement 13 or described EBQI broadly as a continuous quality improvement method. 14 Rubenstein et al. 15 defined EBQI as “a continuous quality improvement approach whose goal is translation of research on care delivery models into routine practice.” Figure 2 shows the terms used in the identified publications.

EBQI semantic definition overview.

Review Question 2 Synthesis: How Is Evidence Used to Inform Evidence-Informed Quality Improvement Initiatives?

The second evidence table (SDC Table 2 ) shows all 25 EBQI–labeled studies that reported on an evaluation of a QI initiative (listed first), followed by the 42 EBQI–compatible primary care evaluations, for a total of 67 EBQI–labeled or EBQI–compatible studies. The table shows the wide range of clinical topic areas and interventions addressed and describes their implementation strategy in detail. Across studies, most used published research literature to select interventions to be implemented in the QI initiative.

Review Question 2a Synthesis: How Is Evidence Defined?

In the 25 EBQI–labeled evaluations, 17 studies that provided information on the utilized evidence referred to published literature identified in a literature review. Ten EBQI studies referred to the use of local data. Six studies used expert panels and consensus meetings. Six studies referred to clinical practice guidelines that were reviewed to identify the QI intervention. Studies used these sources either alone or in combination.

Review Question 2b Synthesis: What Are the Components of EBQI?

Table 1 shows the 10 potential EBQI features that we abstracted for each study, the number of features characterizing each study, and the overall frequency of features across studies. EBQI–labeled studies (top half of Table 1 ) are followed by EBQI–compatible studies (bottom half of Table 1 ). Table 2 provides a summary of features across all 67 studies. Across studies, two thirds of studies reported having used evidence to identify an effective intervention, engaging stakeholders, using an iterative development, and involving frontline clinicians. In addition, all 67 identified studies used data to determine the success of the QI initiative.

When we compared features across the subsets, evidence to identify the target of the quality improvement intervention was more frequently reported in EBQI–labeled studies than in EBQI–compatible studies (72% vs 43%). Across EBQI–labeled and EBQI–compatible studies, involvement of leadership in priority setting for the quality improvement target (44% vs 21%) and the provision of analytic support (36% vs 26%) were least frequently reported. EBQI studies consistently reported more EBQI features: the median number of components used within study was 7 for the EBQI–labeled sample (maximum of 10) and 5.5 for the EBQI–compatible sample. The distributions in the two sets differed statistically significantly ( p = 0.037; Mann-Whitney U test).

Review Question 3 Synthesis: EBQI Effectiveness

We abstracted data from all 14 evaluations of primary care QI initiatives that used the term EBQI and that reported on a patient health outcome (SDC Table 3 ). Not all studies provided sufficient detail to allow effect size calculation. None of the studies compared two quality improvement strategies in a head-to-head comparison; hence, the documented effectiveness represents the effectiveness of the combined EBQI and implemented intervention. The forest plot in Figure 3 shows effect estimates for four studies reporting categorical outcomes, expressed as relative risk (RR), that could be combined in a meta-analysis. Studies assessed the implementation of a breast-feeding protocol in primary care, 16 an intervention targeting primary care referrals to smoking cessation clinics, 13 the implementation of collaborative care for depression, 14 and a program to increase adherence to immunization guidelines for adults with diabetes. 17 The effectiveness estimates varied widely by quality improvement target and study, only one of the studies reported a statistically significant effect, but all suggested more improvements in the EBQI group.

EBQI effectiveness.

The scoping review shows that the evidence base for EBQI is growing, and to our knowledge, this is the first study that provides an overview of the available EBQI literature.

We identified EBQI components and their relative frequency, both across EBQI–labeled studies and in comparison to studies that were similar in approach to EBQI without using EBQI terminology. The focus on evidence at multiple stages of the QI initiative and the strong emphasis on engaging stakeholders were key features.

However, “evidence” was often not systematically described in the identified studies. Not all studies reported a review of the evidence to identify a target for the QI initiative (54% across EBQI–labeled and EBQI–compatible studies). This gap calls into question the focus of these studies on using evidence to identify and define QI aims, a critical entry point for introducing evidence into the QI process. Most, but not all (88%) of the studies reviewed evidence to select and shape the QI intervention design, another critical entry point for applying published research, local data, and implementation science knowledge. More complete reporting on evidence use across studies would promote assessment of fidelity to the EBQI process, which is critical to evaluation of the success of the QI initiative and our ability to learn from initiatives across settings. 18

Our review also shows that overall, there is still insufficient information regarding the effectiveness of EBQI. We only found a small number of studies using EBQI that reported on key and patient-centered outcomes, i.e., patient health, and studies addressed substantially different intervention targets, ranging from breast-feeding to depression treatment. We did not find studies that compared EBQI with other quality improvement strategies in head-to-head comparisons; hence, the effect of EBQI in the included studies was invariably confounded with the QI content. It is not known yet how EBQI compares to other quality improvement strategies, in particular quality improvement interventions that are based on anecdotal evidence. Future research should evaluate the comparative effectiveness of EBQI to provide more information on this critical aspect.

Our review has several limitations. While we systematically identified all known EBQI publications, we sampled the literature for EBQI–compatible studies and restricted to those published in recent years and limited to primary care given the large QI literature. 19 The sampling strategy was chosen to obtain a systematic and pragmatic sample that would serve as an exemplar of EBQI–compatible studies. However, it should be noted that earlier approaches were not included, which undoubtedly left out important approaches, and EBQI–compatible approaches in other fields, such as improvements in hospitals in international settings, could have provided additional important information.

We show that EBQI is a promising and growing strategy that aims to integrate prior scientific findings and methods into QI initiatives. Commonly used EBQI features integrate evidence throughout the improvement process, from the initial developmental phase of the QI initiative through to its evaluation. Future research should clearly document EBQI processes to enable better characterization of core initiative features and should assess the comparative effectiveness and success in addressing patient-centered goals.

Institute for Healthcare Improvement Website http://www.ihi.org/ihi . Accessed 5/2/2022.

Deming WE . Quality, Productivity and Competitive Position. Cambridge, MA: MIT Center for Advanced Engineering Studies; 1982.

Google Scholar  

Meyers D, Miller T, Genevro J, Zhan C, De La Mare J, Fournier A, et al. EvidenceNOW: balancing primary care implementation and implementation research. Ann Fam Med. 2018;16(Suppl 1):S5-s11. https://doi.org/10.1370/afm.2196

Article   PubMed   PubMed Central   Google Scholar  

Nelson K, Reddy A, Stockdale SE, Rose D, Fihn S, Rosland A-M, et al. , editors. The Primary Care Analytics Team: Integrating Research and Clinical Care Within the Veterans Health Administration Office of Primary Care. Healthcare; 2021: Elsevier.

Ovretveit J, Hempel S, Magnabosco JL, Mittman BS, Rubenstein LV, Ganz DA . Guidance for research-practice partnerships (R-PPs) and collaborative research. J Health Manag. 2014;28(1):115-26. https://doi.org/10.1108/JHOM-08-2013-0164

Melnyk BM, Buck J, Gallagher-Ford L . Transforming quality improvement into evidence-based quality improvement: a key solution to improve healthcare outcomes. Worldviews Evid.-Based Nurs. 2015;12(5):251-2. https://doi.org/10.1111/wvn.12112

Article   PubMed   Google Scholar  

Shojania KG, Grimshaw JM . Evidence-based quality improvement: the state of the science. Health Aff (Millwood). 2005;24(1):138-50. https://doi.org/10.1377/hlthaff.24.1.138

Stockdale SE, Zuchowski J, Rubenstein LV, Sapir N, Yano EM, Altman L, et al. Fostering evidence-based quality improvement for patient-centered medical homes: initiating local quality councils to transform primary care. Health Care Manag Rev. 2018;43(2):168-80. https://doi.org/10.1097/hmr.0000000000000138

Article   Google Scholar  

Arksey H, O'Malley L . Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:19-32.

Hempel S, Bolshakova M, Turner B, Rose D, Dinalo J, Motala A, et al. Scoping Review Protocol: Evidence-Based Quality Improvement as an Implementation Strategy for Evidence-Based Practices. 2020. https://osf.io/hr5bj/ . Accessed 5/2/2022.

Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467-73. https://doi.org/10.7326/M18-0850

Ottawa Hospital Research Institute, University of Oxford. PRISMA for Scoping Reviews. 2015. http://www.prisma-statement.org/Extensions/ScopingReviews . Accessed 27 July 2020.

Yano EM, Rubenstein LV, Farmer MM, Chernof BA, Mittman BS, Lanto AB, et al. Targeting primary care referrals to smoking cessation clinics does not improve quit rates: implementing evidence-based interventions into practice. Health Serv Res. 2008;43(5 Pt 1):1637-61. https://doi.org/10.1111/j.1475-6773.2008.00865.x

Chaney EF, Rubenstein LV, Liu CF, Yano EM, Bolkan C, Lee M, et al. Implementing collaborative care for depression treatment in primary care: a cluster randomized evaluation of a quality improvement practice redesign. Implement Sci. 2011;6:121. https://doi.org/10.1186/1748-5908-6-121

Rubenstein LV, Chaney EF, Ober S, Felker B, Sherman SE, Lanto A, et al. Using evidence-based quality improvement methods for translating depression collaborative care research into practice. Fam Syst Health. 2010;28(2):91-113. https://doi.org/10.1037/a0020302

Dumphy D, Thompson J, Clark M . A breastfeeding quality improvement project in rural primary care. J Hum Lac. 2016;32(4):633-41. https://doi.org/10.1177/0890334416662240

Gottlieb RP, Dols JD . Improving vaccination rates in adults with type 2 diabetes in a family practice setting through the use of evidence-based interventions. J Doctoral Nurs Pract. 2018;11(2):151-9. https://doi.org/10.1891/2380-9418.11.2.151

Stockdale SE, Hamilton AB, Bergman AA, Rose DE, Giannitrapani KF, Dresselhaus TR, et al. Assessing fidelity to evidence-based quality improvement as an implementation strategy for patient-centered medical home transformation in the Veterans Health Administration. Implement Sci. 2020;15(1):18. https://doi.org/10.1186/s13012-020-0979-y

Hempel S, Rubenstein LV, Shanman RM, Foy R, Golder S, Danz M, et al. Identifying quality improvement intervention publications--a comparison of electronic search strategies. Implement Sci. 2011;6:85. https://doi.org/10.1186/1748-5908-6-85

Badru M. A Clinical Practice Guideline to Reduce Behavioral Outbursts in Veterans with Posttraumatic Stress Disorder. Clinical Practice Guideline to Reduce Behavioral Outbursts in Veterans with Posttraumatic Stress Disorder. 2017:1-.

Bennett JG . Implementing Lipid Screening Guidelines for Children in a Rural Health Clinic. Implementing Lipid Screening Guidelines for Children in a Rural Health Clinic. 2016:1-.

Cohen AN, Chinman MJ, Hamilton AB, Whelan F, Young AS . Using patient-facing kiosks to support quality improvement at mental health clinics. Med Care. 2013;51(3 Suppl 1):S13-20. https://doi.org/10.1097/MLR.0b013e31827da859

Fortney J, Enderle M, McDougall S, Clothier J, Otero J, Altman L, et al. Implementation outcomes of evidence-based quality improvement for depression in VA community based outpatient clinics. Implement Sci. 2012;7:30. https://doi.org/10.1186/1748-5908-7-30

Fortney JC, Enderle MA, Clothier JL, Otero JM, Williams JS, Pyne JM . Population level effectiveness of implementing collaborative care management for depression. Gen Hosp Psychiatry. 2013;35(5):455-60. https://doi.org/10.1016/j.genhosppsych.2013.04.010

Fox AB, Hamilton AB, Frayne SM, Wiltsey-Stirman S, Bean-Mayberry B, Carney D, et al. Effectiveness of an evidence-based quality improvement approach to cultural competence training: the Veterans Affairs’ “Caring for Women Veterans” Program. J Contin Educ Health Prof. 2016;36(2):96-103. https://doi.org/10.1097/ceh.0000000000000073

Gadbois C, Chin ED, Dalphonse L . Health promotion in an opioid treatment program an evidence-based nursing quality improvement project. J Addict Nurs. 2016;27(2):127-42. https://doi.org/10.1097/JAN.0000000000000124

Klause KT, Dodds VA, Selleck C, Deupree JP . Addressing intimate partner violence at a safety-net clinic for adults. J Nurs Pract. 2020;16(2):154-7. https://doi.org/10.1016/j.nurpra.2019.08.019

Le Flore G. Applying Clinical Guidelines to Curtail Opioid Overprescribing in Primary Care. Applying Clinical Guidelines to Curtail Opioid Overprescribing in Primary Care. 2017:1-.

Meredith LS, Batorsky B, Cefalu M, Darling JE, Stockdale SE, Yano EM, et al. Long-term impact of evidence-based quality improvement for facilitating medical home implementation on primary care health professional morale. BMC Fam Pract. 2018;19(1):149. https://doi.org/10.1186/s12875-018-0824-4

Ong A. Ripple effect: shared governance and nurse engagement. Nurs Manag. 2017;48(10):28-34. https://doi.org/10.1097/01.NUMA.0000524811.11040.05

Rizzo KA . Effectiveness of Continuous Subcutaneous Insulin Infusion Therapy Education in a Clinic Setting. Effectiveness Of Continuous Subcutaneous Insulin Infusion Therapy Education In A Clinic Setting. 2018:1-.

Rubenstein LV, Meredith LS, Parker LE, Gordon NP, Hickey SC, Oken C, et al. Impacts of evidence-based quality improvement on depression in primary care: a randomized experiment. J Gen Intern Med. 2006;21(10):1027-35. https://doi.org/10.1111/j.1525-1497.2006.00549.x

Sherman SE, Chapman A, Garcia D, Braslow JT . Improving recognition of depression in primary care: a study of evidence-based quality improvement. Jt Comm J Qual Saf. 2004;30(2):80-8.

PubMed   Google Scholar  

Starkey M, Wiest D, Qaseem A . Improving depression care through an online learning collaborative. Am J Med Qual. 2016;31(2):111-7. https://doi.org/10.1177/1062860614555883

Walker CT, Gullotti DM, Prendergast V, Radosevich J, Grimm D, Cole TS, et al. Implementation of a standardized multimodal postoperative analgesia protocol improves pain control, reduces opioid consumption, and shortens length of hospital stay after posterior lumbar spinal fusion. Neurosurgery. 2019. https://doi.org/10.1093/neuros/nyz312

Walker-Smith TL . A Prospective Quality improvement project using a mammography risk assessment tool to increase screening mammogram use with low-income Hispanic Women: a doctor of nursing practice project report. Corpus Christi, Texas: Texas A&M University-Corpus Christi 2018.

Whitten SK, Stanik-Hutt J . Group cognitive behavioral therapy to improve the quality of care to opioid-treated patients with chronic noncancer pain: a practice improvement project. J Am Assoc Nurs Pract. 2013;25(7):368-76. https://doi.org/10.1111/j.1745-7599.2012.00800.x

Yoon J, Chow A, Rubenstein LV . Impact of medical home implementation through evidence-based quality improvement on utilization and costs. Med Care. 2016;54(2):118-25. https://doi.org/10.1097/mlr.0000000000000478

Young LS, Crausman RS, Fulton JP . Suboptimal opioid prescribing: a practice change project. R I Med J (2013). 2018;101(2):41-4.

Barclay C, Viswanathan M, Ratner S, Tompkins J, Jonas DE . Implementing evidence-based screening and counseling for unhealthy alcohol use with epic-based electronic health record tools. Jt Comm J Qual Patient Saf. 2019;45(8):566-74. https://doi.org/10.1016/j.jcjq.2019.05.009

Bowen DJ, Powers DM, Russo J, Arao R, LePoire E, Sutherland E, et al. Implementing collaborative care to reduce depression for rural native American/Alaska native people. BMC Health Serv Res. 2020;20(1):34. https://doi.org/10.1186/s12913-019-4875-6

Breaux-Shropshire TL, Huie R, Shropshire TS, Wyatt A, Shropshire AT, Estrada CA, et al. First steps in improving blood pressure control among primary care hypertensive veterans utilizing quality improvement tools. Ala Nurs. 2017;44(3):19-22.

Brodie N, McPeak KE . Improving human papilloma virus vaccination rates at an urban pediatric primary care center. Pediatr Qual Saf. 2018;3(5):e098. https://doi.org/10.1097/pq9.0000000000000098

Burge SA, Powell W, Mazour L . A quality improvement endeavor improving depression screening for rural older adults. O J Rural Nurs Health Care. 2019;19(2):44-64. https://doi.org/10.14574/ojrnhc.v19i2.563

Murphy Buschkoetter KL, Powell W, Mazour L . Implementation of a comprehensive diabetic foot exam protocol in rural primary care. Online J Rural Nurs Health Care. 2019;19(1):43-63. https://doi.org/10.14574/ojrnhc.v19i1.560

Camp NL, Robert RC, Nash JE, Lichtenstein CB, Dawes CS, Kelly KP . Modifying provider practice to improve assessment of unhealthy weight and lifestyle in young children: translating evidence in a quality improvement initiative for at-risk children. Child Obes. 2017;13(3):173-81. https://doi.org/10.1089/chi.2016.0124

Campbell K, Carpenter KLH, Espinosa S, Hashemi J, Qiu Q, Tepper M, et al. Use of a Digital Modified Checklist for Autism in Toddlers - revised with follow-up to improve quality of screening for autism. J Pediatr. 2017;183:133-9.e1. https://doi.org/10.1016/j.jpeds.2017.01.021

Colborn KL, Helmkamp L, Bender BG, Kwan BM, Schilling LM, Sills MR . Colorado Asthma toolkit implementation improves some process measures of asthma care. J Am Board Fam Med. 2019;32(1):37-49. https://doi.org/10.3122/jabfm.2019.01.180155

Daaleman TP, Brock D, Gwynne M, Weir S, Dickinson I, Willis B, et al. Implementing lean in academic primary care. Qual Manag Health Care. 2018;27(3):111-6. https://doi.org/10.1097/qmh.0000000000000173

Fabre JC, Andresen PA, Wiltz GM . Closing the loop on electronic referrals: a quality improvement initiative using the care coordination model. J Ambul Care Manag. 2020;43(1):71-80. https://doi.org/10.1097/jac.0000000000000315

Fisher-Borne M, Preiss AJ, Black M, Roberts K, Saslow D . Early outcomes of a multilevel human papillomavirus vaccination pilot intervention in federally qualified health centers. Acad Pediatrics. 2018;18(2):S79-S84. https://doi.org/10.1016/j.acap.2017.11.001

Fortney JC, Pyne JM, Ward-Jones S, Bennett IM, Diehl J, Farris K, et al. Implementation of evidence-based practices for complex mood disorders in primary care safety net clinics. Fam Syst Health. 2018;36(3):267-80. https://doi.org/10.1037/fsh0000357

Garza L, Dols J, Gillespie M . An initiative to improve primary prevention of cardiovascular disease in adults with type II diabetes based on the ACC/AHA (2013) and ADA (2016) guidelines. J Am Assoc Nurs Pract. 2017;29(10):606-11. https://doi.org/10.1002/2327-6924.12492

Gold R, Bunce A, Cowburn S, Davis JV, Hollombe C, Nelson CA, et al. Cardiovascular care guideline implementation in community health centers in Oregon: a mixed-methods analysis of real-world barriers and challenges. BMC Health Serv Res. 2017;17(1):253. https://doi.org/10.1186/s12913-017-2194-3

Green BB, Fuller S, Anderson ML, Mahoney C, Mendy P, Powell SL . A quality improvement initiative to increase colorectal cancer (CRC) screening: collaboration between a Primary Care Clinic and Research Team. J Fam Med. 2017;4(3). https://doi.org/10.26420/jfammed.2017.1115

Hanlin RB, Asif IM, Wozniak G, Sutherland SE, Shah B, Yang J, et al. Measure accurately, act rapidly, and partner with patients (MAP) improves hypertension control in medically underserved patients: Care Coordination Institute and American Medical Association Hypertension Control Project Pilot Study results. J Clin Hypertens (Greenwich). 2018;20(1):79-87. https://doi.org/10.1111/jch.13141

Article   CAS   PubMed   PubMed Central   Google Scholar  

Hawk M, Nowalk MP, Moehling KK, Pavlik V, Raviotta JM, Brown AE, et al. Using a mixed methods approach to examine practice characteristics associated with implementation of an adult immunization intervention using the 4 Pillars Practice Transformation Program. J Healthc Qual. 2017;39(3):153-67. https://doi.org/10.1097/jhq.0000000000000071

Jonas DE, Miller T, Ratner S, McGuirt B, Golin CE, Grodensky C, et al. Implementation and quality improvement of a screening and counseling program for unhealthy alcohol use in an academic general internal medicine practice. J Healthc Qual. 2017;39(1):15-27. https://doi.org/10.1097/jhq.0000000000000069

Knierim KE, Hall TL, Dickinson LM, Nease DE, Jr., de la Cerda DR, Fernald D, et al. Primary care practices' ability to report electronic clinical quality measures in the EvidenceNOW southwest initiative to Improve Heart Health. JAMA Netw Open. 2019;2(8):e198569. https://doi.org/10.1001/jamanetworkopen.2019.8569

Implementation of psychiatric e-consultation in family medicine community health centers. Int J Psychiatry Med. 2019;54(4/5):296-306. https://doi.org/10.1177/0091217419869081

Makelarski JA, DePumpo M, Boyd K, Brown T, Kho A, Navalkha C, et al. Implementation of systematic community resource referrals at small primary care practices to promote cardiovascular disease self-management. J Healthc Qual. 2019. https://doi.org/10.1097/jhq.0000000000000234

Minsky N, Tamler R , editors. Endocrine eConsults improve access to care for the underserved. ACM International Conference Proceeding Series; 2017. March 13_KQ2_CINAH, PubMed, SCOPUS.

Modica C, Lewis JH, Bay C . Colorectal cancer: applying the value transformation framework to increase the percent of patients receiving screening in federally qualified health centers. Prev Med Rep. 2019;15:100894. https://doi.org/10.1016/j.pmedr.2019.100894

Nagykaldi ZJ, Scheid D, Zhao D, Mishra B, Greever-Rice T . An innovative community-based model for improving preventive care in rural counties. J Am Board Fam Med. 2017;30(5):583-91. https://doi.org/10.3122/jabfm.2017.05.170035

Nowalk MP, Moehling KK, Zhang S, Raviotta JM, Zimmerman RK, Lin CJ . Using the 4 pillars to increase vaccination among high-risk adults: who benefits? Am J Manage Care. 2017;23(11):651-5.

Ober AJ, Watkins KE, Hunter SB, Ewing B, Lamp K, Lind M, et al. Assessing and improving organizational readiness to implement substance use disorder treatment in primary care: findings from the SUMMIT study. BMC Fam Pract. 2017;18(1):107. https://doi.org/10.1186/s12875-017-0673-6

Quanbeck A, Brown RT, Zgierska AE, Jacobson N, Robinson JM, Johnson RA, et al. A randomized matched-pairs study of feasibility, acceptability, and effectiveness of systems consultation: a novel implementation strategy for adopting clinical guidelines for opioid prescribing in primary care. Implement Sci. 2018;13(1):21. https://doi.org/10.1186/s13012-018-0713-1

Regan ME . Implementing an evidence-based clinical decision support tool to improve the detection, evaluation, and referral patterns of adult chronic kidney disease patients in primary care. J Am Assoc Nurs Pract. 2017;29(12):741-53. https://doi.org/10.1002/2327-6924.12505

Richards JE, Bobb JF, Lee AK, Lapham GT, Williams EC, Glass JE, et al. Integration of screening, assessment, and treatment for cannabis and other drug use disorders in primary care: an evaluation in three pilot sites. Drug Alcohol Depend. 2019;201:134-41. https://doi.org/10.1016/j.drugalcdep.2019.04.015

Roderick SS, Burdette N, Hurwitz D, Yeracaris P . Integrated behavioral health practice facilitation in patient centered medical homes: a promising application. Fam Syst Health. 2017;35(2):227-37. https://doi.org/10.1037/fsh0000273

Savas A, Smith E, Hay B . EHR quality indicator tracking: a process improvement pilot project to meet MACRA requirements. Nurs Pract. 2019;44(4):30-9. https://doi.org/10.1097/01.NPR.0000554084.05450.0e

Schaeffer AM, Jolles D . Not missing the opportunity: improving depression screening and follow-up in a multicultural community. Jt Comm J Qual Patient Saf. 2019;45(1):31-9. https://doi.org/10.1016/j.jcjq.2018.06.002

Schiff GD, Reyes Nieva H, Griswold P, Leydon N, Ling J, Federico F, et al. Randomized trial of reducing ambulatory malpractice and safety risk: results of the Massachusetts PROMISES Project. Med Care. 2017;55(8):797-805. https://doi.org/10.1097/mlr.0000000000000759

Schurman JV, Deacy AD, Johnson RJ, Parker J, Williams K, Wallace D, et al. Using quality improvement methods to increase use of pain prevention strategies for childhood vaccination. World J Clin Pediatr. 2017;6(1):81-8. https://doi.org/10.5409/wjcp.v6.i1.81

Senger JJ . A Concussion Toolkit Educational Session: promoting evidence-based management of youth concussion in a rural primary care setting. Concussion Toolkit Educational Session: Promoting Evidence-Based Management Of Youth Concussion In A Rural Primary Care Setting. 2018:1-.

Shah T, Patel-Teague S, Kroupa L, Meyer AND, Singh H . Impact of a national QI programme on reducing electronic health record notifications to clinicians. BMJ Qual Saf. 2019;28(1):10-4. https://doi.org/10.1136/bmjqs-2017-007447

Sloand E, Vangraafeiland B, Holm A, MacQueen A, Polk S . Text message quality improvement project for influenza vaccine in a low-resource largely Latino pediatric population. J Healthc Qual. 2019;41(6):362-8. https://doi.org/10.1097/JHQ.0000000000000190

van Eeghen C, Kennedy AG, Pasanen ME, MacLean CD . A new quality improvement toolkit to improve opioid prescribing in primary care. J Am Board Fam Med. 2020;33(1):17-26. https://doi.org/10.3122/jabfm.2019.01.190238

Weiner BJ, Rohweder CL, Scott JE, Teal R, Slade A, Deal AM, et al. Using practice facilitation to increase rates of colorectal cancer screening in community health centers, North Carolina, 2012-2013: Feasibility, Facilitators, and Barriers. Prev Chronic Dis. 2017;14:E66. https://doi.org/10.5888/pcd14.160454

Williams MD, Sawchuk CN, Shippee ND, Somers KJ, Berg SL, Mitchell JD, et al. A quality improvement project aimed at adapting primary care to ensure the delivery of evidence-based psychotherapy for adult anxiety. BMJ Open Qual. 2018;7(1):e000066. https://doi.org/10.1136/bmjoq-2017-000066

Yusupov E, Krishnamachari B, Rand S, Abdalla M, Zwibel H . Quality of hypertension care: an improvement initiative in two outpatient health care centers. J Eval Clin Pract. 2019;25(3):463-8. https://doi.org/10.1111/jep.13067

Download references

Acknowledgements

The study was funded by the Department of Veteran Affairs. The findings are those of the authors and do not necessarily represent the views of the Department of Veteran Affairs or the United States Government.

Author information

Authors and affiliations.

Southern California Evidence Review Center, University of Southern California, Los Angeles, CA, USA

Susanne Hempel PhD, Maria Bolshakova BS, Aneesa Motala BA & Ning Fu PhD

Gehr Family Center for Health Systems Science and Innovation, University of Southern California, Los Angeles, CA, USA

Susanne Hempel PhD, Barbara J. Turner MD & Aneesa Motala BA

Center for the Study of Healthcare Innovation, Implementation & Policy, VA Greater Los Angeles Healthcare System, Los Angeles, CA, USA

Danielle Rose PhD & Susan Stockdale PhD

School of Economics, Shanghai University of Finance and Economics, Shanghai, China

Ning Fu PhD

University of Southern California, Los Angeles, CA, USA

Jennifer Dinalo PhD & Chase G. Clemesha MD

RAND Health, RAND Corporation, Santa Monica, CA, USA

Susanne Hempel PhD, Aneesa Motala BA & Lisa Rubenstein MD

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Ning Fu PhD .

Ethics declarations

Conflict of interest.

The authors declare that they do not have a conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The paper abstract has been presented for “Poster Session: Clinical Care Settings: System-level Interventions” at the Academy Health Virtual D&I Conference, 2020.

Supplementary Information

(DOCX 656 kb)

Rights and permissions

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Hempel, S., Bolshakova, M., Turner, B.J. et al. Evidence-Based Quality Improvement: a Scoping Review of the Literature. J GEN INTERN MED 37 , 4257–4267 (2022). https://doi.org/10.1007/s11606-022-07602-5

Download citation

Published : 29 September 2022

Issue Date : December 2022

DOI : https://doi.org/10.1007/s11606-022-07602-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• quality improvement
• evidence-based
• scoping review
• Find a journal
• Publish with us
• Track your research

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 3, Issue 1
• A quality improvement project to improve the effectiveness and patient-centredness of management of people with mild-to-moderate kidney disease in primary care
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Nicola Thomas ,
• Hugh Gallagher ,
• Neerja Jain
• London South Bank University, Epsom and St Helier NHS Trust and Kidney Research UK
• Correspondence to Nicola Thomas nicola.thomas{at}lsbu.ac.uk

Chronic kidney disease (CKD) stages 3 to 5, affects 6-7% of the adult population and is an important risk factor for both advanced kidney disease and cardiovascular disease. This paper describes a quality improvement project that aimed to establish consistent implementation of best practice in people with stage 3-5 kidney disease who were managed in primary care.

The intervention was a Care Bundle for CKD. The bundle included three evidence-based, high impact interventions based on National Institute for Care Excellence (NICE, 2008) guidance, with an additional and novel self-management element.

29 GP Practices in England and Wales began the study. They undertook training in clinical management of CKD and in facilitation of self-management, with the self-management content designed and led by patients. Practices were asked to report baseline and then monthly outcome data extracted from practice computer systems. The project team provided implementation and ongoing quality improvement support for participating Practices.

Ten Practices dropped out of the study following the training. Data submissions were incomplete in six Practices who continued to apply the care bundle. At the project end, a decision was taken by the study team to perform the final analysis on those thirteen Practices which completed the project and submitted at least six sets of monthly Practice-level outcome data. In these Practices the Care Bundle was applied to under 20% of the registered CKD stage 3 to 5 population in 5 Practices, 20-29% in 3 Practices, 30-49% in 2 Practices and ≥50% in 3 Practices (998 patients in total). Of these, 671 patients (75%) agreed to the self-management component of the intervention.

The reliability (at project end) in those who received the Bundle was 100%. The Bundle was applied to an additional 315 patients in the six Practices who completed the project but did not submit regular practice-level monthly data.

In the thirteen remaining Practices, the achievement of NICE (2008) blood pressure targets at the start of the project was 74.8% in people with CKD stage 3-5 and no diabetes and 48% in people with CKD stage 3-5 and diabetes. At the project end these figures in the same Practices were 76.7% and 49.2% respectively. These improvements were achieved in spite of Practices increasing their recording of prevalence rate (that is, identifying and recording more patients with CKD on the Practice CKD Register).

In conclusion, a care bundle can be implemented in primary care. However, maintaining engagement with primary care health care professionals and maximising exposure to an intervention in patients seen infrequently are significant challenges to generalisation and sustainability.

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See:

http://creativecommons.org/licenses/by-nc/2.0/

http://creativecommons.org/licenses/by-nc/2.0/legalcode

https://doi.org/10.1136/bmjquality.u201337.w825

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

There is widespread variation in the identification and treatment of kidney disease in primary care. The recorded prevalence of chronic kidney disease (CKD) within a Practice (measured by the number of patients on the CKD Register) is approximately 60% of that expected from epidemiological data (1). Performance against quality markers in those patients who have been correctly identified is also highly variable (1).

There is therefore a need for better systems to enable consistent implementation of best practice. However, the evidence base for quality improvement (QI) in mild-to-moderate CKD is limited. A systematic review identified very few studies involving QI interventions to lower systolic blood pressure in CKD, and those there were largely focused on high risk groups (2), although more recent data suggest that a programme of audit-based education of healthcare professionals may be of benefit (3).

Furthermore, people with early CKD often wish to manage their condition with support from healthcare professionals (4), but anecdotal evidence from patients suggests that some healthcare professionals are not aware of or confident in encouraging their patients to self-manage. In addition, many people with a coded CKD diagnosis are not informed of it, and thereby denied any opportunity to self-care (5). Few CKD-specific educational materials have been developed to support those with early stage disease.

We therefore designed and tested an intervention to: facilitate the delivery of consistent care to people with CKD in primary care; and offer patients the opportunity to self-manage.

Chronic kidney disease stages 3 to 5 (CKD) affects between 6 and 7% of the population (6). It is an important risk factor for both end-stage renal disease and for cardiovascular disease (CVD) (7,8), with which it frequently co-exists, but is frequently unrecognised, since it causes no symptoms in the early stages.

In the UK, CKD is generally managed in primary care, and has been included under the General Practice pay-for-performance scheme since 2006. NICE guidelines on the identification and management of CKD were published in 2008 and will be updated in 2014 (9). Early identification is beneficial as there are effective interventions to slow progression and reduce cardiovascular risk (10). Implementation of best practice may be cost effective (11).

Best practice for CKD includes treatment of high blood pressure, with preferential use of drugs inhibiting the renin-angiotensin system where there is proteinuria. NICE also recommend that patients are asked about opportunities for self-management (9).

Baseline Measurement

In those thirteen Practices which completed the project and submitted at least six sets of monthly Practice-level outcome data the mean recorded prevalence of CKD at baseline was 4% ± 1.54 % (SD). The proportion of patients on the CKD Register treated to the NICE (2008) target for blood pressure control was 74.8% (with no diabetes) and 48.0% (with diabetes).

We employed a Care Bundle approach. Care Bundles are groups of evidence-based, high impact interventions that have been well described in secondary care (12, 13). To our knowledge this is one of the first care bundle projects that have been described in primary care in England and Wales. A CKD-specific Care Bundle for primary care had been developed by the project team and piloted in a single Practice in 2008-2009 (14). Concurrently a patient-focused intervention had been designed by an expert group, which included patients. This too was piloted in a single Practice (15).

We combined the practitioner and patient components into a single Bundle, the components of which were:

Asking the patient if they wished to participate in a self-management programme

Measuring and documenting proteinuria and prescribing an ACE inhibitor or Angiotensin Receptor Blocker (ACEi/ARB) if significant proteinuria was present

Documenting blood pressure and treating if above NICE target

Documenting cardiovascular risk

The three clinical components of the Bundle are recommended by NICE (9). The self-management intervention, for those that chose to participate, involved offering an educational resource (DVD and written information) and short questionnaires/prompts to help patients identify their needs for discussion at their appointment.

Group education for patients

In addition a group education session for patients who had agreed to self-management was held in five Practices. Group education sessions were led by people who had CKD and were supported by a renal nurse. Sessions lasted two hours and included specific advice on how to self-manage early CKD (blood pressure control, tablets, diet, and smoking cessation) and also general advice on how to get the best out of a consultation with a doctor or nurse. People with CKD were therefore enabled to develop a pro-active interest in managing their health with a view to changing behaviour.

Training of Practices

Prior to commencing the project, participating Practices received dedicated training lasting 2-2.5 hours from two members of the project team (a renal nurse plus a member of the patient and service user advisory group (PSUAG), see below). The training covered both general information on CKD management in primary care and specific detail on the QI intervention to be implemented.

Patient involvement

The project was co-designed and delivered by patients. The PSUAG was established and its purpose was to help effect change and improvement in the involved Practices. This group consisted of six people who either had experience of kidney disease as service users/carers, or who had experienced other long-term vascular conditions (diabetes and heart disease). Their role was to develop and inform the project alongside other (clinical) members of the project team. Members of the PSUAG attended the grant application interview for project funding, created the self-management package, led on the group education sessions for patients, and developed and delivered self-management training for practitioners (see below). They were supported by the wider team in these activities. Later in the project, PSUAG members played an important role in sustaining and disseminating the work, and contributed to follow-on project proposals.

Members of the PSUAG received the same honoraria for their time as healthcare practitioners in the project. The project team felt this to be an important factor recognising the value of service user involvement.

Practices were asked to apply the Care Bundle to patients on their CKD Register. The project protocol did not prescribe a specific setting for application of the Bundle, but suggested settings included a dedicated CKD clinic, a generic long-term conditions clinic or ad-hoc delivery.

Each month Practices were asked to send anonymised Care Bundle data to the project team. Each element of the Bundle was analysed separately as a process measure and overall Bundle reliability was recorded on a run chart. After collation of Bundle data, monthly teleconferences were held with participating Practices to discuss and review progress. The project team was represented on these calls by one of the project co-ordinators (a renal nurse) and, usually, a member of the PSUAG.

The Model for Improvement Plan-Do-Study-Act (PDSA) cycles were used as a framework. After each call the project co-ordinator emailed a summary of the call, tests of change, and action points to all those attending the call. At each subsequent teleconference the action points from the previous call were reviewed.

Data collection

Practice-level data outcome were collected at baseline, and then monthly, using data queries run on Practice computer systems. Data were sent electronically by Practices to the project team following email prompts from the project administrator. Search queries were written in-house by Practices, aided if necessary by the project team. Different IT systems were in use across the participating Practices.

The outcomes recorded were:

Recorded prevalence of CKD (i.e. % of adult population on CKD register)

Proportion of patients on CKD Register treated to NICE target for blood pressure control (below 140/90 mmHg or below 130/80 mmHg if patient had Type 1 or Type 2 diabetes and/or proteinuria (albumin-creatinine ratio, ACR > 70)

We intended to achieve 95% reliability in the application of a Care Bundle for chronic kidney disease (CKD) in patients on the CKD Register within participating Practices by October 2012. Our main outcome goal was to achieve NICE targets for systolic blood pressure (BP) in 80% of people on the CKD Register within Practices (if 95% reliability not achieved).

Twenty nine GP Practices were recruited to the project and trained. Three Practices dropped out immediately after the training and three Practices never sent any baseline data, so twenty three Practices started the project. One Practice dropped out during the project. Thirteen of the remaining Practices sent at least 6 months practice-level data. An additional six Practices continued to apply the care bundle but sent less than 6 months practice-level data. These Practices have been excluded from our analyses. The Bundle was applied to an additional 315 patients in the six Practices who completed the project but did not submit regular practice-level monthly data.

Whilst we did not achieve our target, there were improvements in blood pressure management. In the 13 Practices who submitted ≥ 6 months data, NICE blood pressure targets were achieved in 61.4% at the start of the project and 62.8 % at the project end. These improvements were achieved in spite of a number of Practices increasing their recorded prevalence rate of CKD. One Practice increased their recorded prevalence rate by 3.5% (151 people) but at the same time increased the proportion of people achieving NICE blood pressure targets by 22%. One Practice in North Wales saw its recorded prevalence rise from 4.8 to 7.6%. Overall prevalence recording in the thirteen Practices increased during the project to 4.9% (±1.62%).

In thirteen Practices the Care Bundle was applied to 20-29% of the registered CKD stage 3 to 5 population in 3 Practices, 30-39% in 2 Practices and ≥50% in 3 Practices (998 patients in total). Of these, 671 patients (75%) agreed to the self-management component of the intervention.

Lessons and Limitations

Intervention exposure

We demonstrated that the Care Bundle for CKD could be implemented reliably, and improvements in care were evident in some practices, such as increased prevalence recording and blood pressure control. The most challenging aspect of the project was achieving adequate intervention exposure. Our ability to assess the impact of the bundle upon practice-level measures has been limited by the proportions of patients on Practices’ CKD Registers who received the intervention. A Practice with a list size of 10,000 may have 600-700 patients on their CKD register, a majority of whom might only be reviewed annually. Many of these patients have multiple co-morbidities and the settings for such reviews vary between Practices and between patients within Practices. The team therefore worked with a small number of Practices using a goal-defined approach to increase the intervention exposure, and by the end of the project more than 50% of registered patients had been seen in three Practices.

Achieving and maintaining ‘buy-in’ from Practices, especially in the ever-changing primary care environment, was also difficult.

Service-user involvement

The project’s particular strength was the initiation and development of a patient and carer advisory group (PSUAG), that clearly demonstrated how patients and service users can meaningfully participate in quality improvement projects and lead on some elements.

All Practices informally reported benefits of the initial training they received on CKD management. Even in Practices where the penetration of the Bundle was low, practitioners reported greater awareness of CKD and its treatment, and more tangible benefits in terms of improved identification, monitoring, and blood pressure management. One Practice reported that there had been a ‘big impact of the education’ on their already educated and relatively affluent population.

Informing patients about their condition

In addition, practitioners reported that more patients were being advised of their diagnosis of CKD, which is widely recognised as an area of difficulty (16). Designing and delivering training was more complex than expected, as the motivations for participation in the project differed between Practices.

The impact of the Care Bundle varied across Practices and across areas of care within each Practice. Some Practices focused on the self-management aspects of the Bundle - in one practice over 90% of their patients agreed to take part in self-management – whereas others were less engaged in this element.

On the basis of our data it is not possible to distill the effects of the Bundle per se against the influence more generally within practices of a greater exposure to kidney disease. A number of factors outside the intervention and the project team will also have important impacts on performance, including pay for performance targets, Practice philosophy and the emphasis placed on education and decision-making, and nature of the consultation between individual health care professionals and the patient.

Participation

General practice as a setting for quality improvement projects presents unique challenges. Practices are small businesses and making the financial case for participation at the Practice level is important. Buy-in from a number of stakeholders was required for participation, typically a GP or Practice nurse “champion” within the Practice, GP partners, health care assistants, and practice managers. In some Practices the time taken from first expression of interest to delivery of the training programme was as long as four months. Even when the key people were enthusiastic, there was a significant amount of reinforcement and support required to embed the bundle. We also found that routine interruptions (e.g. Quality and Outcome Framework scheme pay for performance data returns, influenza vaccinations, staff changes, and sickness) distracted focus from the project.

Sustainability

A significant number of Practices did not complete the project. Stated reasons for this included a view that the ‘project changed as it went on’, the increased administrative burden of the extraction of evaluation data, a perceived need to recall patients earlier than would otherwise be necessary, issues around staffing in healthcare professionals and administrative staff, and the rapidly changing environment of the NHS/General Practice. However the differing levels of engagement across Practices does replicate that suggested by Rogers (17) and other studies of improvement science (18). Collecting monthly Practice-level data from Practices was particularly challenging and required a significant level of input and support from the project team. The team found that there was a wide range of capability amongst Practices with regard to their ability to manipulate the practice computer system.

Benefits of Involvement to PSUAG team members

The team was very focused on the meaningful involvement of patient and service user representatives through the PSUAG. An introductory paper, project overview, and role description were prepared by the chair of the PSUAG to ensure that group members had a common understanding of the project and its goals. The core project group placed great emphasis upon a flat team structure and inclusivity.

The PSUAG was involved in a number of crucial activities throughout the project. PSUAG members provided personal and anecdotal evidence about their own experiences which were used when talking to healthcare professionals who were considering joining the study and thereby supported recruitment. Patients and service users played a central role in the delivery of self-management training and workshops to Practices and subsequent support at teleconferences with Practices. PSUAG members also developed and delivered the self-management training to patient groups.

We have demonstrated that a care bundle can be reliably implemented in primary care. However maximising exposure to the intervention in patients seen infrequently can be difficult, particularly outside the setting of a dedicated long term clinic. This in turn makes it difficult to evaluate the impact of the Bundle on care. The effect on biomedical outcomes at Practice level has been variable, and will have been influenced by factors both within and outside the study, such as the engagement of practice staff, the model for delivery of CKD care, and the Quality and Outcomes Framework cycle.

For a quality improvement intervention in primary care to be successful, it is important to have dedicated training on QI science prior to the start of the project, regular coaching from a QI team (from within or outside the organisation), service user (patient) involvement throughout, and buy-in from the senior management team such as GP partners and Practice managers.

We believe that a care bundle is a feasible vehicle for quality improvement in primary care, and that the lessons learned here may usefully inform further projects. Where there is a clear financial case for participation, test practices with a high degree of IT capability, and a patient population that is seen regularly in a defined setting, such an approach could facilitate the delivery of more consistent care. In kidney disease this might include a group at particularly high risk of adverse events (for example those with declining kidney function and heavy proteinuria, who can be identified using electronic searches) who are seen in a dedicated clinic. Any future Bundle might include a safety element as patients with CKD are at increased risk from poor prescribing. The involvement of patients on an equal footing from the outset will be a critical factor for success.

All materials from the project can be found at

https://www.kidneyresearchuk.org/health-information/resources/package-of-innovation

Health and Social Care Information Centre 2012; Quality and Outcomes Framework achievement data 2011/12 ( http://www.qof.ic.nhs.uk/

Gallagher H, de Lusignan S, Harris K, Cates C. Quality-improvement strategies for the management of hypertension in chronic kidney disease in primary care: a systematic review. British Journal of General Practice 2010; Jun;60(575):e258-65.

de Lusignan S, Gallagher H, Jones S, Chan T, van Vlymen J, Tahir A, et al. Audit-based education lowers systolic blood pressure in chronic kidney disease: the Quality Improvement in chronic kidney disease (QICKD) trial results. Kidney International 2010; 84:609-620

Constantini, L et al The self-management experience of people with mild to moderate chronic kidney disease. Nephrology Nursing Journal 2008; Mar-Apr;35(2):147-55;

Health and Social Care Information Centre 2009. Health Survey for England, Volume 1: Health and Lifestyles ( https://catalogue.ic.nhs.uk/publications/public-health/surveys/heal-surv-heal-life-eng-2009/heal-surv-heal-life-eng-2009-rep-v2.pdf )

de Lusignan, S., Tomson, C.R., Harris, K., Van Vlymen, J., & Gallagher, H. UK prevalence of chronic kidney disease for the adult population is 6.76% based on two creatinine readings. Nephron Clinical Practice 2012; 120, c107 7.

Matsushita, K., van, d., V, Astor, B.C., Woodward, M., Levey, A.S., de Jong, P.E., Coresh, J., & Gansevoort, R.T. Association of estimated glomerular filtration rate and albuminuria with all-cause and cardiovascular mortality in general population cohorts: a collaborative meta-analysis. Lancet 2010; 375, (9731) 2073-2081

Gansevoort, R.T., Matsushita, K., van, d., V, Astor, B.C., Woodward, M., Levey, A.S., de Jong, P.E., & Coresh, J. 2011. Lower estimated GFR and higher albuminuria are associated with adverse kidney outcomes. A collaborative metaanalysis of general and high-risk population cohorts. Kidney Int; 80, (1) 93-104

National Institute for Health and Clinical Excellence. Early identification and management of chronic kidney disease in adults in primary and secondary care. NICE clinical guideline 73, 2008.

James MT, Hemmelgarn BR, Tonelli M. Early recognition and prevention of chronic kidney disease. Lancet 2010; 375(9722):1296-309.

Klebe B, Irving J, Stevens PE, O'Donoghue DJ, de Lusignan S, Cooley R, Hobbs H, Lamb EJ, John I, Middleton R, New J, Farmer CK. The cost of implementing UK guidelines for the management of chronic kidney disease. Nephrol Dial Transplant. 2007; 22(9):2504-12.

Resar R, Pronovost P, Haraden C, Simmonds T, Rainey T, Nolan T. Using a bundle approach to improve ventilator care processes and reduce ventilator-associated pneumonia. Joint Commission Journal on Quality and Patient Safety 2005; 31(5):243-48.

Guerin K, Wagner J, Rains K, Bessesen M. Reduction in central line-associated bloodstream infections by implementation of a post-insertion care bundle. American Journal of Infection Control 2010; 38(6):430-33.

Thomas N. Can care bundles reduce variability in standards in primary care? British Journal Renal Medicine 2011; 16(1):19-22.

Thomas, N., Bryar, R. & Makanjuola, D. Development of a self-management package for people with diabetes at risk of chronic kidney disease (CKD). Journal of Renal Care, 2008; 34(3), 151-8.

Abdi, Z., Gallagher, H and O’Donoghue, D. Telling the truth: why disclosure matters in chronic kidney disease. Br J Gen Pract. Apr 2012; 62(597): 172–173.

Rogers E. Diffusion of Innovations. 4th ed. New York: Free Press, 2005.

Brandrud AS, Schreiner A, Hjortdahl P, Helljesen G, Nyen B, Nelson EC. Three success factors for continual improvement in healthcare: an analysis of the reports of improvement team members. BMJ Quality & Safety 2011; 20(3):251-59.

Declaration of interests

HG is a member of the NICE Clinical Guideline Development Group (2012-2014) which is updating 2008 guidance on CKD. NT was a deputy on the NICE Clinical Guideline Development Group (2008).

Acknowledgements

All participating Practices and their patients, the Welsh Renal Network and the Primary Care Research Network.

Funding for the project was awarded by The Health Foundation under the ‘Closing the Gap through Clinical Communities’ award.

Read the full text or download the PDF:

Home > ETDs > DNP Projects

Doctor of Nursing Practice (DNP) Projects

Dnp projects from 2023 2023.

Implementing an African American Cultural Awareness Program for Skilled Nursing Facility Staff , ADEKEMI T. ADEDIPE

Community Health Worker Certification Program - Motivational Interview, Advocacy, Communication and Conflict Resolution , Stella Antony

Predicting the Risk of Falling with Artificial Intelligence , Ann Aquino

Mindfulness Based Self Care Toolkit for Psychiatric Healthcare Staff , Harkirat K. Bajwa

Mindfulness Based Self Care Toolkit for Psychiatric Healthcare Staff , Harkirat Kaur Bajwa

Trauma-Informed Treatment for Alcohol Use Disorder: Improving Long-Term Recovery , Aline M. Bales

Improving Patient Safety for Surgical Clearance: A PreOp One Stop Shop , Anna Benedictos

Using Critical Incident Debriefing after Code Blue Events to Support Registered Nurses , David L. Boyd

Improving New Nurse Manager Orientation and Onboarding Program , Leanne E. Deegenaars

Effects of Leadership Education and Mentoring for Assistant Nurse Managers , Karen T. Descent

Promoting Parent-Child Relationships Through Community-Based Family Wellness Programs to Improve Mental Health in Children , Maria Elena Falcon

Supporting Mental Health in Children by Providing Basic Skills and Knowledge of Mental Health to Middle-School Teachers , MariaElena Falcon

Reducing 30-day Heart Failure Hospital Readmissions Through the Implementation of a Telehealth Education and Screening Program , Madison Geib, Jo Loomis, and Maria (Dupi) Gomez Cogan

Responding to a Deteriorating Patient: An Educational Intervention for Outpatient Clinic Nurses , Angela N. Ikeme

Screening for Uterine Fibroids in Black Women , Winnie N. Kagendo

School-Based Development and Implementation of Adolescent Mental Health Educational Toolkits , Chantel M. Kilford Ms

School-Based Development and Implementation of an Educational Toolkit , Chantel M. Kilford Ms

Assuring a Continuum of Care for Heart Failure Patients Through Post-Acute Care Collaboration , Purnima Krishna

Save the Children Community Health Worker Program – Adverse Childhood Events Prevention , Jieun Lee

Educating Community Health Workers and Gatekeepers to Screen for Suicide , Jennifer Maina

The importance of suicide screening training for healthcare providers as part of suicide prevention , Jennifer Maina

Mental Health Services: Reaching the Homeless , Gurdeep Mann

Improving Outcomes in Persons Who Inject Drugs: A Multidisciplinary Healthcare Initiative , Carla S. Martin

Implementation of a Community Health Worker (CHW) training program to improve birth outcomes , Emiko Maruri

Beyond the Physical Wounds: A Proactive Approach to Mental Health Recovery After a Traumatic Injury , Christine McGahey

Environmental and Occupational Health Risks: Educating Undergraduate (ADN-BSN) Nursing Students for Safer Practice , Dorinda A. Mercado

Enhancing Mental Health Assessments for Hysterectomy Patients: Implementing a Depression Screening Tool for Healthcare Clinicians , Dede Mihedji

Depression Screening Tool for Hysterectomy Patients , Dede C. Mihedji

Developing and Implementing an Infection Prevention Practice Fellowship Program , Sejal Naik

Implementing Community Health Workers to Improve the Management of Chronic Non-Communicable Diseases in Children , Julia L. Newell

Implementing an Infant-Driven Feeding Practice Model , Ann P. Nguyen

Reducing 30-Day Psychiatric Inpatient Hospital Readmission of Mentally Ill Homeless Men with Substance Use Disorder by Using a Discharge Checklist , Chibuogwu E. Ogbuka

A Quality Improvement Project on Anxiety Management Before Ketamine-Assisted Psychotherapy , Kiera Paulo

Enhancing Nurse Manager Resilience with a Resilience-Enhancing Toolkit , Rosarina A. J. Pelikan

Strategies to Reduce Chronic Disease: Adding Role to Community Health Workers , Ingrid J. Renwanz DNP, PMHFNP-BC/APRN, MSN, RNC

Community Health Worker Program: Adverse Childhood Experiences and Early Childhood , Arelis Sandoval

Implicit Bias in Nursing: Raising Awareness, Fostering Change , Jeanette C. Sandoval

Determining Care Delivery Model Feasibility Using Discrete-Event-Simulation , Tanya Scott

Implementation of Implicit Bias Training in a Doctor of Nurse Practitioner Program , MacDana Selecon

Improving Health Outcomes for LGBTQ+ Youth Through Provider Education , Marissa Simko

Implementation of Fast-Track Triage Process to Improve Ambulance Patient Offloading Time (APOT) in a Psychiatric Emergency Services (PES) Unit , OKE AUGUSTINE UMEUGOJI

Improving Health Outcomes Among Patients with CHF Through Implementation of Telehealth Depression Screening and CHF Symptom Management , Akudo Udodiri Unanwa

The Dedicated Education Unit: Increasing Capacity and Confidence , Heather Marie Van Housen

DNP Projects from 2022 2022

Save the Children Community Health Worker Program—Project Management , Lauren Loree Burchfield

Increasing vaccination rates in the Latin X communities through a public health initiative for increasing education and vaccination , Gabriela Carrico

Educating Providers Without Prescriptive Authority About Psychopharmacology Basics: Bridging the Interdisciplinary Work Among Psychotropic Prescribers and Psychologists , Olivia Ceja

Psychosocial Distress Screening for Patients with Cancer: A Value-Based Approach to the Integration and Delivery of Holistic Care , Candy Cheung

Engaging Rural Providers About the Potential of Low-Earth Orbit Internet Satellite Supported Rural Telehealth Programs , Joshua M. Cleary

Development and Implementation of a Standardized Protocol for Nurse Practitioners Working in an AUD Treatment Telehealth Program , CJ Dea

Oleoresin Capsicum Gel: Enhancing Safety for the Home Health Clinician , Lilly Dickens

Diabetes Prevention and Management Education for Punjabi-Sikh Older Adults , Mandeep K. Gill

Improving Home Health Nurses' Knowledge of Heart Failure Self-Care Management , Nenette Hoffman

Improving Colorectal Cancer Screening in Primary Care , Bushra iqbal

Improving Colorectal Cancer Screening in Primary Care , Bushra Iqbal

Community Health Worker Program - Substance Use Disorder in Pregnancy , Alicia Claire Kletter DNP

Improving Diabetes Self-Management (DSM) Among Patients with Uncontrolled Type 2 Diabetes Mellitus (T2DM): A Patient-Centered Education Model , Maria Hannah Tiangha Mandecote

Save the Children® Community Health Worker Program - Prenatal Care , Deirdre A. Martinez

MITIGATE Toolkit for Outpatient Antibiotic Stewardship: Enhancing Safe Antibiotic Prescribing Practices , Allyssa Marie Montemayor and Allyssa Marie Montemayor

Implementing a Diabetic Foot Care Program in a Virtual Primary Care Clinic , Zahra Naderi Asiabar

Decreasing Seclusion and Restraint Events Among Clients Within an Inpatient and Crisis Stabilization Behavioral Health Facility , Karen Lee Richards

Utilization of the Nurse Practitioner Skillset in Partnership with the Medical Device Industry , Daniel C. Rowland

Integrating Culturally Tailored Interventions to Increase Cervical Cancer Screening: A Quality Improvement Initiative in a Primary Care Clinic , Marife Centeno Solomon

Advancing Treatment for Opioid Use Disorder in a Rural Emergency Department , Jill M. Wabbel

DNP Projects from 2021 2021

Technology-Based Advance Care Planning Education for Primary Care Patients , Taryn Achong

Development and Implementation of an Evidence-Based Practice Mentor Fellowship , Stacy L. Alves

Promoting Antibiotic Stewardship in Primary Care Setting: An Evaluation Process of Treatment Practices for Uncomplicated Urinary Tract Infections in Premenopausal Non-Pregnant Women , Ira Amayun

Manager Onboarding to Improve Retention, Knowledge, and Confidence to Lead , Christine Asiimwe

Manager Onboarding to Improve Knowledge and Confidence to Lead , Christine (Tina) K. Asiimwe

Designing and Implementing a De-Escalation Toolkit to Improve Staff Education and Competency on De-Escalation within a Mental Health Outpatient Setting , Balraj Bajwa

Increasing Chlamydia Screening in the High-Risk Population using Electronic Notification and Targeted Education: A Primary Care Approach , Michael Barnett

Transcatheter Aortic Valve Replacement: Implementation of Shared Decision-Making , LIESEL BUCHNER

Mindfulness-based Cognitive Therapy Staff Training Program to Tackle Postpartum Depression , Helen XUMIN Chen

Implementation of an Acute Care for the Elderly (ACE) Unit in a Community Hospital , Kimberly A. Colonnelli

Bullying in the C-Suite: A Nurse Leader Perspective , Kimberly Ann Colonnelli

Educating Providers to Screen for Post-Traumatic Stress Disorder , Alyssa Fraino

Creating a Pipeline to Increase Diversity in Executive Leadership Positions , Trarina Harris

Developing a Culturally Relevant Mental Health Assessment for Persons of African Descent , Roberta Iyamu

A Peer Feedback Process to Improve Assistant Nurse Manager Job Satisfaction , Juanita A. Jularbal-Walton

Development of an eHealth Harm Reduction Resource to Address Opioid Use Among Youth in Rural Northern California , Carolyn Perrotti Lake

Implementation of a Mental Health Environmental Risk Assessment Tool , Alan Le

Purposeful Nurse Hourly Rounding: A Plan To Decrease Patient Falls During a Pandemic , Robbie Masangkay

Purposeful Nurse Hourly Rounding: Plan to Decrease Patient Falls During a Pandemic , ROBBIE MASANGKAY

Regulatory Orientation to Support Transition to Academic Leadership , Annmarie Munana

Preparing Nurses in Management Positions for Bedside Care During Times of Crisis , Trevor Murray

Utilizing Digital Technology to Address Loneliness and Isolation in Older Adults through a Community-Based Connection Model: Responding to a Pandemic , Melissa A. Nagel

The Nurse Will See You Now: Improving Nurse-Led Chemotherapy Teaching , Laura Oka

PROVIDING HYPERTENSION EDUCATION FOR AFRICAN IMMIGRANTS AT A SOUTHERN CALIFORNIA CONGREGATION DURING THE COVID-19 PANDEMIC , Esther Oshunluyi

PROVIDING HYPERTENSION EDUCATION TO AFRICAN IMMIGRANTS AT A SOUTHERN CALIFORNIA CONGREGATION DURING THE COVID-19 PANDEMIC , Esther Oshunluyi

Increasing Continuous Positive Airway Pressure Compliance with Various Telemedicine Interventions: An Integrated Literature Review , Tracy Partington

Telehealth Appointments to Increase Therapy Compliance in Patients with MOSA , Tracy J. Partington

Behavioral Health Patients and Agitation in the Emergency Department: A Synthesis of Literature , Adam Pelzl

Implementation of an Agitation Scale in Three Emergency Departments in an Urban Area of Northern California , Adam Pelzl

Targeting Pre-Operative Booking Processes to Decrease Risks of "Never Events" , Rebecca Lynn Poths and Rebecca Lynn Poths

Building Excellence through Shared Governance and Continuous Process Improvement , Deborah Reitter

Streamlining Care for Children with Autism Spectrum Disorder , Siavash Rostami Jafarabad

Nurse Practitioner-Led Care Pods: A Team Communication Enhancement Model , Miranda Saint-Louis and Miranda Saint Louis

Use of Simulation to Improve Nurse's Response to COVID-19 Code Blue: A Literature Review , Charity Shelton

Using Simulation to Improve COVID-19 Code Blue Outcomes , Charity Shelton

Utilizing Group Prenatal Care to Support Underserved Pregnant Women , Kathleen Shrader

Advanced Search

• Notify me via email or RSS
• Collections
• Disciplines

Author Corner

• Submit Research
• Nursing and Health Professions Submission Guide

Library Links

• Gleeson Library
• Zief Law Library

Home | About | FAQ | My Account | Accessibility Statement

Privacy Copyright

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Quality improvement...

Quality improvement into practice

Read the full collection.

• Related content
• Peer review
• Adam Backhouse , quality improvement programme lead 1 ,
• Fatai Ogunlayi , public health specialty registrar 2
• 1 North London Partners in Health and Care, Islington CCG, London N1 1TH, UK
• 2 Institute of Applied Health Research, Public Health, University of Birmingham, B15 2TT, UK
• Correspondence to: A Backhouse adam.backhouse{at}nhs.net

What you need to know

Thinking of quality improvement (QI) as a principle-based approach to change provides greater clarity about ( a ) the contribution QI offers to staff and patients, ( b ) how to differentiate it from other approaches, ( c ) the benefits of using QI together with other change approaches

QI is not a silver bullet for all changes required in healthcare: it has great potential to be used together with other change approaches, either concurrently (using audit to inform iterative tests of change) or consecutively (using QI to adapt published research to local context)

As QI becomes established, opportunities for these collaborations will grow, to the benefit of patients.

The benefits to front line clinicians of participating in quality improvement (QI) activity are promoted in many health systems. QI can represent a valuable opportunity for individuals to be involved in leading and delivering change, from improving individual patient care to transforming services across complex health and care systems. 1

However, it is not clear that this promotion of QI has created greater understanding of QI or widespread adoption. QI largely remains an activity undertaken by experts and early adopters, often in isolation from their peers. 2 There is a danger of a widening gap between this group and the majority of healthcare professionals.

This article will make it easier for those new to QI to understand what it is, where it fits with other approaches to improving care (such as audit or research), when best to use a QI approach, making it easier to understand the relevance and usefulness of QI in delivering better outcomes for patients.

How this article was made

AB and FO are both specialist quality improvement practitioners and have developed their expertise working in QI roles for a variety of UK healthcare organisations. The analysis presented here arose from AB and FO’s observations of the challenges faced when introducing QI, with healthcare providers often unable to distinguish between QI and other change approaches, making it difficult to understand what QI can do for them.

How is quality improvement defined?

There are many definitions of QI ( box 1 ). The BMJ ’s Quality Improvement series uses the Academy of Medical Royal Colleges definition. 6 Rather than viewing QI as a single method or set of tools, it can be more helpful to think of QI as based on a set of principles common to many of these definitions: a systematic continuous approach that aims to solve problems in healthcare, improve service provision, and ultimately provide better outcomes for patients.

Definitions of quality improvement

Improvement in patient outcomes, system performance, and professional development that results from a combined, multidisciplinary approach in how change is delivered. 3

The delivery of healthcare with improved outcomes and lower cost through continuous redesigning of work processes and systems. 4

Using a systematic change method and strategies to improve patient experience and outcome. 5

To make a difference to patients by improving safety, effectiveness, and experience of care by using understanding of our complex healthcare environment, applying a systematic approach, and designing, testing, and implementing changes using real time measurement for improvement. 6

In this article we discuss QI as an approach to improving healthcare that follows the principles outlined in box 2 ; this may be a useful reference to consider how particular methods or tools could be used as part of a QI approach.

Principles of QI

Primary intent— To bring about measurable improvement to a specific aspect of healthcare delivery, often with evidence or theory of what might work but requiring local iterative testing to find the best solution. 7

Employing an iterative process of testing change ideas— Adopting a theory of change which emphasises a continuous process of planning and testing changes, studying and learning from comparing the results to a predicted outcome, and adapting hypotheses in response to results of previous tests. 8 9

Consistent use of an agreed methodology— Many different QI methodologies are available; commonly cited methodologies include the Model for Improvement, Lean, Six Sigma, and Experience-based Co-design. 4 Systematic review shows that the choice of tools or methodologies has little impact on the success of QI provided that the chosen methodology is followed consistently. 10 Though there is no formal agreement on what constitutes a QI tool, it would include activities such as process mapping that can be used within a range of QI methodological approaches. NHS Scotland’s Quality Improvement Hub has a glossary of commonly used tools in QI. 11

Empowerment of front line staff and service users— QI work should engage staff and patients by providing them with the opportunity and skills to contribute to improvement work. Recognition of this need often manifests in drives from senior leadership or management to build QI capability in healthcare organisations, but it also requires that frontline staff and service users feel able to make use of these skills and take ownership of improvement work. 12

Using data to drive improvement— To drive decision making by measuring the impact of tests of change over time and understanding variation in processes and outcomes. Measurement for improvement typically prioritises this narrative approach over concerns around exactness and completeness of data. 13 14

Scale-up and spread, with adaptation to context— As interventions tested using a QI approach are scaled up and the degree of belief in their efficacy increases, it is desirable that they spread outward and be adopted by others. Key to successful diffusion of improvement is the adaption of interventions to new environments, patient and staff groups, available resources, and even personal preferences of healthcare providers in surrounding areas, again using an iterative testing approach. 15 16

What other approaches to improving healthcare are there?

Taking considered action to change healthcare for the better is not new, but QI as a distinct approach to improving healthcare is a relatively recent development. There are many well established approaches to evaluating and making changes to healthcare services in use, and QI will only be adopted more widely if it offers a new perspective or an advantage over other approaches in certain situations.

A non-systematic literature scan identified the following other approaches for making change in healthcare: research, clinical audit, service evaluation, and clinical transformation. We also identified innovation as an important catalyst for change, but we did not consider it an approach to evaluating and changing healthcare services so much as a catch-all term for describing the development and introduction of new ideas into the system. A summary of the different approaches and their definition is shown in box 3 . Many have elements in common with QI, but there are important difference in both intent and application. To be useful to clinicians and managers, QI must find a role within healthcare that complements research, audit, service evaluation, and clinical transformation while retaining the core principles that differentiate it from these approaches.

Alternatives to QI

Research— The attempt to derive generalisable new knowledge by addressing clearly defined questions with systematic and rigorous methods. 17

Clinical audit— A way to find out if healthcare is being provided in line with standards and to let care providers and patients know where their service is doing well, and where there could be improvements. 18

Service evaluation— A process of investigating the effectiveness or efficiency of a service with the purpose of generating information for local decision making about the service. 19

Clinical transformation— An umbrella term for more radical approaches to change; a deliberate, planned process to make dramatic and irreversible changes to how care is delivered. 20

Innovation— To develop and deliver new or improved health policies, systems, products and technologies, and services and delivery methods that improve people’s health. Health innovation responds to unmet needs by employing new ways of thinking and working. 21

Why do we need to make this distinction for QI to succeed?

Improvement in healthcare is 20% technical and 80% human. 22 Essential to that 80% is clear communication, clarity of approach, and a common language. Without this shared understanding of QI as a distinct approach to change, QI work risks straying from the core principles outlined above, making it less likely to succeed. If practitioners cannot communicate clearly with their colleagues about the key principles and differences of a QI approach, there will be mismatched expectations about what QI is and how it is used, lowering the chance that QI work will be effective in improving outcomes for patients. 23

There is also a risk that the language of QI is adopted to describe change efforts regardless of their fidelity to a QI approach, either due to a lack of understanding of QI or a lack of intention to carry it out consistently. 9 Poor fidelity to the core principles of QI reduces its effectiveness and makes its desired outcome less likely, leading to wasted effort by participants and decreasing its credibility. 2 8 24 This in turn further widens the gap between advocates of QI and those inclined to scepticism, and may lead to missed opportunities to use QI more widely, consequently leading to variation in the quality of patient care.

Without articulating the differences between QI and other approaches, there is a risk of not being able to identify where a QI approach can best add value. Conversely, we might be tempted to see QI as a “silver bullet” for every healthcare challenge when a different approach may be more effective. In reality it is not clear that QI will be fit for purpose in tackling all of the wicked problems of healthcare delivery and we must be able to identify the right tool for the job in each situation. 25 Finally, while different approaches will be better suited to different types of challenge, not having a clear understanding of how approaches differ and complement each other may mean missed opportunities for multi-pronged approaches to improving care.

What is the relationship between QI and other approaches such as audit?

Academic journals, healthcare providers, and “arms-length bodies” have made various attempts to distinguish between the different approaches to improving healthcare. 19 26 27 28 However, most comparisons do not include QI or compare QI to only one or two of the other approaches. 7 29 30 31 To make it easier for people to use QI approaches effectively and appropriately, we summarise the similarities, differences, and crossover between QI and other approaches to tackling healthcare challenges ( fig 1 ).

How quality improvement interacts with other approaches to improving healthcare

• Download figure
• Open in new tab
• Download powerpoint

QI and research

Research aims to generate new generalisable knowledge, while QI typically involves a combination of generating new knowledge or implementing existing knowledge within a specific setting. 32 Unlike research, including pragmatic research designed to test effectiveness of interventions in real life, QI does not aim to provide generalisable knowledge. In common with QI, research requires a consistent methodology. This method is typically used, however, to prove or disprove a fixed hypothesis rather than the adaptive hypotheses developed through the iterative testing of ideas typical of QI. Both research and QI are interested in the environment where work is conducted, though with different intentions: research aims to eliminate or at least reduce the impact of many variables to create generalisable knowledge, whereas QI seeks to understand what works best in a given context. The rigour of data collection and analysis required for research is much higher; in QI a criterion of “good enough” is often applied.

Relationship with QI

Though the goal of clinical research is to develop new knowledge that will lead to changes in practice, much has been written on the lag time between publication of research evidence and system-wide adoption, leading to delays in patients benefitting from new treatments or interventions. 33 QI offers a way to iteratively test the conditions required to adapt published research findings to the local context of individual healthcare providers, generating new knowledge in the process. Areas with little existing knowledge requiring further research may be identified during improvement activities, which in turn can form research questions for further study. QI and research also intersect in the field of improvement science, the academic study of QI methods which seeks to ensure QI is carried out as effectively as possible. 34

Scenario: QI for translational research

Newly published research shows that a particular physiotherapy intervention is more clinically effective when delivered in short, twice-daily bursts rather than longer, less frequent sessions. A team of hospital physiotherapists wish to implement the change but are unclear how they will manage the shift in workload and how they should introduce this potentially disruptive change to staff and to patients.

Before continuing reading think about your own practice— How would you approach this situation, and how would you use the QI principles described in this article?

Adopting a QI approach, the team realise that, although the change they want to make is already determined, the way in which it is introduced and adapted to their wards is for them to decide. They take time to explain the benefits of the change to colleagues and their current patients, and ask patients how they would best like to receive their extra physiotherapy sessions.

The change is planned and tested for two weeks with one physiotherapist working with a small number of patients. Data are collected each day, including reasons why sessions were missed or refused. The team review the data each day and make iterative changes to the physiotherapist’s schedule, and to the times of day the sessions are offered to patients. Once an improvement is seen, this new way of working is scaled up to all of the patients on the ward.

The findings of the work are fed into a service evaluation of physiotherapy provision across the hospital, which uses the findings of the QI work to make recommendations about how physiotherapy provision should be structured in the future. People feel more positive about the change because they know colleagues who have already made it work in practice.

QI and clinical audit

Clinical audit is closely related to QI: it is often used with the intention of iteratively improving the standard of healthcare, albeit in relation to a pre-determined standard of best practice. 35 When used iteratively, interspersed with improvement action, the clinical audit cycle adheres to many of the principles of QI. However, in practice clinical audit is often used by healthcare organisations as an assurance function, making it less likely to be carried out with a focus on empowering staff and service users to make changes to practice. 36 Furthermore, academic reviews of audit programmes have shown audit to be an ineffective approach to improving quality due to a focus on data collection and analysis without a well developed approach to the action section of the audit cycle. 37 Clinical audits, such as the National Clinical Audit Programme in the UK (NCAPOP), often focus on the management of specific clinical conditions. QI can focus on any part of service delivery and can take a more cross-cutting view which may identify issues and solutions that benefit multiple patient groups and pathways. 30

Audit is often the first step in a QI process and is used to identify improvement opportunities, particularly where compliance with known standards for high quality patient care needs to be improved. Audit can be used to establish a baseline and to analyse the impact of tests of change against the baseline. Also, once an improvement project is under way, audit may form part of rapid cycle evaluation, during the iterative testing phase, to understand the impact of the idea being tested. Regular clinical audit may be a useful assurance tool to help track whether improvements have been sustained over time.

Scenario: Audit and QI

A foundation year 2 (FY2) doctor is asked to complete an audit of a pre-surgical pathway by looking retrospectively through patient documentation. She concludes that adherence to best practice is mixed and recommends: “Remind the team of the importance of being thorough in this respect and re-audit in 6 months.” The results are presented at an audit meeting, but a re-audit a year later by a new FY2 doctor shows similar results.

Before continuing reading think about your own practice— How would you approach this situation, and how would you use the QI principles described in this paper?

Contrast the above with a team-led, rapid cycle audit in which everyone contributes to collecting and reviewing data from the previous week, discussed at a regular team meeting. Though surgical patients are often transient, their experience of care and ideas for improvement are captured during discharge conversations. The team identify and test several iterative changes to care processes. They document and test these changes between audits, leading to sustainable change. Some of the surgeons involved work across multiple hospitals, and spread some of the improvements, with the audit tool, as they go.

QI and service evaluation

In practice, service evaluation is not subject to the same rigorous definition or governance as research or clinical audit, meaning that there are inconsistencies in the methodology for carrying it out. While the primary intent for QI is to make change that will drive improvement, the primary intent for evaluation is to assess the performance of current patient care. 38 Service evaluation may be carried out proactively to assess a service against its stated aims or to review the quality of patient care, or may be commissioned in response to serious patient harm or red flags about service performance. The purpose of service evaluation is to help local decision makers determine whether a service is fit for purpose and, if necessary, identify areas for improvement.

Service evaluation may be used to initiate QI activity by identifying opportunities for change that would benefit from a QI approach. It may also evaluate the impact of changes made using QI, either during the work or after completion to assess sustainability of improvements made. Though likely planned as separate activities, service evaluation and QI may overlap and inform each other as they both develop. Service evaluation may also make a judgment about a service’s readiness for change and identify any barriers to, or prerequisites for, carrying out QI.

QI and clinical transformation

Clinical transformation involves radical, dramatic, and irreversible change—the sort of change that cannot be achieved through continuous improvement alone. As with service evaluation, there is no consensus on what clinical transformation entails, and it may be best thought of as an umbrella term for the large scale reform or redesign of clinical services and the non-clinical services that support them. 20 39 While it is possible to carry out transformation activity that uses elements of QI approach, such as effective engagement of the staff and patients involved, QI which rests on iterative test of change cannot have a transformational approach—that is, one-off, irreversible change.

There is opportunity to use QI to identify and test ideas before full scale clinical transformation is implemented. This has the benefit of engaging staff and patients in the clinical transformation process and increasing the degree of belief that clinical transformation will be effective or beneficial. Transformation activity, once completed, could be followed up with QI activity to drive continuous improvement of the new process or allow adaption of new ways of working. As interventions made using QI are scaled up and spread, the line between QI and transformation may seem to blur. The shift from QI to transformation occurs when the intention of the work shifts away from continuous testing and adaptation into the wholesale implementation of an agreed solution.

Scenario: QI and clinical transformation

An NHS trust’s human resources (HR) team is struggling to manage its junior doctor placements, rotas, and on-call duties, which is causing tension and has led to concern about medical cover and patient safety out of hours. A neighbouring trust has launched a smartphone app that supports clinicians and HR colleagues to manage these processes with the great success.

This problem feels ripe for a transformation approach—to launch the app across the trust, confident that it will solve the trust’s problems.

Before continuing reading think about your own organisation— What do you think will happen, and how would you use the QI principles described in this article for this situation?

Outcome without QI

Unfortunately, the HR team haven’t taken the time to understand the underlying problems with their current system, which revolve around poor communication and clarity from the HR team, based on not knowing who to contact and being unable to answer questions. HR assume that because the app has been a success elsewhere, it will work here as well.

People get excited about the new app and the benefits it will bring, but no consideration is given to the processes and relationships that need to be in place to make it work. The app is launched with a high profile campaign and adoption is high, but the same issues continue. The HR team are confused as to why things didn’t work.

Outcome with QI

Although the app has worked elsewhere, rolling it out without adapting it to local context is a risk – one which application of QI principles can mitigate.

HR pilot the app in a volunteer specialty after spending time speaking to clinicians to better understand their needs. They carry out several tests of change, ironing out issues with the process as they go, using issues logged and clinician feedback as a source of data. When they are confident the app works for them, they expand out to a directorate, a division, and finally the transformational step of an organisation-wide rollout can be taken.

Education into practice

Next time when faced with what looks like a quality improvement (QI) opportunity, consider asking:

How do you know that QI is the best approach to this situation? What else might be appropriate?

Have you considered how to ensure you implement QI according to the principles described above?

Is there opportunity to use other approaches in tandem with QI for a more effective result?

How patients were involved in the creation of this article

This article was conceived and developed in response to conversations with clinicians and patients working together on co-produced quality improvement and research projects in a large UK hospital. The first iteration of the article was reviewed by an expert patient, and, in response to their feedback, we have sought to make clearer the link between understanding the issues raised and better patient care.

Contributors: This work was initially conceived by AB. AB and FO were responsible for the research and drafting of the article. AB is the guarantor of the article.

Competing interests: We have read and understood BMJ policy on declaration of interests and have no relevant interests to declare.

Provenance and peer review: This article is part of a series commissioned by The BMJ based on ideas generated by a joint editorial group with members from the Health Foundation and The BMJ , including a patient/carer. The BMJ retained full editorial control over external peer review, editing, and publication. Open access fees and The BMJ ’s quality improvement editor post are funded by the Health Foundation.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ .

• Olsson-Brown A
• Dixon-Woods M ,
• Batalden PB ,
• Berwick D ,
• Øvretveit J
• Academy of Medical Royal Colleges
• Nelson WA ,
• McNicholas C ,
• Woodcock T ,
• Alderwick H ,
• ↵ NHS Scotland Quality Improvement Hub. Quality improvement glossary of terms. http://www.qihub.scot.nhs.uk/qi-basics/quality-improvement-glossary-of-terms.aspx .
• McNicol S ,
• Solberg LI ,
• Massoud MR ,
• Albrecht Y ,
• Illingworth J ,
• Department of Health
• ↵ NHS England. Clinical audit. https://www.england.nhs.uk/clinaudit/ .
• Healthcare Quality Improvement Partnership
• McKinsey Hospital Institute
• ↵ World Health Organization. WHO Health Innovation Group. 2019. https://www.who.int/life-course/about/who-health-innovation-group/en/ .
• Sheffield Microsystem Coaching Academy
• Davidoff F ,
• Leviton L ,
• Taylor MJ ,
• Nicolay C ,
• Tarrant C ,
• Twycross A ,
• ↵ University Hospitals Bristol NHS Foundation Trust. Is your study research, audit or service evaluation. http://www.uhbristol.nhs.uk/research-innovation/for-researchers/is-it-research,-audit-or-service-evaluation/ .
• ↵ University of Sheffield. Differentiating audit, service evaluation and research. 2006. https://www.sheffield.ac.uk/polopoly_fs/1.158539!/file/AuditorResearch.pdf .
• ↵ Royal College of Radiologists. Audit and quality improvement. https://www.rcr.ac.uk/clinical-radiology/audit-and-quality-improvement .
• Gundogan B ,
• Finkelstein JA ,
• Brickman AL ,
• Health Foundation
• Johnston G ,
• Crombie IK ,
• Davies HT ,
• Hillman T ,
• ↵ NHS Health Research Authority. Defining research. 2013. https://www.clahrc-eoe.nihr.ac.uk/wp-content/uploads/2014/04/defining-research.pdf .

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Glob Health Sci Pract
• v.11(1); 2023 Feb 28

The Quality Management Improvement Approach: Successes and Lessons Learned From a Workforce Development Intervention in Rwanda’s Health Supply Chain

Rogers kigenza.

a U.S. Agency for International, Development Global Health Supply Chain Program-Procurement and Supply Management, Kigali, Rwanda.

Eliezer Nsengiyumva

b Clinical and Public Health Governance Directorate, Ministry of Health of Rwanda, Kigali, Rwanda.

Vincent Sabagirirwa

A workforce development intervention called the Quality Management Improvement Approach has proven to be a successful training and capacity-building platform for supply chain management, improving end-to-end data visibility and communication.

Key Findings

• High staff turnover, limited training, and lack of capacity for supply chain activities contributed to challenges in the supply chain management system, including stock-outs and stock inaccuracies.
• The Quality Management Improvement Approach contributed to significant improvements in staff recording of commodity consumption, using the electronic logistics management information system, and keeping accurate inventory data, as well as reduced stock-outs for all types of commodities.
• The approach addressed performance gaps created by staff turnover, fostered an improvement in communication between supply chain workers at different levels, and facilitated developing local solutions to resolve issues and challenges in supply chain operation and management.

Key Implication

• This approach can be implemented in other settings having issues with end-to-end data visibility and with supply chain personnel operation and communication.

Background:

Rwanda’s supply chain is decentralized, with regional warehouse pharmacies and service delivery points (SDPs) ensuring a reliable health commodities supply. However, in 2016, the Rwanda Ministry of Health (MOH) identified challenges with the supply chain workforce contributing to a high stock-out rate, inventory stock inaccuracies, and underutilization of the electronic logistics management information system (e-LMIS). We aimed to document successes and lessons learned from implementing a workforce development intervention to address these challenges.

Program Description:

Rwanda’s MOH and the U.S. Agency for International Development Global Health Supply Chain Program-Procurement and Supply Management project implemented a workforce development intervention, the Quality Management Improvement Approach (QMIA), to improve and sustain Rwanda’s service delivery and medicines availability. The QMIA monitors the performance of supply chain professionals while continuously building their capacity to run a smooth supply chain operation. The QMIA includes supervisory visits to measure key performance indicators and discussion sessions for experience-sharing and skill-building. The central level conducts both components, supporting regional warehouses that in turn assist SDPs through supervisory visits and discussion sessions.

Since 2017, the QMIA has been conducted twice a year in 596 SDPs and 30 regional warehouses and has trained 1,296 supply chain staff members. The intervention has contributed to major improvements in several key supply chain outcomes (2017 versus 2019): increased utilization of the e-LMIS (55% to 96%), higher inventory data accuracy (25% to 85%), and reduced stock-outs (10% to 1%).

Conclusions:

The QMIA has proven to be a successful training and capacity-building platform for supply chain management in Rwanda, enhancing system utilization and availability of accurate data to guide decision-making. Notably, the approach fosters a positive behavioral change to build local solutions and improves communication.

Over the last few years, the Government of Rwanda has invested significantly in improving the country’s health care system, including the accessibility and availability of essential health commodities to its citizens. 1 As a result, quality control, storage capacity, and rational use of health commodities have greatly improved in the country. 2 However, the Rwanda Health Sector Strategic Plan 2018–2024 (HSSP IV) 2 still recognizes the limited capacity of supply chain management at different levels as a crucial challenge. Thus, strengthening the capacity in supply chain management systems, including storage conditions, human resources, financing, supply plans, and supply regulations, together with upgrading information and technology tools, constitute a national strategic direction.

Rwanda has a well-structured supply chain management system that is decentralized from the Ministry of Health (MOH). The MOH coordinates all the supply chain operations through the Logistics Management Office. Rwanda Food and Drug Authority regulates all the processes for pharmaceutical management to ensure quality and safety, using the Pharmaceutical Regulatory Information Management System to operate. Rwanda Biomedical Center promotes high-quality, affordable, and sustainable health care services to the population through evidence-based interventions and practices.

Currently, the country uses a “pull system” for all product categories where data moves in a reverse direction of the commodities ( Figure 1 ). 3 The supply chain system is managed by the Rwanda Medical Supply (RMS), a large-scale corporation created and owned by the Government of Rwanda to ensure the availability of medicines and health commodities and overcome past problems of low availability at the public central medical store (called the Medical Production and Procurement Division). 4

Commodity and Data Flow in Rwanda’s Health Supply Chain

Abbreviations: BUFMAR, Bureau des Formations Médicales Agréées du Rwanda; FDA, Food and Drug Authority; HQ, headquarters; Mediasol, Medical & Allied Service Solutions; RBC, Rwanda Biomedical Center; RMS, Rwanda Medical Supply.

The RMS, Bureau des Formations Médicales Agréées du Rwanda, Medical & Allied Service Solutions, and the private-sector stores operate as central medical stores to ensure the procurement and distribution of health commodities using the electronic logistics management information system (e-LMIS) and warehouse management system. Supply chain master data include facility master data and national product master data.

The RMS headquarters in Kigali is where senior management and a central warehouse are located. The central warehouse supplies the RMS branches, which are district-level warehouses that supply service delivery points (SDPs), including hospitals and health centers. The 30 RMS branches located in 30 districts (1 RMS branch per district) play a key role in storage and inventory management, transportation of commodities to health facilities, resupply, and support to and supervision of SDPs within their respective districts. Together with health posts and community health workers, the system ensures that medicines and supplies reach patients in a timely manner and in the right quantity, condition, and cost. RMS branches use the e-LMIS launched in 2014. 5 Therefore, any supply chain performance improvement at lower levels will depend on how well the RMS branches perform.

In 2016, the MOH conducted supply chain monitoring, training, and planning sessions in collaboration with the U.S. Agency for International Development Global Health Supply Chain Program-Procurement and Supply Management (GHSC-PSM) project. These sessions identified several challenges related to the supply chain workforce, such as high staff turnover (in some cases more than 30% due to excessive workloads, lack of training, or staff receiving better offers elsewhere), minimal time dedicated to supply chain activities at SDPs (e.g., nurses already having other work assignments), and lack of supply chain professionals at the SDP level, among others. 6 These challenges resulted in a high stock-out rate (10%), underutilization of the e-LMIS (55%), and inventory stock inaccuracies (25%), as reported by SDPs and district pharmacies. 6 The MOH and GHSC-PSM concluded that the key to successfully improving supply chain performance was to focus not only on areas that were underperforming but also on areas that were not aligned with the overall supply chain strategy. 6

Challenges with high staff turnover and lack of capacity for supply chain activities led to the need for a strategy to improve supply chain professionals’ performance.

GHSC-PSM offered procurement and technical assistance in Rwanda to help ensure uninterrupted availability of high-quality health commodities, focusing on those to prevent and treat HIV/AIDS; malaria; family planning; and maternal, neonatal, child, and community health. 7 In particular, since 2017, GHSC-PSM has supported the MOH in strengthening the last-mile capacity-building system using the Quality Management Improvement Approach (QMIA). This approach is a workforce development intervention the MOH uses to monitor the performance of supply chain professionals while continuously building their capacity to run a smooth supply chain operation.

We describe the successes and lessons learned from implementing the QMIA and offer recommendations for different levels of the supply chain management system.

IMPLEMENTING QMIA

Quality improvement practices have been helpful in improving supply chain operations and access to medicines, as previously shown by an intervention establishing resupply procedures and quality improvement teams in Malawi and Rwanda. 8 In Uganda, a strategy based on performance assessment and recognition effectively built supply chain management capacity in lower-level health care facilities and improved supply chain management. 9

Thus, to solve the identified challenges related to the supply chain workforce in Rwanda, the MOH and GHSC-PSM aimed to establish a workforce development intervention called the QMIA. They defined the QMIA concept for the public health supply chain in Rwanda as “a participative, continuous, patient-centered improvement approach that stresses total staff commitment to customer/patient satisfaction in terms of medicines availability and quality by creating and implementing process improvement.” 10 The approach is considered patient centered because it aims to ensure that patients get efficient service delivery in terms of time and quality. For example, patients provide feedback on time spent at hospitals and SDPs (before and after QMIA, although not a specific indicator of this activity).

QMIA Guiding Principles and Objectives

The QMIA is based on a set of guiding principles 10 ( Figure 2 ): people involvement, based on the principles of teamwork, empowerment, and accountability; built-in quality (“do it right the first time, eliminate rework”); standardization (documenting best practices and establishing standard operating procedures); just-in-time delivery (ensuring the “7 rights of supply chain management” using an established inventory flow and pull system for reordering of health commodities); and continuous improvement (“every aspect should be challenged to get better”). 10

Five Guiding Principles of the Quality Management Improvement Approach

Abbreviation: QMIA, Quality Management Improvement Approach.

The QMIA’s primary objective was to ensure continuous performance improvement and prevention of supply chain issues that may result in stock-outs, expiries, and overstocks at both RMS branches and SDPs ( Box 1 ).

Specific Objectives of the Quality Management Improvement Approach for Health Supply Chain Management in Rwanda

• Ensure an uninterrupted supply of health commodities at both Rwanda Medical Supply branches and service delivery points
• Encourage a proper working environment and empower the staff to motivate them to improve services to patients/customers and foster their will to innovate and act in an atmosphere of trust and respect
• Share experiential knowledge on problem solving and problem prevention with other facilities
• Use measurements to support decisions (using measurements, the staff can spot trends, make corrections before problems arise and, as part of problem solving, investigate why problems happened and what can be done to prevent them from happening again)
• Provide capacity-building and supervision of staff

Techniques and Tools Used for Process Improvement

To achieve its objectives, the QMIA is based on the Lean Six Sigma and the hybrid Lean Six Sigma methodologies, tailored for best-in-class supply executions, continuous improvement, and supply chain efficiency at all levels of the supply chain in Rwanda. The utility of applying these methods has been demonstrated in the context of health care logistics, adding value to the availability of health commodities. 11 – 14 By relying on a collaborative team effort, MOH and GHSC-PSM used the QMIA to apply these methodologies to improve supply chain performance by systematically removing waste and reducing variation.

Roles of Partners and Estimated Costs of the Approach

GHSC-PSM provided financial support to implement the activity. On average, the costs of implementing this approach were approximately US$300,000 per year during the first 2 years. Because the need for training workshops and continuous supervision has diminished in subsequent years, the costs have decreased to an estimated US$100,000–US$150,000 per year. The estimated cost for continuous implementation of the program is expected to be also around US$100,000–US$150,000 per year.

In addition, GHSC-PSM worked in conjunction with MOH staff to conduct field supervisory visits and data collection to evaluate improvements in service delivery at SDPs and RMS branches; analyzed and reported data for dissemination to the MOH; trained staff on QMIA at SDPs; mentored and advised staff at SDPs and RMS branches on areas that deserve improvement (using the information collected from previous visits); and participated in QMIA sessions together with MOH staff. With the overall oversight of the MOH, GHSC-PSM continues to provide technical assistance to build the necessary capacity and support the RMS branches to effectively supervise lower-level health facilities but also mentor them to measure and improve their performance. The participating staff are paid by their respective institutions and do not receive any economic incentive to participate.

QMIA Intervention Areas and Components

The QMIA includes continuous capacity-building through mentorship, supervision, and performance improvement and measurement. The intervention also provides on-the-job mentorship to health facility staff regarding the planning and implementing of improvements in supply chain components, such as warehousing and inventory control, resupply, storage, waste management, data management and recordkeeping using the e-LMIS, pest control, and transportation, among others. The QMIA sessions have covered various key functional and cross-cutting supply chain areas ( Box 2 ).

Intervention Areas of the Quality Management Improvement Approach for Health Supply Chain Management in Rwanda

Key Functional Areas

• Resupply (quantification and forecasting)
• Ordering (procurement)
• Warehousing and inventory management
• Distribution/transportation (Rwanda Medical Supply branches only)

Cross-Cutting Areas

• e-LMIS utilization
• Data analysis
• Data utilization for decision-making
• Financing of supply chain activities at Rwanda Medical Supply branches and service delivery points
• Human resources
• Communications and coordination (synergy between different services/departments and stakeholders)

The QMIA includes continuous capacity-building through mentorship, supervision, and performance improvement and measurement.

The QMIA has 2 components: periodic supervisory visits and discussion sessions. The central level, consisting of the MOH and central medical stores, performs these 2 components. Then, the RMS branches conduct supervisory visits and discussion sessions at SDPs with support from GHSC-PSM. Between 2017 and 2021, the intervention included 30 RMS branches and 596 SDPs.

Conducting QMIA Supervisory Visits

During their supervisory visits, government and GHSC-PSM staff (1) measure the supply chain performance using defined key performance indicators (KPIs) ( Table 1 ); (2) identify key performance gaps; (3) provide capacity-building in supply chain management of health commodities to health facilities; (4) monitor progress toward actions taken during the previous sessions to address gaps; and (5) ensure the correct use of management tools, including the e-LMIS. Supervisory visits are planned in advance, with precise logistical and technical instructions to be applied before the visit, on arrival at the site, and after the visit.

Key Performance Indicators Measured During Quality Management Improvement Approach Supervisory Visits in Rwanda a

Abbreviations: ACT, artemisinin-based combination therapy; e-LMIS, electronic logistics management information system; HMIS, health management information system.

a These indicators are then aggregated at district and national levels.

Conducting QMIA Discussion Sessions

After the visits, discussion sessions are held semiannually within the district by RMS branches with SDPs at identified locations. Discussion sessions aim to solve issues identified during supervisory visits and provide collaborative learning opportunities by sharing experiences, building skills in areas that need improvement, and setting targets. The sessions aim to: (1) monitor improvement recommendations for supply chain performance, (2) review the current performance, (3) train and conduct problem solving, and (4) provide recommendations for continuous improvement. For each selected supply chain functional area, the process follows several steps ( Table 2 ).

Objectives, Inputs, and Outputs of Steps in the Quality Management Improvement Approach Discussion Sessions in Rwanda

The QMIA has proven to be a successful training and capacity-building platform for supply chain management in Rwanda. The team for this activity is composed of 10 trainers from the central level (Rwanda’s MOH and GHSC-PSM) who have trained 30 RMS branch managers, who in turn conduct training at SDPs and collect data during supervisory visits ( Table 3 ). From 2017 to 2021, 1,296 members of the health supply chain staff participated in the QMIA.

Participants in the Quality Management Improvement Approach in Rwanda

Abbreviations: RMS, Rwanda Medical Supply; SDP, service delivery point.

QMIA has helped in many aspects of supply chain management, such as inventory accuracy, expiration rate, and stock according to plan, although challenges remain (e.g., data triangulation). From 2017 to 2019, the program contributed to significant improvements in several key supply chain outcomes: recording of commodity consumption improved from 35% to 95%, e-LMIS utilization improved from 55% to 96% (although some barriers remain at SDPs primarily due to lack of equipment or skills to operate the e-LMIS), the accuracy of inventory data improved from 25% to 85%, and stock-outs for all types of commodities were reduced from 10% to 1%.

QMIA has helped in many aspects of supply chain management, such as inventory accuracy, expiration rate, and stock according to plan, although challenges remain.

QMIA KPIs at different RMS branches and SDPs are shown in Figure 3 . Product availability was the KPI with the best performance both at RMS branches and SDPs, followed by order completeness at RMS branches and on-time delivery at SDPs (all >80%). Expiration rate and invoice generated were the indicators with the lowest performance (<40%), while data triangulation could not be calculated in many cases because of discrepancies between medicines dispensed to patients versus patient numbers and/or improper recordkeeping. KPIs by program followed a similar pattern ( Figure 4 ). We did not find trends for specific districts whose SDPs were performing particularly good or bad (data not shown).

Overall Quality Management Improvement Approach Key Performance Indicators at RMS Branches and SDPs a,b

Abbreviations: e-LMIS, electronic management information system; RMS, Rwanda Medical Supply; SDP, service delivery point.

a Data from May 2021.

b The indicator “Accuracy of e-LMIS consumption data” was only measured in SDPs, since RMS branches do not directly treat patients.

Overall Quality Management Improvement Approach Key Performance Indicators By Program Using Data From RMS Branches a,b

Abbreviations: e-LMIS, electronic management information system; RMS, Rwanda Medical Supply.

b The indicator “Invoice generated” was only applicable to some of the programs; for others there was no invoicing when the site visit was conducted.

Notably, the QMIA effectively addressed performance gaps created by the supply chain staff turnover and staff rotation within an SDP. Thus, the upgrades contributed to greater system efficiency. Although these results are expected to lead to higher patients’ satisfaction, specific data for this outcome have not been collected yet, and there are plans to include this outcome among KPIs in the future.

The program also improved communication between supply chain workers at different levels. For example, facilitators created a WhatsApp group including all district personnel and another between each central team with the district representative. Using social media platforms such as WhatsApp to communicate about performance and resolve issues has fostered a positive behavioral change. This is contributing to the development of local solutions to address challenges in the supply chain operation and management.

However, in 2020, QMIA was not implemented because of disruptions caused by the COVID-19 pandemic, including lockdowns, travel restrictions across districts, and supply chain staff’s efforts shifting to COVID-19 prevention and control activities. Rwanda’s drug supply system was interrupted by the emergence of the pandemic, primarily because of the limited importation of goods from abroad (the country’s pharmaceutical sector heavily relies on imports, mainly from China and India) and the panic-buying practice among customers and some institutions. 15 The prices of different drugs increased and some local pharmaceutical companies shifted their manufacturing capacity to infection prevention–related products such as hand sanitizers or face masks. All these factors led to a shortage of some products. 15 As a result of the pandemic-related disruptions to the QMIA program, there was a reduction in KPIs: e-LMIS utilization decreased to 88% and inventory accuracy to 70% in 2021. With the subsequent reduction of COVID-19 cases and an increased number of people vaccinated, supply chain staff have since resumed their efforts to ensure services are provided at RMS branches and SDPs. This has since shifted both staffing numbers and KPIs to the previous numbers as travel and restrictions were lifted.

During supervisory visits conducted until 2021, the main challenges detected in RMS branches included shortage of inventory for some essential medicines; confusion over the new scope of RMS branches to support health facilities; lack of standard operating procedures for data triangulation, particularly for malaria and family planning commodities; and stock-out of essential medicines. Most of the challenges detected in 2021 were largely manageable and required refresher trainings to staff, which were already provided. Regarding the lack of inventory for some essential medicines, the main cause was delayed use of order forms, leading to stock-outs. This has been solved by training staff on timely use of order forms.

The main challenges identified at health facilities during supervisory visits included high turnover of store managers in charge of the health facility pharmacy (to solve this, the MOH is currently designing incentives that will foster staff retention and motivation, such as trainings, promotions to high performers, and better remunerations); lack of dedicated staff to manage pharmacy operations; and insufficient time allocated to pharmacy-related activities. Additional challenges included a lack of the following: training in inventory management and e-LMIS use for new staff; harmonized dispensing register and information and technology infrastructure needed to perform pharmacy-related activities; an automated process for recording consumption; standard operating procedures for data triangulation, particularly for malaria and family planning commodities; and key information on family planning registers from community health workers. These challenges have been addressed by providing refresher training to staff so they can become more conversant with using the e-LMIS or other technology infrastructure and by upgrading the technology infrastructure to be more user friendly and better meet users’ demands. Finally, the time-consuming procurement of commodities outside of RMS branches is being streamlined so that SDPs can easily access the necessary medicines at every RMS branch.

LESSONS LEARNED

The main lesson learned from the application of the QMIA was that continuous capacity-building is critical to a well-performing supply chain. In addition, the recommendations from training and supervisory visits have provided a better understanding of the reasons behind the poor performance. One of the lessons learned from the impact of the COVID-19 pandemic on supply chain operations was the need to have electronic tools for continuous capacity-building, such as online learning platforms. As a solution to disruptions caused by the pandemic, the MOH and GHSC-PSM jointly developed virtual supply chain training modules and some of the QMIA in-person sessions were conducted online. These materials continue to be used to train supply chain staff at all levels and strengthen the workforce’s capacity at SDPs and regional and central medical stores. In this way, the staff learns about supply chain best practices at their workstations, saving time and resources from supervision and attendance at workshops. However, in-person QMIA sessions will also be conducted in the future to continue capacity-building and improve data quality along the supply chain.

The main lesson learned from the application of the QMIA was that continuous capacity-building is critical to a well-performing supply chain.

We also learned there was a critical need at the SDPs for staff with supply chain backgrounds and experience and that supply chain professionals need a career development plan that promotes motivation and professional growth. In this sense, the selection and provision of adequate training is crucial. Non-accredited training (i.e., training not recognized by the MOH, the World Health Organization, or other accredited bodies) should be excluded since it cannot contribute to professional growth in the supply chain workforce. The main building blocks, successes, and lessons learned are summarized in Figure 5 .

Building Blocks, Successes, and Lessons Learned: How the Quality Management Improvement Approach Contributed to Supply Chain Improvement in Rwanda a

Abbreviation: KPI, key performance indicator.

a  First, the goals and success of the activity were clearly defined. Then, the strategy, data, KPIs, and visual data management were used as building blocks to promote team engagement and action to solve challenges in supply chain operation. This resulted in improved supply chain performance.

Strengths and Limitations

The main strength of the QMIA implementation lies in its establishment of a process of continuous improvement and knowledge transfer, where the central level trains RMS branches and RMS branches train SDPs. Another important point is the government ownership of the QMIA, which is reinforced through collaboration with development partners. These interventions are fully institutionalized, being adopted by the MOH and RMS branches under the technical assistance of GHSC-PSM. After the project ends in 2023, the MOH will assume responsibility for QMIA implementation by providing financial support to be applied independently by RMS. The MOH will continue to carefully evaluate the impacts and challenges that QMIA faces through supervisory visits and discussions and may decide to focus on the most problematic RMS branches or SDPs.

The main strength of the QMIA implementation lies in its establishment of a process of continuous improvement and knowledge transfer, where the central level trains RMS branches and RMS branches train SDPs.

The main limitations of this approach have been observed at the SDP level because a majority of supply chain staff do not have an academic background in supply chain management (most of them are nurses), and there is a limited number of supply chain cadres. To address this challenge, the MOH, through the Human Resource for Health Secretariat, is ensuring the availability of enough trained staff to manage the health supply chain.

Another challenge is the increased workload due to limited staff handling multiple tasks at SDPs. This is being addressed by allocating enough funds to district hospitals to hire staff to be sent to the SDPs, hence reducing the workload of existing staff.

Other supply chain–related interventions conducted in the last few years in Rwanda, including quantification, stock status monitoring, and other capacity-building sessions for RMS and MOH staff, may have also influenced the observed results.

RECOMMENDATIONS TO BUILD RESILIENCE

Given the noticeable improvement in crucial supply chain outcomes with QMIA implementation, we recommend that other countries facing similar challenges in supply chain management adopt this approach to improve the performance of supply chain processes at different levels.

Between 2017 and 2021, QMIA supervision detected performance gaps in supply chain management at both RMS branches and SDPs and proposed the following recommendations that can help build resilience at different levels.

• Support health facilities to have a fully dedicated store manager with the knowledge to support supply chain operations.
• Improve interoperability of hospital systems (e-LMIS and hospital management systems, such as electronic medical records) to ensure unique entry of data and minimize efforts.
• Reinforce data validation through data triangulation between RMS branches and district hospitals to ensure that planning is based on accurate information.

RMS Branches

• Increase the availability of essential medicines at RMS headquarters and branches.
• Build capacity of health facility staff on stock management of health commodities and tools (e.g., e-LMIS, stock cards, etc.).
• Develop a capacity-building plan for new staff at RMS branches and SDPs.
• Train several staff members to manage pharmacy operations to address gaps created by staff turnover.
• Monitor the performance of supply chain operations at SDP level.
• Reinforce data validation through data triangulation between RMS branches and district hospitals.
• Maintain motivated full-time staff dedicated to supply chain management at the SDP level.
• Record consumptions on a regular basis in the e-LMIS.
• Conduct a monthly data quality meeting at SDPs before reporting (including data and store managers, head of facility, and monitoring and evaluation staff).
• Ensure that product data are correctly recorded and promptly reported, in collaboration with data managers, the service responsible, and community health workers.
• Reinforce data validation between the data manager, laboratory technicians, and store managers at health facilities.

These recommendations are in the context of QMIA activity in Rwanda, but other countries willing to apply this approach need to first identify the main gaps and evaluate the performance of the supply chain management and human resources in their own context. Some of the enablers for the success of the QMIA in Rwanda include having policies and strategies that reinforced the national commitment to health and development (e.g., policies promoting the digitalization of the health sector) and the existence of institutions that support the health sector, such as RMS and the Rwanda Biomedical Center.

CONCLUSIONS

The Rwanda MOH and GHSC-PSM implemented the QMIA workforce development intervention to overcome challenges identified in the supply chain of health commodities. Through the QMIA, supply chain professionals in Rwanda have received successful training and capacity-building, which has contributed to significantly improved key supply chain outcomes. This approach can be implemented in other settings having issues with end-to-end data visibility and with supply chain personnel operation and communications.

Acknowledgments

We thank the Rwanda Ministry of Health for the provision of the required resources and collaboration in implementing the Quality Management Improvement Approach countrywide.

This work is being funded by the U.S. Agency for International Development Global Health Supply Chain Program-Procurement and Supply Management project, which has supported the implementation of the activity.

Author contributions

RK, VS, and EZ all contributed to study design, data acquisition and analysis, and article preparation and review.

Competing interests

None declared.

Peer Reviewed

First published online: January 18, 2023.

Cite this article as: Kigenza R, Nsengiyumva E, Sabagirirwa V. The Quality Management Improvement Approach: successes and lessons learned from a workforce development intervention in Rwanda’s health supply chain. Glob Health Sci Pract . 2023;11(1):e2200295. https://doi.org/10.9745/GHSP-D-22-00295

Please log in