multiple embedded case study

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Research Guides

Multiple Case Studies

Nadia Alqahtani and Pengtong Qu

Description

The case study approach is popular across disciplines in education, anthropology, sociology, psychology, medicine, law, and political science (Creswell, 2013). It is both a research method and a strategy (Creswell, 2013; Yin, 2017). In this type of research design, a case can be an individual, an event, or an entity, as determined by the research questions. There are two variants of the case study: the single-case study and the multiple-case study. The former design can be used to study and understand an unusual case, a critical case, a longitudinal case, or a revelatory case. On the other hand, a multiple-case study includes two or more cases or replications across the cases to investigate the same phenomena (Lewis-Beck, Bryman & Liao, 2003; Yin, 2017). …a multiple-case study includes two or more cases or replications across the cases to investigate the same phenomena

The difference between the single- and multiple-case study is the research design; however, they are within the same methodological framework (Yin, 2017). Multiple cases are selected so that “individual case studies either (a) predict similar results (a literal replication) or (b) predict contrasting results but for anticipatable reasons (a theoretical replication)” (p. 55). When the purpose of the study is to compare and replicate the findings, the multiple-case study produces more compelling evidence so that the study is considered more robust than the single-case study (Yin, 2017).

To write a multiple-case study, a summary of individual cases should be reported, and researchers need to draw cross-case conclusions and form a cross-case report (Yin, 2017). With evidence from multiple cases, researchers may have generalizable findings and develop theories (Lewis-Beck, Bryman & Liao, 2003).

Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.). Los Angeles, CA: Sage.

Lewis-Beck, M., Bryman, A. E., & Liao, T. F. (2003). The Sage encyclopedia of social science research methods . Los Angeles, CA: Sage.

Yin, R. K. (2017). Case study research and applications: Design and methods . Los Angeles, CA: Sage.

Key Research Books and Articles on Multiple Case Study Methodology

Yin discusses how to decide if a case study should be used in research. Novice researchers can learn about research design, data collection, and data analysis of different types of case studies, as well as writing a case study report.

Chapter 2 introduces four major types of research design in case studies: holistic single-case design, embedded single-case design, holistic multiple-case design, and embedded multiple-case design. Novice researchers will learn about the definitions and characteristics of different designs. This chapter also teaches researchers how to examine and discuss the reliability and validity of the designs.

Creswell, J. W., & Poth, C. N. (2017). Qualitative inquiry and research design: Choosing among five approaches . Los Angeles, CA: Sage.

This book compares five different qualitative research designs: narrative research, phenomenology, grounded theory, ethnography, and case study. It compares the characteristics, data collection, data analysis and representation, validity, and writing-up procedures among five inquiry approaches using texts with tables. For each approach, the author introduced the definition, features, types, and procedures and contextualized these components in a study, which was conducted through the same method. Each chapter ends with a list of relevant readings of each inquiry approach.

This book invites readers to compare these five qualitative methods and see the value of each approach. Readers can consider which approach would serve for their research contexts and questions, as well as how to design their research and conduct the data analysis based on their choice of research method.

Günes, E., & Bahçivan, E. (2016). A multiple case study of preservice science teachers’ TPACK: Embedded in a comprehensive belief system. International Journal of Environmental and Science Education, 11 (15), 8040-8054.

In this article, the researchers showed the importance of using technological opportunities in improving the education process and how they enhanced the students’ learning in science education. The study examined the connection between “Technological Pedagogical Content Knowledge” (TPACK) and belief system in a science teaching context. The researchers used the multiple-case study to explore the effect of TPACK on the preservice science teachers’ (PST) beliefs on their TPACK level. The participants were three teachers with the low, medium, and high level of TPACK confidence. Content analysis was utilized to analyze the data, which were collected by individual semi-structured interviews with the participants about their lesson plans. The study first discussed each case, then compared features and relations across cases. The researchers found that there was a positive relationship between PST’s TPACK confidence and TPACK level; when PST had higher TPACK confidence, the participant had a higher competent TPACK level and vice versa.

Recent Dissertations Using Multiple Case Study Methodology

Milholland, E. S. (2015). A multiple case study of instructors utilizing Classroom Response Systems (CRS) to achieve pedagogical goals . Retrieved from ProQuest Dissertations & Theses Global. (Order Number 3706380)

The researcher of this study critiques the use of Classroom Responses Systems by five instructors who employed this program five years ago in their classrooms. The researcher conducted the multiple-case study methodology and categorized themes. He interviewed each instructor with questions about their initial pedagogical goals, the changes in pedagogy during teaching, and the teaching techniques individuals used while practicing the CRS. The researcher used the multiple-case study with five instructors. He found that all instructors changed their goals during employing CRS; they decided to reduce the time of lecturing and to spend more time engaging students in interactive activities. This study also demonstrated that CRS was useful for the instructors to achieve multiple learning goals; all the instructors provided examples of the positive aspect of implementing CRS in their classrooms.

Li, C. L. (2010). The emergence of fairy tale literacy: A multiple case study on promoting critical literacy of children through a juxtaposed reading of classic fairy tales and their contemporary disruptive variants . Retrieved from ProQuest Dissertations & Theses Global. (Order Number 3572104)

To explore how children’s development of critical literacy can be impacted by their reactions to fairy tales, the author conducted a multiple-case study with 4 cases, in which each child was a unit of analysis. Two Chinese immigrant children (a boy and a girl) and two American children (a boy and a girl) at the second or third grade were recruited in the study. The data were collected through interviews, discussions on fairy tales, and drawing pictures. The analysis was conducted within both individual cases and cross cases. Across four cases, the researcher found that the young children’s’ knowledge of traditional fairy tales was built upon mass-media based adaptations. The children believed that the representations on mass-media were the original stories, even though fairy tales are included in the elementary school curriculum. The author also found that introducing classic versions of fairy tales increased children’s knowledge in the genre’s origin, which would benefit their understanding of the genre. She argued that introducing fairy tales can be the first step to promote children’s development of critical literacy.

Asher, K. C. (2014). Mediating occupational socialization and occupational individuation in teacher education: A multiple case study of five elementary pre-service student teachers . Retrieved from ProQuest Dissertations & Theses Global. (Order Number 3671989)

This study portrayed five pre-service teachers’ teaching experience in their student teaching phase and explored how pre-service teachers mediate their occupational socialization with occupational individuation. The study used the multiple-case study design and recruited five pre-service teachers from a Midwestern university as five cases. Qualitative data were collected through interviews, classroom observations, and field notes. The author implemented the case study analysis and found five strategies that the participants used to mediate occupational socialization with occupational individuation. These strategies were: 1) hindering from practicing their beliefs, 2) mimicking the styles of supervising teachers, 3) teaching in the ways in alignment with school’s existing practice, 4) enacting their own ideas, and 5) integrating and balancing occupational socialization and occupational individuation. The study also provided recommendations and implications to policymakers and educators in teacher education so that pre-service teachers can be better supported.

Multiple Case Studies Copyright © 2019 by Nadia Alqahtani and Pengtong Qu is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

• Research article
• Open access
• Published: 14 January 2022

An embedded multiple case study: using CFIR to map clinical food security screening constructs for the development of primary care practice guidelines

• Sabira Taher   ORCID: orcid.org/0000-0001-5368-2998 1 ,
• Naoko Muramatsu 2 ,
• Angela Odoms-Young 3 ,
• Nadine Peacock 2 ,
• C. Fagen Michael 1 &
• K. Suh Courtney 4  

BMC Public Health volume  22 , Article number:  97 ( 2022 ) Cite this article

2642 Accesses

2 Citations

Metrics details

Food insecurity (FI), the limited access to healthy food to live an active and healthy life, is a social determinant of health linked to poor dietary health and difficulty with disease management in the United States (U.S.). Healthcare experts support the adoption of validated screening tools within primary care practice to identify and connect FI patients to healthy and affordable food resources. Yet, a lack of standard practices limits uptake. The purpose of this study was to understand program processes and outcomes of primary care focused FI screening initiatives that may guide wide-scale program implementation.

This was an embedded multiple case study of two primary care-focused initiatives implemented in two diverse health systems in Chicago and Suburban Cook County that routinely screened patients for FI and referred them to onsite food assistance programs. The Consolidated Framework for Implementation Research and an iterative process were used to collect/analyze qualitative data through semi-structured interviews with N  = 19 healthcare staff. Intended program activities, outcomes, actors, implementation barriers/facilitators and overarching implementation themes were identified as a part of a cross-case analysis.

Programs outcomes included: the number of patients screened, identified as FI and that participated in the onsite food assistance program. Study participants reported limited internal resources as implementation barriers for program activities. The implementation climate that leveraged the strength of community collaborations and aligned internal, implementation climate were critical facilitators that contributed to the flexibility of program activities that were tailored to fill gaps in resources and meet patient and clinician needs.

Highly adaptable programs and the healthcare context enhanced implementation feasibility across settings. These characteristics can support program uptake in other settings, but should be used with caution to preserve program fidelity. A foundational model for the development and testing of standard clinical practice was the product of this study.

Peer Review reports

• Food insecurity

Food insecurity (hereafter FI) is a social determinant of health and economic condition where limited access to safe, high-quality, nutritious food prevents individuals from leading active and healthy lives [ 1 ]. The World Bank estimated that acute food insecurity increased drastically and on a global scale due to COVID-19, and that the majority of these cases were connected to hunger in International Development Association countries driven by climate change, long-lasting conflict, and other economic conditions [ 2 ]. Low levels of education and limited social networks were consistent variables across countries that increased the risk for FI. Yet, country-specific economic, political and sociocultural factors varied greatly between countries, which highlights the need to utilize country-specific interventions and policies to reduce FI on a local level [ 3 ]. This study specifically focuses on FI and the healthcare context in the U.S., which, when compared to peer nations, has the highest number of preventable chronic illnesses related to poor nutrition, as well as hospitalizations and deaths [ 4 ].

Prior to the COVID-19 pandemic, approximately 33.5 million Americans experienced FI [ 1 ]. Since April 2020, that number has nearly doubled [ 5 ].. FI in the U.S. is characterized by the overconsumption of poor quality, highly processed, calorie dense food that is extremely affordable and widely available [ 6 ]. U.S. households most affected by FI are low-income, ethnic and minority communities, especially those affected by unemployment and job loss [ 7 ] FI contributes to the limited ability to eat a healthy diet, often the first recommended step for disease management. Thus FI contributes to the high prevalence of obesity, diabetes, cardiovascular disease, and difficulty with disease management among low-income U.S. populations [ 6 ].

Recommendations for screening and linking patients

Studies show that local and federal U.S. food assistance services remain underutilized due to limited awareness about their existence, the stigma associated with using welfare programs, and complex enrollment processes that can discourage use [ 1 , 11 , 12 , 13 ]. There is a growing body of evidence that illustrates how partnerships between healthcare systems and local food assistance programs can increase the use of services and help improve dietary health [ 8 , 9 , 10 ]. Research shows that recommendations for identifying FI patients through routine FI screening with the validated Hunger Vital Signs™ tool, and referring patients to evidence-based programs, such the Supplemental Nutrition Assistance Program (hereafter SNAP) and community food pantries, increases the use of food assistance services, and has demonstrated immediate dietary improvements in cancer, diabetes and hypertensive patients [ 11 , 12 , 13 , 14 , 15 ].

Addressing food insecurity in primary care practice

Among screening initiatives that exist, those implemented in primary care settings demonstrate the most potential to address FI because primary care practice is the most common form of health care delivery in the U.S. Moreover, primary care is recognized by professional and government healthcare organizations as the most typical setting where referrals to social and community resources often occur that connect patients to basic needs for disease management [ 8 , 16 , 17 , 18 ].

Lack of evidence that points to improved health outcomes

Existing literature points to FI screening practices that are largely guided by broad, national principles that have been interpreted in many ways—perhaps due to their rapid and organic evolution to fill a growing FI crisis. Evidence suggests that program activities, actors, implementation processes are driven by each healthcare contexts, program outcomes vary across clinical settings, and as a result, the long-term impact on health outcomes of screening programs cannot be determined [ 9 , 19 , 20 ]. The challenge stems from the lack of translational research and rigorously tested standard practices in this relatively new area of clinical practice [ 20 , 21 ]. The gaps in the literature suggest that we need to examine how these programs operate in real-world settings, and identify which program activities have the most potential for generalization. This can inform the development of practice guidelines that can be tested in effectiveness trials, and eventually implemented and tested on a wider scale.

Implementation science

In implementation science, theory derived frameworks are used to study implementation context—specifically how multilevel system wide factors (e.g. individual level organizational level) and multisector factors (i.e. policies, external partnerships and community needs) interact and determine the quality of implementation outcomes [ 22 ]. Findings from implementation science studies allow researchers to hypothesize the relationship between implementation factors. These contextual variables can be tested in other settings where program adaptations maybe considered that lend themselves to wide-scale dissemination of evidence-based practice [ 22 ].

Implementation science research has been supported in several healthcare research studies, most notably by the National Institutes of Health for a variety of social and behavioral health research, such as tobacco cessation and diabetes prevention. The purpose of these studies was to understand how the complex and interdependent sociocultural, economic and political factors within the implementation context affected the process of program implementation. There is an opportunity to apply implementation science in the context of clinical FI initiatives because theoretical underpinnings of implementation have not yet been explored. Ultimately, researchers used findings to determine how to effectively disseminate and adapt these programs into other healthcare settings and contexts [ 22 , 23 ].

• Consolidated framework for implementation research

The Consolidated Framework for Implementation Research (hereafter CFIR) is an Implementation determinants framework comprised of theoretically derived domains and constructs as seen in Table  1 . CFIR has been empirically tested and is widely used in healthcare settings to understand multidimensional, interrelated implementation barriers and facilitators within specific healthcare organizations [ 24 , 25 , 26 ].

The framework is broad with over 30 theoretically-derived constructs. When used to map implementation factors, CFIR can help researchers establish a foundation from which semantic relationships between implementation factors can be constructed. Hypothesized relationships between constructs can be used to develop a conceptual model that describes the implementation of a specific intervention. The framework is made up of implementation drivers that are categorized into five domains: 1) The intervention characteristics that point to the quality of the program, compatible design, its cost and adaptability across settings. 2) The inner setting, which directly relates to the physical and cultural setting where daily program processes occur. 3) The outer setting, which refers to any factor external to the program itself, including community needs, influences, local mandates, policies or regulations that affect implementation processes. 4) Characteristics of program staff/individuals, which are their knowledge and beliefs about the program from their own perspective. 5) Implementation processes, which include the steps used in planning, execution and ongoing management of the program [ 22 ].

The purpose of this study was to understand implementation processes and outcomes of two distinctively different FI screening initiatives. One program was implemented in primary care clinics located within the context of an urban, government funded health system. The other program was implemented in primary care clinics associated with a suburban, private, academic medical center. A total of N  = 19 healthcare staff participated in one-on-one interviews in this study to provide their perspectives about implementation. We used the Consolidated Framework for Implementation Research (CFIR) to identify common implementation barriers and facilitators from each interview as a part of a cross-case analysis. Findings from this study were used to develop a formative conceptual model that can guide the development, refinement and testing of standard screening practices in future research.

Selection of study cases

Cook County is located in Illinois, a state located in the Midwestern region of the U.S. Within Cook County is a complex healthcare network that serves 5.2 million people. Two FI screening and referral programs were selected for this study using criterion sampling from a larger sample of 13 programs implemented within primary care settings identified in a previous study. The two programs selected for this study (hereafter Program A and Program B) differed in the type of setting (i.e. one public, government funded organization, the other an academic medical center). Distinct program differences listed in Table 1 allowed for the exploration of program implementation in different contexts and the extraction of common, overarching implementation themes.

Inclusion criteria were based on previous research and national recommendations for clinical FI screening initiatives [ 8 , 17 , 18 , 27 ]. Study cases met the following criteria: 1) Programs that utilized the standardized two question Hunger Vital Signs tool to screen patients for FI; 2) Programs that incorporated a referral system to onsite food services for FI patients; 3) Programs that incorporated a referral system for FI to enroll in SNAP and other federal benefits; 4) Programs that had been implemented for a minimum of one year. The last criterion allowed for the examination of programs that had been presumably functioning long enough that initial challenges common to start-up programs had already been addressed.

Study design

An embedded multiple case study design was used to examine the phenomenon of primary care situated FI screening and referral processes [ 28 , 29 ]. The embedded nature of this study refers to the multiple units of analysis within each case [ 29 ]. Preliminary research for this study indicated that the healthcare context (e.g. clinicians at the practice level) drove how FI screening programs were implemented and what types of food assistance programs were incorporated for referral. Therefore, each case in this study was identified as one individual screening initiative and the units of analysis were clinical program actors within the healthcare setting as illustrated in Fig.  1 .

Units of analysis across healthcare organizations in this study ( N  = 19)

Participants

From September 2019 to March 2020, an iterative sampling approach was used to recruit participants for this study from a convenience sample of implementation actors at each case until data saturation was achieved (N = 19). Through a purposive sampling process, implementation leaders, clinicians and other healthcare staff critical to program implementation were recruited for this study [ 30 ].

Study instrument and data collection

The interview guides used with organizational leaders and frontline providers were developed for this study using the adapted CFIR framework (available in Additional File  1 “Interview Guide for Key Program Planners” and “Frontline Provider Interview Guide”). As in similar research, interview questions broadly asked about program activities, implementation processes, program outcomes and asked participants to identify major challenges/facilitators that affected feasibility and fidelity of program implementation [ 31 ].

A trained qualitative researcher (ST) conducted semi-structured, key informant interviews for this study. The interviews were conducted face-to-face at each program site or over the telephone at the study participant’s discretion. Each interview lasted 30–45 min and were audio recorded for data analysis purposes. Participants recruited for the study were made aware of the audio recording at the beginning of each session and were required to provide verbal consent prior to participation in the study. This study and the verbal consent process were approved for a claim of exemption (Protocol # 2019–0610) from the University of Illinois at Chicago Office for the Protection of Research Subjects Institutional Review Board on August 30, 2019.

The researcher took detailed notes during each interview that provided initial insights to the study. Revisions to the instrument guide were made after each interview for clarity and to collect additional program details.

Data collection, coding and analysis

Data were collected, managed and analyzed concurrently over a period of seven months until data saturation was achieved. Transcriptions of the interviews were uploaded to Atlas.ti v.8 Qualitative Data Analysis Software for data management, coding and assistance with analysis. All personal identifying information was removed from the data prior to analysis. All data were stored on a password protected computer only accessible by the researcher. A codebook developed a priori based on the adapted CFIR framework for data interpretation was used during data collection and analysis (see Additional File  2 , “Study Codebook”). Codes were added to or removed from the codebook based on previous research, organizational theory, and as new ideas and concepts emerged, illustrated in Fig.  2 [ 9 , 32 , 33 ].

Adapted CFIR framework for this study

Two experienced PhD level university students (ST and LC) established interrater reliability of the coding process until 80% agreement was achieved as recommended for qualitative research [ 34 ]. As data were collected, memos were used to document progress, study decisions and emerging themes [ 35 ]. Matrices and frameworks were developed to guide thematic analysis and anchor emerging concepts to specific CFIR constructs [ 36 ]. The themes and patterns that emerged from each interview were compared to previous interview findings. This allowed the identification of commonalities, disparities and outliers in the data and for a rich understanding of program implementation to emerge [ 28 ].

For each case, program activities, time of occurrence and implementation actors were confirmed. Implementation processes were also described as originally intended, as well as unanticipated implementation facilitators and challenges and the unique implementation context that resulted in program adaptations.

Program outcomes were also collected to assess implementation feasibility, effectiveness, as well as overall program fidelity [ 31 , 37 ]. The following program outcomes were identified across cases: the number of patients screened; the number that identified as FI; the number of patients referred to food assistance programs; the number of patients that participated in the food assistance program. The frequency that clinicians completed essential program activities was also collected to tie outcomes to specific program elements. During the cross-case analysis, the binding implementation themes were identified and gave meaning to program outcomes.

Atlas.ti v8 exploratory functions were used to further analyze and confirm findings, and for source triangulation between participants. Any overlap of themes helped to establish the semantic relationships between CFIR constructs. Prior to the finalization of study results, one program leader and one clinician from each case were asked to participate in member-checks. They each reviewed the results from their respective case, and provided feedback where necessary to ensure validity of study findings.

Program activities and process outcomes

Study findings revealed similar intended program activities and processes within and across cases illustrated in Tables  2 and 3 .

Both cases intended to screen all of their patients for FI using their EMR systems and projected that 45% of their patient population would be identified as FI based on previous community-wide data [ 38 ]. Across cases, screening took place by a clinician approximately one time a year prior to the doctor’s visit during intake or in the clinic waiting room. Positive responses were documented and flagged in the patient’s medical record to prompt the physician to discuss FI during the patient visit. Between 24 and 31% of patients were actually identified through screening and were referred to an onsite food distribution program that provided fresh produce at no cost to FI patients. During the doctor’s visit, all patients identified as FI were projected to receive information or list about local food assistance resources and should have been referred to the Social Worker to enroll in SNAP benefits if eligible. These data were either unavailable for this study or this program activity was not performed.

Clinicians across cases intended to use phone call reminders as an opportunity to remind and educate FI patients about the benefits of participating in the food distribution programs. Between 8 and 22% of patients received phone call reminders.

Thematic analysis findings

Figure  3 represents a formative conceptual model utilizing CFIR concepts that sums up overarching themes described below. The model illustrates the semantic relationships between core CFIR concepts revealed during thematic analysis that helps to explain why process outcomes were lower than projected.

Formative conceptual model for implementation

Barriers within the inner setting

Clinicians across cases reported that physical space, clinician capacity, financial resources and EMR technology were resource challenges that inhibited program implementation. The study revealed a hierarchical relationship where barriers study participants identified trickled down from the organizational systems level that resulted in challenges with the delivery of care at the clinic level and ultimately affected how patients experienced FI screening and referral processes as seen in Fig. 3 . Identified in similar intervention studies, these challenges speak to the broader applicability of study findings to other U.S. healthcare settings [ 39 ].

Study participants agreed that due to their health system’s pre-programmed EMR software, clinician prompts for FI screening and resource lists were not functional for the realities of day-to-day clinical care. Study participants reported that electronic screening tools were either unavailable or were only available intermittently (e.g. once per year), which did not allow for clinicians to capture the episodic and cyclical nature of FI. Resource lists were embedded so deeply within the EMR system that study participants also reported that they did not have time to navigate to those lists during patient visits.

Limited financial resources that health systems could allocate to screening and referral initiatives negatively affected the frequency of food distribution, program sustainability and reach. Study participants reported that screening and referral programs were supported by a finite amount of in-kind donations from their community food assistance partner and local and federal grants. As a result, funding would last for only a fixed amount of time, and study participants reported that the money just did not pay for enough food meet patient needs. Moreover, grant dollars had spending restrictions, and required patients to be enrolled in SNAP benefits. Study participants reported that because of this spending rule, they had to turn several of their low-income patients away during program activities if they were not eligible for SNAP benefits, which negatively affected program reach.

Rigid workflow processes across cases provided a small window of opportunity for FI screening and referral during the patient visit. Study participants said that check-in and intake were activities that occurred in settings with very little privacy, but indicated that this was typical for how clinic waiting room or nurse’s intake stations functioned. Physical space became a barrier to screening, referral and food distribution. Across cases, patients displayed discomfort when presented with FI questions, which study participants believed was because of the lack of privacy from other patients. Some patients commonly denied experiencing FI, even if clinicians knew that this was not the case. A lack of privacy could explain why a lower than expected number of patients were identified as FI, and were referred to and participated in the food distribution programs.

Study participants reported that clinician capacity to deliver patient care was dependent on the health system’s workflow, and consistently reported that its rigidity did not provide enough time to distribute food assistance resource lists and counsel patients about FI. Clinicians also reported the inability to conduct patient outreach and phone calls for the purpose of increasing awareness about when food distribution was scheduled—an important component for increasing program reach. This finding could explain why there were lower than expected number of patients participated in the food distribution programs overall.

Leveraging the implementation context as a facilitator

Across cases, study participants reported that the support of community partnerships and the internal work culture created an aligned implementation context. As seen in Fig. 3 , the community climate—cosmopolitanism—played a big role during program planning and execution and clinic-level autonomy allowed clinicians to make timely program adaptations when faced with resource challenges.

Multi-sector networks supported inherent synergy with existing community initiatives, which gave each health system access to an existing program model, expertise and infrastructure. The local food justice organization and food depository that partnered with the health systems advocated for an equitable and sustainable food system through existing initiatives throughout the community and played a fundamental role in bringing together health systems, local food growers and other health and wellness community organizations; the collective strength and presence of which created a supportive implementation climate where study participants reported an activation of knowledge, awareness and advocacy work among clinical staff in preparation for program implementation.

For example, study participants reported the presence of farm stands across the health system campus, medical and dietetics students involved in program activities, the use of offsite community centers to increase program participation and the assistance of local grant opportunities to increase food production and reach.

A culture of clinic-level autonomy facilitated the equitable distribution of decision-making authority. “Managers need to take ownership because they know their patients and their staff,” said one study participant and in turn, when asked about this, clinicians responded with, “We do things differently here,” and “the way we do it is we want to reach everyone…” Statements of this kind referred to linkages, voucher activities and food distribution processes that study participants reported were adapted for universal distribution to reduce the stigma of FI. This adaptation could explain why the number of patients that participated in food distribution programs were much greater than the number of phone call reminders, and SNAP enrollment activities that clinicians actually engaged in.

Adaptability and trialability of program characteristics

Study participants reported that program activities were highly adaptable and testable. Adaptability refers to the degree to which the core program components can be tailored to fit the implementation context [ 22 ]. Trialability refers to the ability for stakeholders to pilot an intervention on a small scale and engaged in quality improvement efforts [ 22 ]. As seen in Fig. 3 , both constructs emerged in the context of limited program resources. It appeared that when limited program resources were supplemented, high levels of adaptability and trialability characteristics were revealed. Conversely, the high level of adaptability and trialability of each program allowed for ongoing exploration of alternative and creative methods to improve program implementation, reach and sustainability, suggesting a bi-directional relationship between these program characteristics.

Table  4 lists the overarching study themes and illustrative quotes that assisted in the interpretation of findings.

This is the first study, to this author’s knowledge, that applied CFIR to examine system-wide implementation factors of clinical FI screening initiatives within the context of healthcare settings and primary care. Empirically tested and theoretically derived CFIR concepts guided the development of a conceptual implementation model, while integrating program outcomes strengthened the interpretation of qualitative findings. The conceptual model in Fig. 3 may be tested and refined in follow-up studies to facilitate implementation and increase program reach, impact and sustainability.

In this early stage of formative research, one optimal combination of clinical screening and referral activities did not emerge as generalizable for testing on a larger scale, which is a necessary step in the translational research pipeline [ 40 ]. The U.S. healthcare system’s fragmented payer system, lack of universal coverage and disparities in cost and quality of healthcare may have contributed to this finding, and was reflective in this study when each program was operationalized in different ways to meet the unique challenges, needs and context of each health system and patient population.

Nevertheless, overarching themes that emerged across cases that maybe generalizable. Salient to this study were the CFIR concepts, program adaptability and trialability that made implementation feasible across both cases, while maintaining core screening and referral activities. This is consistent with the scalability and implementation framework literature that relies on assessing context, such as available human capital, technical resources, financial costs, and any other contextual factors that may not be replicable in a larger study, but that provide information about the authenticity and feasibility of delivering core intervention activities in clinical practice [ 26 , 40 , 41 ]. This finding is also reflective in and policy recommendations for SDOH screening practices that identify the flexibility of SDOH screening program activities to meet the health system context, including patient and staff needs [ 42 , 43 ].

Building on this concept, the proactive support of intervention modifications has been proposed in emerging health equity research as a way to address disparities in healthcare delivery, access, resources and outcomes in our most vulnerable populations [ 44 ]. It requires the documentation of intervention modifications, which enhance fit or effectiveness in a given context that can lead to improved engagement, acceptability and clinical outcomes [ 44 , 45 ]. Documentation of key adaptations can also facilitate more rigorous feasibility studies when researchers clarify the context of adaptations, such as the reasoning, timing, and process of modifications that facilitated implementation, scale-up, spread or sustainability, and should be considered in future clinical FI screening research that builds on this study [ 45 ].

Moreover, adaptability and trialability highlighted the significance of the CFIR cosmopolitanism concept in this study. Specifically, the interaction between the inner culture and community context drove program design and filled healthcare resource gaps. This finding reflects the current literature on the existences of clinical-community linkages to address FI through clinical screening and referral mechanisms [ 8 , 9 ]. It also points to a multi-sector response that has already demonstrated effective collaborations between primary care and community organizations in the control and management of communicable and chronic diseases by establishing a medical home that is patient and community centered [ 46 , 47 ].

Recommendations

Study findings resulted in the following recommendations for health systems: 1) Allow for adaptations with caution. Unique implementation contexts can foster implementation feasibility. Yet, considerations need to be made about how adaptations may negatively impact fidelity, reach and effectiveness. 2) Consider how the context can support intervention activities through clinician input about workflow, program responsibilities and time management. 3) Conduct asset mapping and outreach to potential community partners that have a strong presence in the community, aligned goals and objectives and resources that can be leveraged during program design and implementation. This recommendation raises its own challenges about whose responsibility within the health system it is to make community-wide connections and manage relationships, but is key for establishing a truly patient and community-centered medical home. 4) Consider non-traditional forms of staff support. In this study, allied health and medical students were motivated to work as interns in exchange for hands-on, experiential learning. Generally, students are subject to high turnover and may not always be the best solution to fill staffing shortages that require a long-term commitment. An alternative solution is to leverage the role and expertise of community health workers that are trusted sources of information for patients because they often live within the communities they serve.

Limitations

As a study instrument, the researcher was positioned alongside study participants during the process of information discovery during data collection and analysis. As such, this was a subjective process that may have been affected by the researcher’s own biases and experiences [ 48 ]. The researcher utilized source triangulation and member checks to negate the effect of these factors during data analysis and interpretation.

While this study incorporated the perspective of multiple implementation actors representative of the implementation context, the sample size may be considered small at first glance. What is important to note is that data saturation was achieved, and that qualitative research of this nature requires the deep exploration of the context to interpret findings in a meaningful way. The scope of the study may have been expanded to incorporate more programs and program staff if time and resources to complete this study had not been limited.

The study did not include patients’ perspectives or in vivo observations of screening and referral processes. Real-time data could have enhanced study findings, and patients’ perspectives could have provided insight about how screening and referral processes affected their clinical experience. The amount of time allotted for this study limited the scope of the study to the perspective of implementation actors only. Moreover, due to patient privacy laws, the study sites would not allow researchers to sit in during clinical visits. Future studies should consider patient interviews and immediate, post visit surveys to gauge a patient’s perspective about screening and referral processes.

Due to time restrictions, data that were collected at only one point in time and relied on the memory of each participant. Future studies should consider the collection of data from participants at multiple time points to capture the dynamic process of implementation and to further validate findings.

Lastly, this study is applicable only to the context of the U.S. healthcare system and characteristics of FI within the U.S. Nevertheless, a community-clinical integrated model may have the potential to address hunger in other countries.

Implications

This study makes significant contributions to the limited body of literature in the emerging field of clinical FI screening programs in primary care practice. In particular, the proposed conceptual model is a foundation for the development of theory-driven standard practices. Though formative in nature the model identifies areas of exploration that have not been considered in previous research, such as intervention adaptability, internal work culture and the community climate.

Study findings have implications for practice-based research. The exploration of external factors and creative uses of internal assets for program support should be considered due to the scarcity of funding for community-based interventions implemented in low-resource clinics. Future work should consider how these factors may enhance limited internal resources long-term. Community-engaged formative research with patients could help to tailor primary care focused initiatives to the realities of patient needs. Engaging the patient community could provide critical insights about stigma, privacy, trust and workflow processes from the patient’s perspective, as well as provide deeper understanding about the cyclical nature of household FI that may inform frequency of screening and can be used to advocate for additional health services. Study findings also have implications for ongoing policy work of universal social determinants of health screening practices supported by national healthcare experts.

The key take away from this study is that due to limited healthcare resources, primary care practices that serve low-income communities need to be supported in their ability to adapt program activities to their specific context. While high program fidelity and intended program outcomes may not have been achieved in this study, findings demonstrate how implementation feasibility can be achieved when community partnerships and an internal resources are leveraged for program adaptations and support. With this in mind, future research may continue to build on the proposed conceptual model, which is formative in nature and sets the stage for development of standard screening practices. As our healthcare system continues its transition to a value-based model of care, we need to consider how primary care focused FI screening initiatives can effectively connect patients to food resources. If we can reduce the inequitable access to affordable and healthy food, we may eventually see long-term improvements in the quality of life of our most vulnerable populations.

Abbreviations

Food Insecurity/Food Insecure

United States

Supplemental Nutrition Assistance Program

Special Supplemental Nutrition Assistance Program for Women’s Infants and Children

Electronic Health Records

Electronic Medical Records

Consolidated Framework for Implementation Research

Alliance for Health Equity

Services USD of AER. Food and Nutrition Assistance, Food Security in the U.S., Measurement [Internet]. Vol. 2017. 2016. Available from: https://www.ers.usda.gov/topics/food-nutrition-assistance/food-security-in-the-us/measurement/

Food Security and COVID-19 [Internet]. [cited 2021 May 25]. Available from: https://www.worldbank.org/en/topic/agriculture/brief/food-security-and-covid-19

Smith MD, Rabbitt MP. Coleman- Jensen a. who are the world’s food insecure? New evidence from the food and agriculture organization’s food insecurity experience scale. World Dev. 2017;93:402–12.

Article   Google Scholar  

U.S. Health Care from a Global Perspective, 2019 | Commonwealth Fund [Internet]. [cited 2021 Jun 18]. Available from: https://www.commonwealthfund.org/publications/issue-briefs/2020/jan/us-health-care-global-perspective-2019

Food insecurity is at the highest levels since the Great Depression because of the COVID-19 pandemic [Internet]. [cited 2021 May 11]. Available from: https://slate.com/technology/2020/11/food-insecurity-crisis-thanksgiving.html

Seligman KK, Laraia BA, Kushel MB. Food insecurity and clinical measures of chronic disease. Abstract Presentation, SGIM, National Meeting, PA,. 2008.

Gundersen C, Engelhard EE, Crumbaugh AS, Seligman HK. Brief assessment of food insecurity accurately identifies highrisk US adults. Public Health Nutr. 2017;20(08):1367–71.

Barnidge E, Stenmark S, Seligman H. Clinic-to-community models to address food insecurity. JAMA Pediatr. 2017;171(6):507–8.

Lundeen EA, Siegel KR, Calhoun H, Kim SA, Garcia SP, Hoeting NM, et al. Clinical-community partnerships to identify patients with food insecurity and address food needs. Prev Chronic Dis. 2017;14(E113).

Joshi K, Smith S, Bolen SD, Osborne A, Benko M, Trapl ES. Implementing a Produce Prescription Program for Hypertensive Patients in Safety Net Clinics. Health Promot Pract [Internet]. 2019 Jan 1 [cited 2020 May 24];20(1):94–104. Available from: http://journals.sagepub.com/doi/10.1177/1524839917754090

Patel KG, Borno HT, Seligman HK. Food insecurity screening: A missing piece in cancer management. Vol. 125, Cancer. John Wiley and Sons Inc.; 2019. p. 3494–501.

Seligman HK, Laraia BA, Kushel MB. Food insecurity is associated with chronic disease among low-income NHANES participants. J Nutr. 2010;140(2):304–10.

Article   CAS   Google Scholar  

Cavanagh M, Jurkowski J, Bozlak C, Hastings J, Klein A. Veggie Rx: an outcome evaluation of a healthy food incentive programme. Public Health Nutr. 2017;20(14):2636–41.

Nestle M. The Supplemental Nutrition Assistance Program (SNAP): History, politics, and public health implications. Am J Public Health [Internet]. 2019 Dec 6 [cited 2020 May 28];109(12):1631–5. Available from: https://ajph.aphapublications.org/doi/10.2105/AJPH.2019.305361

Trapl ES, Smith S, Joshi K, Osborne A, Matos AT, Bolen S. Dietary impact of produce prescriptions for patients with hypertension. Prev Chronic Dis 2018 1;15(11).

Gupta DM, Boland RJ Jr, Aron DC. The physician’s experience of changing clinical practice: a struggle to unlearn. Implement Sci. 2017;12(1):22–8.

Garg A, Toy S, Tripodis Y, Silverstein M, Freeman E. Addressing Social Determinants of Health at Well Child Care Visits: A Cluster RCT. Pediatrics [Internet]. 2015;135(2):e296–304 1p. Available from: http://search.ebscohost.com/login.aspx?direct=true&db=rzh&AN=103794683&site=ehost-live

Barnidge E, LaBarge G, Krupsky K, Arthur J. Screening for food insecurity in pediatric clinical settings: opportunities and barriers. J Community Health. 2017;42(1):51–7.

Torres J, De Marchis E, Fichtenberg C, Gottlieb L. Identifying food insecurity in health care settings: a review of the evidence. San Francisco, CA: Social Interventions Research and Evaluation Network; 2017.

Google Scholar  

De Marchis EH, Torres JM, Benesch T, Fichtenberg C, Allen IE, Whitaker EM, et al. Interventions addressing food insecurity in health care settings: A systematic review. Vol. 17, Annals of Family Medicine. Annals of Family Medicine, Inc; 2019. p. 436–47.

Downer S, Berkowitz SA, Berkowitz SA, Harlan TS, Olstad DL, Mozaffarian D. Food is medicine: Actions to integrate food and nutrition into healthcare [Internet]. Vol. 369, The BMJ. BMJ Publishing Group; 2020 [cited 2021 May 25]. Available from: https://doi.org/10.1136/bmj.m2482http://www.bmj.com/

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

Glasgow RE, Vinson C, Chambers D, Khoury MJ, Kaplan RM, Hunter C. National Institutes of Health approaches to dissemination and implementation science: current and future directions. Am J Public Heal [Internet]. 2012;102(7):1274–81. Available from: https://doi.org/10.2105/AJPH.2012.300755 .

Damschroder LJ. Clarity out of chaos: use of theory in implementation research. Psychiatry Res. 2020;283:112461.

Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the consolidated framework for implementation research. Vol. 11, Implementation Science. BioMed Central Ltd.; 2016.

Li SA, Jeffs L, Barwick M, Stevens B. Organizational contextual features that influence the implementation of evidence-based practices across healthcare settings: a systematic integrative review. Syst Rev. 2018;7(1):1–19.

NOPREN. Food Insecurity Screening Algorithm for Pediatric Patients [Internet]. Vol. 2017. 2017. Available from: http://frac.org/wp-content/uploads/food-insecurity-screening-algorithm-pediatric-patients.pdf

Stake RE. The art of case study research. Thousand Oaks, CA: Sage; 1995.

Yin KR. Case study research and applications: design and methods. 6th ed. Thousand Oaks, CA: SAGE Publications Inc.; 2018.

Creswell JW, Poth CN. Qualitative Inquiry and Research Design: Choosing Among Five Approaches. 4th ed. Thousand Oaks, CA: SAGE Publications Inc.; 2018. 100, 148 p.

Gold R, Bunce A, Cottrell E, Marino M, Middendorf M, Cowburn S, et al. Study protocol: A pragmatic, stepped-wedge trial of tailored support for implementing social determinants of health documentation/action in community health centers, with realist evaluation. Implement Sci [Internet]. 2019 Jan 28 [cited 2021 Feb 26];14(1):1–17. Available from: doi: https://doi.org/10.1186/s13012-019-0855-9 .

Safaeinili N, Brown-Johnson C, Shaw JG, Mahoney M, Winget M. CFIR simplified: Pragmatic application of and adaptations to the Consolidated Framework for Implementation Research (CFIR) for evaluation of a patient-centered care transformation within a learning health system. Learn Heal Syst [Internet]. n/a(n/a):e10201. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1002/lrh2.10201

Ferlie EB, Shortell SM. Improving the quality of health Care in the United Kingdom and the United States: a framework for change. Milbank Q. 2001;79(2):281–315.

Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu Rev Public Health. 2014;35:29–45.

Patton MQ. Qualitative research and evaluation methods. 4th ed. Thousand Oaks, CA: SAGE Publications, Inc.; 2015.

Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol [Internet]. 2013;13:117. Available from: https://www.ncbi.nlm.nih.gov/pubmed/24047204 .

Haynes A, Brennan S, Redman S, Williamson A, Gallego G, Butow P. Figuring out fidelity: A worked example of the methods used to identify, critique and revise the essential elements of a contextualised intervention in health policy agencies. Implement Sci [Internet]. 2016 Feb 24 [cited 2021 Apr 1];11(1):23. Available from: http://www.implementationscience.com/content/11/1/23

Community Needs Assessment Central Region CHNA. Chicago, IL: Illinois Public Health Institute; 2016.

Stenmark SH, Steiner JF, Marpadga S, Debor M, Underhill K, Seligman H. Lessons Learned from Implementation of the Food Insecurity Screening and Referral Program at Kaiser Permanente Colorado. Perm J [Internet]. 2018 [cited 2021 May 25];22:18–093. Available from: https://pubmed.ncbi.nlm.nih.gov/30296400/

Beets MW, Weaver RG, Ioannidis JPA, Geraci M, Brazendale K, Decker L, et al. Identification and evaluation of risk of generalizability biases in pilot versus efficacy/effectiveness trials: A systematic review and meta-analysis [Internet]. Vol. 17, International Journal of Behavioral Nutrition and Physical Activity. BioMed Central Ltd.; 2020 [cited 2021 May 28]. p. 1–20. Available from: https://link.springer.com/articles/10.1186/s12966-020-0918-y

Damschroder J. L, Aron C. D, Keith E. R, Kirsh R. S, Alexander A. J, Lowery C. J. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science.

Byhoff E, Freund KM, Garg A. Accelerating the Implementation of Social Determinants of Health Interventions in Internal Medicine. J Gen Intern Med [Internet]. 2017 [cited 2021 Jun 16];33(2):223–8. Available from: www.who.int/social_determinants/en/ ,

De Marchis EH, Hessler D, Fichtenberg C, Fleegler EW, Huebschmann AG, Clark CR, et al. Assessment of social risk factors and interest in receiving health care-based social assistance among adult patients and adult caregivers of pediatric patients. JAMA Netw Open. 2020;1:3(10).

Baumann AA, Cabassa LJ. Reframing implementation science to address inequities in healthcare delivery. BMC Health Serv Res [Internet]. 2020 Mar 12 [cited 2020 Apr 21];20(1):190. Available from: https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-020-4975-3

Stirman SW, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci. 2019;6:14(1).

Martin-Misener R, Valaitis R, Wong ST, Macdonald M, Meagher-Stewart D, Kaczorowski J, et al. A scoping literature review of collaboration between primary care and public health. Prim Heal Care Res Dev [Internet]. 2012 [cited 2021 May 11];13(4):327–46. Available from: http://strengthenphc.mcmaster.cahttps//www.cambridge.org/core/terms . 10.1017/S1463423611000491, Downloaded from https://www.cambridge.org/core

Franz BA, Murphy JW. The Patient-Centered Medical Home as a Community-based Strategy. Perm J [Internet]. 2017 [cited 2021 Jun 1];21:17. Available from: /pmc/articles/PMC5528840/.

Stake RE. Multiple case study analysis. New York, NY: The Guilford Press; 2006.

Download references

Acknowledgements

The authors would like to thank all study participants for taking time out of their busy schedules to contribute to this study. We would like to thank Kathy Chan and Lena Hatchett for their assistance with participant recruitment.

Availability of data

Datasets from this study are not publicly available due to institutional review board regulations, but selective, di-identified, aggregated data may be made available upon reasonable request. Please contact corresponding author Sabira Taher at [email protected] .

This study was funded by an internal dissertation research award funded by the researcher’s academic institution. The funding body played no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Author information

Authors and affiliations.

Department of Preventive Medicine, Northwestern Feinberg School of Medicine, 680 N Lake Shore Drive Suite 1400, Chicago, IL, 60611, USA

Sabira Taher & C. Fagen Michael

Department of Community Health Sciences, School of Public Health, University of Illinois at Chicago, 1603 W. Taylor Street, Chicago, IL, 60612, USA

Naoko Muramatsu & Nadine Peacock

Division of Nutritional Sciences, College of Human Ecology, Cornell University, Martha Van Rensselaer Hall, Ithaca, NY, 14853, USA

Angela Odoms-Young

Department of Family Medicine, Loyola Stritch School of Medicine, 2160 S 1st Ave, Maywood, IL, 60153, USA

K. Suh Courtney

You can also search for this author in PubMed   Google Scholar

Contributions

ST led the conception and design of the study, development of study instruments, conducted interviews, data analysis, data interpretation and led manuscript writing. NM assisted with critical revisions of the manuscript for content. AOY assisted with developing the study design, data interpretation and critical revisions of the manuscript. MF, NP, and CS assisted with data interpretation and critical review of the manuscript. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Sabira Taher .

Ethics declarations

Ethical approval and consent to participate.

All procedures in this study involving human subjects were in accordance with ethical standards of the University of Illinois at Chicago Office for the Protection of Research Subjects Institutional Review Board and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This study and the verbal consent process were approved for a claim of exemption (Protocol # 2019–0610) from the University of Illinois at Chicago Office for the Protection of Research Subjects Institutional Review Board on August 30, 2019. Informed, verbal consent was obtained from all participants included in the study. Study findings were reported using the Standards for Reporting Qualitative Research.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1., additional file 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Taher, S., Muramatsu, N., Odoms-Young, A. et al. An embedded multiple case study: using CFIR to map clinical food security screening constructs for the development of primary care practice guidelines. BMC Public Health 22 , 97 (2022). https://doi.org/10.1186/s12889-021-12407-y

Download citation

Received : 05 August 2021

Accepted : 10 December 2021

Published : 14 January 2022

DOI : https://doi.org/10.1186/s12889-021-12407-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Food security screening
• Implementation
• Dissemination
• Primary care practice
• Semi-structured interviews
• Produce prescription programs

BMC Public Health

ISSN: 1471-2458

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Improving the Evaluation of Scholarly Work pp 65–80 Cite as

Case Research and Theory in Service Research

• Cristina Mele 4 ,
• Marialuisa Marzullo 4 ,
• Montserrat Díaz-Méndez 5 &
• Evert Gummesson 6  
• First Online: 09 November 2022

135 Accesses

1 Citations

Research methodology is a set of procedures that scholars follow to address their studies and ensure valid and reliable results. Choosing a suitable methodological approach is essential for the research process and represents one of the most challenging decisions for scholars. Case studies research assumes a key role in the debate between qualitative and quantitative methods. A manageable step forward to addressing complexity is offered by the narrative case study that interprets and makes sense of stories told by individuals. A further recent extension of case study research coming from the need to include two theoretical approaches that face complexity more systematically and structured: network theory and systems theory, led to the definition of case theory. Case theory offers higher validity and relevance by focusing on the outcome instead of details of the research process and techniques to augment reliability and rigor. Due to its characteristics, case theory is suitable to the service research and could contribute to new theoretical development.

• Case theory
• Narrative case
• Qualitative method
• Interpretive research

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Alam, M. K. (2020). A systematic qualitative case study: Questions, data collection, NVivo analysis and saturation. Qualitative Research in Organisations and Management: An International Journal, 16 (1), 15–31.

Google Scholar  

Andersen, P. H., Dubois, A., & Lind, F. (2018). Process validation: Coping with three dilemmas in process-based single-case research. Journal of Business & Industrial Marketing., 39 (1), 49–55.

Article   Google Scholar  

Antwi, S. K., & Hamza, K. (2015). Qualitative and quantitative research paradigms in business research: A philosophical reflection. European Journal of Business and Management, 7 (3), 217–225.

Azorín, J. M., & Cameron, R. (2010). The application of mixed methods in organisational research: A literature review. Electronic Journal of Business Research Methods, 8 (2), 95–105.

Benasso, S., Palumbo, M., & Pandolfini, V. (2019). Narrating cases: A storytelling approach to case study analysis in the field of lifelong learning policies. Italian Journal of Sociology of Education, 11 (2), 21–35.

Borghini, S., Carù, A., & Cova, B. (2010). Representing BtoB reality in case study research: Challenges and new opportunities. Industrial Marketing Management, 39 (1), 16–24.

Botturi, D., Curcio Rubertini, B., Desmarteau, R. H., & Lavalle, T. (2015). Investing in social capital in Emilia-Romagna region of Italy as a strategy for making public health work. In C. D. Johnson (Ed.), Social capital: Global perspectives, management strategies and effectiveness (pp. 197–219). Nova Science Publishers, Inc.

Bucic, T., Robinson, L., & Ramburuth, P. (2010). Effects of leadership style on team learning. Journal of Workplace Learning., 22 (4), 228–248.

Coffey, A., & Atkinson, P. (1996). Making sense of qualitative data: Complementary research strategies . Sage Publications, Inc.

Czarniawska, B. (2004). Narratives in social science research. Introducing qualitative methods . Sage Publications.

Denzin, N. K. (1989). Interpretive biography . Sage.

Dilthey, W. (1883). Introduction to the human sciences. In H. P. Richman (Ed.), W. Dilthey: Selected writings (pp. 157–263). London: Cambridge University Press

Dubois, A., & Araujo, L. (2004). Research methods in industrial marketing studies. In H. Håkansson (Ed.), Rethinking marketing: Developing a new understanding of markets (pp. 207–227). Wiley.

Durkheim, E. (1960). Montesquieu’s contribution to the rise of social science. In Montesquieu and Rousseau: Forerunners of sociology . Translated by R. Man- heim with a foreword by H. Peyre (pp. 1–64). University of Michigan Press.

Edmondson, A. C., & McManus, S. E. (2007). Methodological fit in management field research. Academy of Management Review, 32 (4), 1246–1264.

Eisenhardt, K. M., & Graebner, M. E. (2007). Theory building from cases: Opportunities and challenges. Academy of Management Journal, 50 (1), 25–32.

Fischer, M. (2015). Fit for the future? A new approach in the debate about what makes healthcare systems really sustainable. Sustainability, 7 , 294–312.

Flyvbjerg, B. (2006). Five misunderstandings about case-study research. Qualitative Inquiry, 12 (2), 219–245.

George, A. L., & Bennett, A. (2005). Case studies and theory development in the social sciences . MA, MIT Press.

Gillham, B. (2000). Case study research methods . Bloomsbury Publishing.

Gomm, R., Hammersley, M., & Foster, P. (Eds.). (2000). Case study method: Key issues, key texts . Sage.

Goulding, C. (2002). Grounded theory: A practical guide for management, business and market researchers . Sage.

Book   Google Scholar  

Gray, D. E. (2021). Doing research in the real world . Sage.

Gummesson, E. (2002). Relationship marketing and a new economy: It’s time for de‐programming. Journal of services marketing, 16 (7), 585–589.

Gummesson, E. (2007). Case study research and network theory: Birds of a feather. Qualitative Research in Organisations and Management, 2 (3), 226–248.

Gummesson, E. (2014). Service research methodology: From case study research to case theory. Revista Ibero Americana De Estratégia, 13 (4), 08–17.

Gummesson, E. (2015). Innovative case study research in business and management . Sage.

Gummesson, E. (2017). Case theory in business and management: Reinventing case study research . Sage.

Gummesson, E., Mele, C., & Polese, F. (2019). Complexity and viability in service ecosystems. Marketing Theory, 19 (1), 3–7.

Halkier, B. (2011). Methodological practicalities in analytical generalisation. Qualitative Inquiry, 17 (9), 787–797.

Harrison, R. L., III. (2013). Using mixed methods designs in the Journal of Business Research, 1990–2010. Journal of Business Research, 66 (11), 2153–2162.

Helkkula, A. (2010). Service experience in an innovation context . Hanken School of Economics

Helkkula, A., Kelleher, C., & Pihlström, M. (2012). Characterising value as an experience: Implications for service researchers and managers. Journal of Service Research, 15 (1), 59–75.

Hitt, M. A., Hoskisson, R. E., & Ireland, R. D. (1994). A mid-range theory of the interactive effects of international and product diversification on innovation and performance. Journal of Management, 20 (2), 297–326.

Johnston, W. J., Leach, M. P., & Liu, A. H. (1999). Theory testing using case studies in business-to-business research. Industrial Marketing Management, 28 (3), 201–213.

King, G., R. Keohane and S. Verba (1994) Designing Social Enquiry: Scientific Inference in Qualitative Research. Princeton, NJ: Princeton University Press

Knights, D., & McCabe, D. (1997). ‘How would you measure something like that?’: Quality in a retail bank. Journal of Management Studies, 34 (3), 371–388.

Lee, B., Collier, P. M., & Cullen, J. (2007). Reflections on the use of case studies in the accounting, management and organisational disciplines. Qualitative Research in Organisations and Management: An International Journal., 2 (3), 169–178.

Mathisen, L. I. N. E., & Chen, J. S. (2014). Storytelling in a co-creation perspective. In J. S. Chen, M. Uysal, & N. K. Prebensen (Eds.), Co-creation of experience value—A tourist behavior approach . CAB International (in Press).

Modell, S. (2010). Bridging the paradigm divide in management accounting research: The role of mixed methods approaches. Management Accounting Research, 21 (2), 124–129.

Närvänen, E., Gummesson, E., & Kuusela, H. (2014). The collective consumption network. Managing Service Quality, 24 (6), 545–564.

Neuman, S. B., & Dickinson, D. K. (Eds.). (2003). Handbook of early literacy research (Vol. 1, pp. 11–29). Guilford Press.

Nie, Y. (2017). Combining narrative analysis, grounded theory and qualitative data analysis software to develop a case study research. Journal of Management Research, 9 (2), 53–70.

Parlett, M., Hamilton, D. (1976) ‘Evaluation as illumination: A new approach to the study of innovative programs’. In D. A. Tawney (Ed.), Curriculum evaluation today: Trends and implications . Macmillan Education.

Patton, M. Q. (1990). Qualitative evaluation and research methods (p. 532). SAGE Publications, Inc.

Perry, C. (1998). Processes of a case study methodology for postgraduate research in marketing. European Journal of Marketing., 32 (9/10), 785–802.

Piekkari, R., Plakoyiannaki, E., & Welch, C. (2010). ‘Good’ case research in industrial marketing: Insights from research practice. Industrial Marketing Management, 39 (1), 109–117.

Pike, K. L. (1967). Etic and emic standpoints for the description of behavior. In D. C. Hildum (Ed.), Language and thought: An enduring problem in psychology (pp. 32–39). Van Nostrand.

Reis, H. T., & Judd, C. M. (Eds.). (2000). Handbook of research methods in social and personality psychology . Cambridge University Press.

Ridder, H. G. (2017). The theory contribution of case study research designs. Business Research, 10 (2), 281–305.

Riessman, C. K. (1993). Narrative analysis (Vol. 30). Sage.

Rossman, G. B., & Wilson, B. L. (1994). Numbers and words revisited: Being “shamelessly eclectic.” Quality and Quantity, 28 (3), 315–327.

Roller, M. R., & Lavrakas, P. J. (2015). Applied qualitative research design: A total quality framework approach . Guilford Publications.

Russo Spena, T., & Mele, C. (2019). Practising innovation in the healthcare ecosystem: The agency of third-party actors. The Journal of Business and Industrial Marketing, 35 (3), 390–403.

Saunders, M. N., & Bezzina, F. (2015). Reflections on conceptions of research methodology among management academics. European Management Journal, 33 (5), 297–304.

Siltaloppi, J., & Vargo, S. L. (2017). Triads: A review and analytical framework. Marketing Theory, 17 (4), 395–414.

Simons, H. (2014). Case study research: In-depth understanding in context. The Oxford handbook of qualitative research (pp. 455–470).

Sommer Harrits, G. (2011). More than method?: A discussion of paradigm differences within mixed methods research. Journal of Mixed Methods Research, 5 (2), 150–166.

Stake, R. E. (1995). The art of case study research . Sage Publications, Inc.

Stake, R. (1998). Case studies. In N. Denzin, Y. Lincoln (Eds.), Strategies of qualitative inquiry . Sage.

Stake, R. E. (2000). The case study and generalizability. In R. Gomm, M. Hammersley, and P. Foster (Eds.), Case study method. Key issues, key texts (pp. 19–26). Sage Publications.

Stake, R. E. (2013). Multiple case study analysis . Guilford press.

Taleb, N. N. (2007). Black swans and the domains of statistics. The American Statistician, 61 (3), 198–200.

Thyer, B. A. (2001). What is the role of theory in research on social work practice? Journal of Social Work Education, 37 (1), 9–25.

Tight, M. (2010). The curious case of case study: A viewpoint. International Journal of Social Research Methodology, 13 (4), 329–339.

Tsoukas, H., & Hatch, M. J. (2001). Complex thinking, complex practice: The case for a narrative approach to organisational complexity. Human Relations, 54 (8), 979–1013.

Turner, J. H. (2001). The origins of positivism: The contributions of Auguste Comte and Herbert Spencer. In Handbook of social theory (pp. 30–42). Sage Publications.

Weber, M. (1904). Die" Objektivität" sozialwissenschaftlicher und sozialpolitischer Erkenntnis. Archiv Für Sozialwissenschaft Und Sozialpolitik, 19 (1), 22–87.

Webster, L., & Mertova, P. (2007). Using narrative inquiry as a research method: An introduction to using critical event narrative analysis in research on learning and teaching . Routledge.

Welch, C., Piekkari, R., Plakoyiannaki, E., & Paavilainen-Ma¨ntyma¨ki, E. (2011). Theorising from case studies: Towards a pluralist future for international business research. Journal of International Business Studie, 42 , 740–762.

Westerman, M. A. (2011). Defenses in interpersonal interaction: Using a theory-building case study to develop and validate the theory of interpersonal defense. Pragmatic Case Studies in Psychotherapy, 7 (4), 449–476.

Woodside, A. G., & Wilson, E. J. (2003). Case study research methods for theory building. Journal of Business & Industrial Marketing, 18 (6/7), 493–508.

Yin, R. K. (2003). Case study research (3rd ed.). Sage Publications.

Yin, R. K. (Ed.). (2004). The case study anthology . Thousand Oaks.

Yin, R. K. (2014). Case study research design and methods (5th ed.). Sage Publications, Inc.

Download references

Author information

Authors and affiliations.

University of Naples Federico II, Naples, Italy

Cristina Mele & Marialuisa Marzullo

University of Extremadura, Badajoz, Spain

Montserrat Díaz-Méndez

University of Stockholm, Stockholm, Sweden

Evert Gummesson

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Cristina Mele .

Editor information

Editors and affiliations.

Stockholm Business School, Stockholm University, Stockholm, Sweden

Faculty of Economics and Business, University of Extremadura, Badajoz, Spain

University of Birmingham, Birmingham, UK

Michael Saren

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this chapter

Cite this chapter.

Mele, C., Marzullo, M., Díaz-Méndez, M., Gummesson, E. (2022). Case Research and Theory in Service Research. In: Gummesson, E., Díaz-Méndez, M., Saren, M. (eds) Improving the Evaluation of Scholarly Work. Springer, Cham. https://doi.org/10.1007/978-3-031-17662-3_5

Download citation

DOI : https://doi.org/10.1007/978-3-031-17662-3_5

Published : 09 November 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-17661-6

Online ISBN : 978-3-031-17662-3

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Section 2: Home
• Developing the Quantitative Research Design
• Qualitative Descriptive Design
• Design and Development Research (DDR) For Instructional Design
• Qualitative Narrative Inquiry Research
• Action Research Resource
• Case Study Design in an Applied Doctorate

Qualitative Research Designs

Case study design, using case study design in the applied doctoral experience (ade), applicability of case study design to applied problem of practice, case study design references.

• SAGE Research Methods
• Research Examples (SAGE) This link opens in a new window
• Dataset Examples (SAGE) This link opens in a new window
• IRB Resource Center This link opens in a new window

The field of qualitative research there are a number of research designs (also referred to as “traditions” or “genres”), including case study, phenomenology, narrative inquiry, action research, ethnography, grounded theory, as well as a number of critical genres including Feminist theory, indigenous research, critical race theory and cultural studies. The choice of research design is directly tied to and must be aligned with your research problem and purpose. As Bloomberg & Volpe (2019) explain:

Choice of research design is directly tied to research problem and purpose. As the researcher, you actively create the link among problem, purpose, and design through a process of reflecting on problem and purpose, focusing on researchable questions, and considering how to best address these questions. Thinking along these lines affords a research study methodological congruence (p. 38).

Case study is an in-depth exploration from multiple perspectives of a bounded social phenomenon, be this a social system such as a program, event, institution, organization, or community (Stake, 1995, 2005; Yin, 2018). Case study is employed across disciplines, including education, health care, social work, sociology, and organizational studies. The purpose is to generate understanding and deep insights to inform professional practice, policy development, and community or social action (Bloomberg 2018).

Yin (2018) and Stake (1995, 2005), two of the key proponents of case study methodology, use different terms to describe case studies. Yin categorizes case studies as exploratory or descriptive . The former is used to explore those situations in which the intervention being evaluated has no clear single set of outcomes. The latter is used to describe an intervention or phenomenon and the real-life context in which it occurred. Stake identifies case studies as intrinsic or instrumental , and he proposes that a primary distinction in designing case studies is between single and multiple (or collective) case study designs. A single case study may be an instrumental case study (research focuses on an issue or concern in one bounded case) or an intrinsic case study (the focus is on the case itself because the case presents a unique situation). A longitudinal case study design is chosen when the researcher seeks to examine the same single case at two or more different points in time or to capture trends over time. A multiple case study design is used when a researcher seeks to determine the prevalence or frequency of a particular phenomenon. This approach is useful when cases are used for purposes of a cross-case analysis in order to compare, contrast, and synthesize perspectives regarding the same issue. The focus is on the analysis of diverse cases to determine how these confirm the findings within or between cases, or call the findings into question.

Case study affords significant interaction with research participants, providing an in-depth picture of the phenomenon (Bloomberg & Volpe, 2019). Research is extensive, drawing on multiple methods of data collection, and involves multiple data sources. Triangulation is critical in attempting to obtain an in-depth understanding of the phenomenon under study and adds rigor, breadth, and depth to the study and provides corroborative evidence of the data obtained. Analysis of data can be holistic or embedded—that is, dealing with the whole or parts of the case (Yin, 2018). With multiple cases the typical analytic strategy is to provide detailed description of themes within each case (within-case analysis), followed by thematic analysis across cases (cross-case analysis), providing insights regarding how individual cases are comparable along important dimensions. Research culminates in the production of a detailed description of a setting and its participants, accompanied by an analysis of the data for themes or patterns (Stake, 1995, 2005; Yin, 2018). In addition to thick, rich description, the researcher’s interpretations, conclusions, and recommendations contribute to the reader’s overall understanding of the case study.

Analysis of findings should show that the researcher has attended to all the data, should address the most significant aspects of the case, and should demonstrate familiarity with the prevailing thinking and discourse about the topic. The goal of case study design (as with all qualitative designs) is not generalizability but rather transferability —that is, how (if at all) and in what ways understanding and knowledge can be applied in similar contexts and settings. The qualitative researcher attempts to address the issue of transferability by way of thick, rich description that will provide the basis for a case or cases to have relevance and potential application across a broader context.

Qualitative research methods ask the questions of "what" and "how" a phenomenon is understood in a real-life context (Bloomberg & Volpe, 2019). In the education field, qualitative research methods uncover educational experiences and practices because qualitative research allows the researcher to reveal new knowledge and understanding. Moreover, qualitative descriptive case studies describe, analyze and interpret events that explain the reasoning behind specific phenomena (Bloomberg, 2018). As such, case study design can be the foundation for a rigorous study within the Applied Doctoral Experience (ADE).

Case study design is an appropriate research design to consider when conceptualizing and conducting a dissertation research study that is based on an applied problem of practice with inherent real-life educational implications. Case study researchers study current, real-life cases that are in progress so that they can gather accurate information that is current. This fits well with the ADE program, as students are typically exploring a problem of practice. Because of the flexibility of the methods used, a descriptive design provides the researcher with the opportunity to choose data collection methods that are best suited to a practice-based research purpose, and can include individual interviews, focus groups, observation, surveys, and critical incident questionnaires. Methods are triangulated to contribute to the study’s trustworthiness. In selecting the set of data collection methods, it is important that the researcher carefully consider the alignment between research questions and the type of data that is needed to address these. Each data source is one piece of the “puzzle,” that contributes to the researcher’s holistic understanding of a phenomenon. The various strands of data are woven together holistically to promote a deeper understanding of the case and its application to an educationally-based problem of practice.

Research studies within the Applied Doctoral Experience (ADE) will be practical in nature and focus on problems and issues that inform educational practice.  Many of the types of studies that fall within the ADE framework are exploratory, and align with case study design. Case study design fits very well with applied problems related to educational practice, as the following set of examples illustrate:

Elementary Bilingual Education Teachers’ Self-Efficacy in Teaching English Language Learners: A Qualitative Case Study

The problem to be addressed in the proposed study is that some elementary bilingual education teachers’ beliefs about their lack of preparedness to teach the English language may negatively impact the language proficiency skills of Hispanic ELLs (Ernst-Slavit & Wenger, 2016; Fuchs et al., 2018; Hoque, 2016). The purpose of the proposed qualitative descriptive case study was to explore the perspectives and experiences of elementary bilingual education teachers regarding their perceived lack of preparedness to teach the English language and how this may impact the language proficiency of Hispanic ELLs.

Exploring Minority Teachers Experiences Pertaining to their Value in Education: A Single Case Study of Teachers in New York City

The problem is that minority K-12 teachers are underrepresented in the United States, with research indicating that school leaders and teachers in schools that are populated mainly by black students, staffed mostly by white teachers who may be unprepared to deal with biases and stereotypes that are ingrained in schools (Egalite, Kisida, & Winters, 2015; Milligan & Howley, 2015). The purpose of this qualitative exploratory single case study was to develop a clearer understanding of minority teachers’ experiences concerning the under-representation of minority K-12 teachers in urban school districts in the United States since there are so few of them.

Exploring the Impact of an Urban Teacher Residency Program on Teachers’ Cultural Intelligence: A Qualitative Case Study

The problem to be addressed by this case study is that teacher candidates often report being unprepared and ill-equipped to effectively educate culturally diverse students (Skepple, 2015; Beutel, 2018). The purpose of this study was to explore and gain an in-depth understanding of the perceived impact of an urban teacher residency program in urban Iowa on teachers’ cultural competence using the cultural intelligence (CQ) framework (Earley & Ang, 2003).

Qualitative Case Study that Explores Self-Efficacy and Mentorship on Women in Academic Administrative Leadership Roles

The problem was that female school-level administrators might be less likely to experience mentorship, thereby potentially decreasing their self-efficacy (Bing & Smith, 2019; Brown, 2020; Grant, 2021). The purpose of this case study was to determine to what extent female school-level administrators in the United States who had a mentor have a sense of self-efficacy and to examine the relationship between mentorship and self-efficacy.

Suburban Teacher and Administrator Perceptions of Culturally Responsive Teaching to Promote Connectedness in Students of Color: A Qualitative Case Study

The problem to be addressed in this study is the racial discrimination experienced by students of color in suburban schools and the resulting negative school experience (Jara & Bloomsbury, 2020; Jones, 2019; Kohli et al., 2017; Wandix-White, 2020). The purpose of this case study is to explore how culturally responsive practices can counteract systemic racism and discrimination in suburban schools thereby meeting the needs of students of color by creating positive learning experiences. 

As you can see, all of these studies were well suited to qualitative case study design. In each of these studies, the applied research problem and research purpose were clearly grounded in educational practice as well as directly aligned with qualitative case study methodology. In the Applied Doctoral Experience (ADE), you will be focused on addressing or resolving an educationally relevant research problem of practice. As such, your case study, with clear boundaries, will be one that centers on a real-life authentic problem in your field of practice that you believe is in need of resolution or improvement, and that the outcome thereof will be educationally valuable.

Bloomberg, L. D. (2018). Case study method. In B. B. Frey (Ed.), The SAGE Encyclopedia of educational research, measurement, and evaluation (pp. 237–239). SAGE. https://go.openathens.net/redirector/nu.edu?url=https%3A%2F%2Fmethods.sagepub.com%2FReference%2Fthe-sage-encyclopedia-of-educational-research-measurement-and-evaluation%2Fi4294.xml

Bloomberg, L. D. & Volpe, M. (2019). Completing your qualitative dissertation: A road map from beginning to end . (4th Ed.). SAGE.

Stake, R. E. (1995). The art of case study research. SAGE.

Stake, R. E. (2005). Qualitative case studies. In N. K. Denzin and Y. S. Lincoln (Eds.), The SAGE handbook of qualitative research (3rd ed., pp. 443–466). SAGE.

Yin, R. (2018). Case study research and applications: Designs and methods. SAGE.

• << Previous: Action Research Resource
• Next: SAGE Research Methods >>
• Last Updated: Jul 28, 2023 8:05 AM
• URL: https://resources.nu.edu/c.php?g=1013605

• Research article
• Open access
• Published: 23 October 2020

Advancing integrated care evaluation in shifting contexts: blending implementation research with case study design in project SUSTAIN

• Jenny Billings   ORCID: orcid.org/0000-0002-5660-9478 1 ,
• Simone R. de Bruin 2 ,
• Caroline Baan 2 &
• Giel Nijpels 3  

BMC Health Services Research volume  20 , Article number:  971 ( 2020 ) Cite this article

2118 Accesses

4 Citations

Metrics details

Despite many studies evaluating the effectiveness of integrated care, evidence remains inconsistent. There is increasing commentary pointing out the mismatch between the ability to capture the somewhat ‘illusive’ impact of integrated care initiatives and programmes, and the most appropriate way to do this. Focusing on methodology, this paper describes and critically reviews the experiences of SUSTAIN, a Horizon 2020 funded project (2015–2019) with the purpose of advancing knowledge and understanding of cross-European integrated care evaluation. SUSTAIN sought to improve integrated care initiatives for older people in seven countries, and to maximise the potential for knowledge transfer and application across Europe. The methods approach drew from implementation research, employing the participative Evidence Integration Triangle (EIT) and incorporating a mixed method, multiple embedded case study design. A core set of qualitative and quantitative indicators, alongside context and process data, were created and tested within four key project domains (person-centredness, prevention-orientation, safety and efficiency). The paper critically discusses the overall approach, highlighting the value of the EIT and case study design, and signalling the challenges of data collection with frail older people and stakeholder involvement at the sites, as well as difficulties developing the core set of indicators.

Conclusions

Lessons learned and recommendations for advancing integrated care evaluation are put forward that focus on the status of integrated care as a complex intervention and a process. The use of implementation research methods and case study design are recommended as an additional evaluation approach for researchers to consider, alongside suggested ways of improving methods of data collection with frail populations and cost analysis.

Peer Review reports

Despite many studies evaluating the effectiveness of integrated care, evidence remains inconsistent. Increasingly, commentary on the subject of integrated care evaluation is pointing out the mismatch between the ability to capture the impact of integrated care initiatives and programmes, and the selection of the most appropriate methodology to do this. Authors have highlighted a range of evaluation challenges that include the stability and sustainability of initiatives; data collection and the suitability of measures [ 1 , 2 ]; and a lack of appreciation of the complexity [ 3 ]. In addition, the status of integrated care as a ‘process’ must be recognised [ 4 ], meaning it is not a ‘fixed’ intervention, but susceptible to constant development and change. These factors all affect the sturdiness of evaluation designs and what constitutes an outcome. This in turn is prompting the need to fit the evaluation design more with how integrated care is implemented in practice and what integrated care is there to achieve and improve [ 5 , 6 ]. The value of mixed methods studies and suitable frameworks that examine both processes and outcomes has therefore been recognised in this field [ 2 , 7 ].

Regarding the examination of processes, the wide variation in how integrated care is operationalised calls for evaluations that include a range of qualitative methods, so that important contextual information can be examined to identify what seems to work and why. Regarding outcomes, there is a need to ensure that outcome measures have a good pragmatic fit with the shifting context of integrated care interventions and the population group under study. There is a tendency for example for measures for frail older people and people with multimorbidity to focus on general health outcomes (e.g. health status, physical functioning, quality of life), which may not be appropriate to their fluctuating physical and mental status. Outcomes such as experiences of care, independence and autonomy may be more suited to this vulnerable target group, and these inclusions may be more appropriate to ascertain the link between the integrated care processes and what improvements can be expected for the service user in receipt of care.

Given this impetus, researchers are adopting more ‘real world’ methodologies for the evaluation of complex interventions such as integrated care. While mixed methodologies have been advocated for some time to gain a better appreciation of the ‘grass root’ processes involved in integrated care implementation [ 8 ], the emergence of realist approaches drawn from Pawson and Tilley (1997) [ 9 ] has become evident [ 10 , 11 ]. Realist researchers seek to explain the underlying “cause” or mechanisms that generate observed phenomena through the construction of context, mechanism and outcome (CMO) configurations [ 12 ]. To support this, academics are developing frameworks such as the COMIC model for the evaluation of integrated care [ 13 ]. In addition to realist methods, and continuing with the focus on context, researchers are turning to implementation research. This is described as the scientific study of the processes used in the implementation of an initiative alongside the context within which it is taking place [ 14 ]. Its intention is to promote the systematic uptake of research findings and other evidence-based practices into routine practice, and, hence, to improve the quality and effectiveness of health services and care [ 15 ].

This broadening of methods appeal is becoming reflected in funding opportunities. The European Union (EU) funded research initiatives (Horizon 2020) are encouraging the adoption of innovative approaches to the evaluation of integrated care to better understand the impact on vulnerable populations with complex needs. One such project is SUSTAIN – Sustainable Tailored Integrated Care for older people in Europe (2015–2019). This paper focuses on the use of innovative approaches within SUSTAIN, and describes the method by which implementation research and case study design were blended for the evaluation of European integrated care initiatives. The aim is to share methodological experiences and lessons learned with the research community, in order to advance understanding of integrated care evaluation in context and add to the international ‘toolbox’ of methodological approaches. It will commence with a brief introduction to SUSTAIN and an overview of the design. This is followed by a critical discussion of the strengths and weaknesses of our approach and concludes with an assessment of the extent to which we advanced understanding of integrated care evaluation. Lessons learned and recommendations for future integrated care evaluations are put forward.

Overview of the SUSTAIN project

The SUSTAIN project was carried out by thirteen partners from eight European countries: Austria, Belgium, Estonia, Germany, Norway, Spain, the Netherlands, and the United Kingdom. With the exception of Belgium, in all other countries two integrated care initiatives (also referred to as ‘sites’) per country were invited to participate in the SUSTAIN-project, each developing and evaluating two integrated care initiatives ( n  = 14) focusing on older people with complex needs. Sites that were recruited needed to have integrated care initiatives in place, but were motivated to improve and adapt their programmes.

The overall aim of SUSTAIN was to improve integrated care for older people, and to maximise the potential for knowledge transfer and application across Europe [ 16 ]. SUSTAIN had four main themes, pre-set by the Horizon 2020 funding programme of person-centredness, prevention-orientation, safety and efficiency in integrated care, which guided the development, implementation and evaluation of the initiatives. The objectives of SUSTAIN were to:

Support and evaluate improvements to established integrated care initiatives for older people over 65 living at home with multiple long-term conditions and complex needs; and.

Contribute to the adoption and application of these improvements to other health and social care systems, and regions in Europe.

The development and evaluation of SUSTAIN initiatives took place over an 18 month time span, with a 6 month phase of identifying the sites and creating improvement plans, followed by a year long implementation and evaluation period. Ethical approval for the evaluations was obtained through the site-specific governance structures. This paper focuses on the evaluation aspect.

Integrated care comes in many forms, and given that our focus was on developing improvements according to local needs, SUSTAIN has not been exempt from this variability in the selection of initiatives. However, despite the considerable differences in structure and context, two common approaches could be identified within our 14 initiatives;

Sites that aimed to improve services to older people and/or expand collaboration, communication and coordination with other care and support organisations, while also enhancing knowledge and understanding of each other’s roles and responsibilities;

Sites that aimed to improve the actual care delivery process to older people, for example providing care in a more person-centred way, or improving case- and discharge management [ 17 ].

The challenge for the evaluation design thus became one of developing a robust and consistent approach applicable across all country sites, in the face of several important variabilities. These included differing integrated care configurations and contexts, the site-specific pace of implementation, the throughput of service users and their length of exposure to the intervention, and the enduring problem of data accessibility and comparability [ 18 ].

Methods: the evaluation design of SUSTAIN

Evidence integration triangle.

The overall approach to the project was guided by the Evidence Integration Triangle (EIT). This participatory approach is derived from implementation research and aims to tackle the process of translating research and best-practice evidence to implementation [ 19 ].

While evidence on the contribution the EIT is making is still emerging, it is claimed that by focusing on the perspective of stakeholders and the context for application of scientific findings, pragmatic approaches can hasten the integration of research, policy, and practice [ 20 ].

There are three main components to the EIT model, namely the evidence-based intervention programme or policy, participatory implementation processes with stakeholder involvement, and practical measures of progress and outcome (Fig.  1 ). These three components enable stakeholders to use scientific evidence to encourage development and sharing of new knowledge to inform decision-making. In SUSTAIN, stakeholders were involved by organising steering group meetings. Steering groups consisted of local stakeholders (e.g. managers, health and social care professionals, representatives of older people and carers, commissioners, local policy officers) considered relevant to the integrated care initiative. These steering groups were created to design and implement improvement plans, that is, sets of improvements that apply to local, site-specific priorities.

Evidence Integration Triangle [ 19 ]

The high participation levels within this model enabled the research to be relevant and applicable from the onset and ensured that indicators and measures generated to gather evidence remain sensitive of the research and practice environment. Qualitative and quantitative evidence is accumulated and used throughout so that the change process remains dynamic and responsive to improvement.

Context is also pivotal to the EIT. Glasgow [ 19 ] describes the multilevel context as the conditions surrounding health problems and intervention opportunities in a particular place with a particular population, and is a key starting point. Context also changes over time, giving a dynamic aspect to the EIT, with context continually informing the other key components. It is clear that this approach was well suited to the fluctuating integrated care environments within which SUSTAIN was taking place.

• Case study design

Within the practical measures aspect of the EIT (the focus of the development of our evaluation tools and approach), we adopted Yin’s [ 21 ] case study design. A strength of case study design is its ability to support the analysis of multiple qualitative and quantitative data sources – described as ‘embedded’ - to investigate complex phenomena in their everyday contexts and across different contexts [ 22 ]. It was therefore deemed appropriate for examining implementation processes as they unfolded within the EIT cycle in our differing interventions. In addition, it allowed for multiple cases, taking into account the different types of intervention, data source availability and sample size variations across our study sites. As such, with the case study design we aimed to tackle several of the challenges in integrated care evaluation.

Cases are defined by a unit of analysis, common across all sites. With SUSTAIN, our unit of analysis became ‘set of improvements for integrated care initiatives’, as this was a core objective. In addition, we adopted an explanatory approach to our case study design [ 21 ], as we were seeking to develop explanatory models and greater theoretical understanding around two main propositions linked to the four SUSTAIN themes:

Integrated care activities will maintain or enhance person-centredness, prevention orientation, safety and efficiency in care delivery;

Explanations for succeeding in improving existing integrated care initiatives will be identified.

These propositions were accompanied by a number of analytical questions to support analysis, described in the analysis section. Thus Fig. 2 illustrates our overall approach - multiple embedded case study design that is explanatory in nature . In SUSTAIN we differentiated between qualitative and quantitative indicators, a requirement of the Horizon 2020 call. Both produce quantitative data but the former measures attitudes, perceptions and beliefs, and the latter focuses on audit-style data such as hospital admissions [ 23 ]. In addition, the figure includes the data sources and the minimum anticipated samples that were seen as achievable, gauged through discussion at partner sites and within the consortium as a whole, taking into consideration the variability previously mentioned such as the differing speed of service user throughput and variable length of the intervention.

Multiple Embedded Case study design showing data sources and planned samples per site and overall in 14 sites

Practical measures

A key feature of the design was to develop and test a core set of indicators that could be used across our partner countries and potentially be transferable to other areas. While Fig.  2 maps out the discreet data sources aligned to case study design, Table  1 unpacks our data sources further, describing distinct data items and data collection tools that were core to the evaluation of our sites in more detail. For clarity and linkage to Fig. 2 , qualitative and quantitative indicators are highlighted in colour under the data items column. For data collection tools, instrument selection depended upon the goodness of fit with our objectives and four key themes; availability especially in different languages; validation within our population group; and length. Sites also included some site-specific measures in addition to our core set, that were particular to their interventions, such as audit forms to track new general practice referrals (UK), numbers of GPs, nurses and social workers (Spain), and reasons for not using the integrated care centre for people with dementia (Austria). The selection of our core instruments is elaborated upon and critically reviewed in the discussion.

Data collection

In keeping with the EIT cycles and approach to rapid knowledge transfer, data collection took place over 1 year in two waves following a 6 month development phase where baseline information was collected. Stakeholder reviews of preliminary findings (at the 12 month period) and final findings (at the 18 month period) (Fig.  3 ) were built in through steering groups to ascertain what seemed to be working well, and where solutions to problems needed to be identified. In order to enable comparison, we used uniform procedures for data collection for all initiatives.

Data collection and Feedback plan

Analysis strategies within multiple case study designs are focused on triangulation of data, purported by Yin [ 21 ] to strengthen the construct validity of the research. Each data source became one piece of a jigsaw with each piece contributing to understanding of the whole phenomenon [ 25 ]. In SUSTAIN, within each site, data sources were analysed according to their requirements before proceeding to a specific analytical process [ 26 , 27 ]. Uniform templates for analysis of each data source were generated through a discussion among research partners. All data was entered on a shared anonymised database. Of Yin’s [ 21 ] five techniques for analysis, we adopted pattern-matching, seeking rival explanations, linking data to propositions, and explanation building. Exploring rival explanations is an attempt to scan the data to provide an alternate explanation of a phenomenon. To support this, a number of analytical questions were developed to underpin the propositions and our aims, and aid consistency of analytical focus among our evaluation partners:

What seems to work with what outcomes when making improvements to integrated care?

What are the explanations for succeeding and improving integrated care initiatives?

What are the explanations for NOT succeeding and improving integrated care initiatives?

Are there any factors that can be seen as having an impact on integrated care improvements?

What factors can be identified that could apply to integrated care improvements across the EU, and be transferable?

Once each site analysis was completed, an overarching cross case synthesis took place. Overall, the evidence created from this type of analysis is considered robust and reliable [ 28 ].

Discussion: addressing the challenges

The discussion will critically review the SUSTAIN design and its appropriateness as an approach to integrated care evaluation. It will firstly discuss the overarching evaluation approach that incorporated the EIT and case study design, follow with a critical reflection on the development of a core set of indicators, and debate the choice of our design in the context of other suitable approaches, namely realistic evaluation.

Reflecting on the EIT, it proved to be highly suitable as a framework for implementation and evaluation for SUSTAIN, in its applicability and use in real-life contexts. A key feature was its practical ability to support a participative environment through the steering group meetings. Here, the framework promoted engagement through its ability to portray a logical and straightforward approach to implementation and evaluation, enabling members to proactively deal with contextual and hence evaluation challenges. It also enabled a level of knowledge exchange and action between the researchers and stakeholders. Other studies are similarly incorporating ‘fit for purpose’ research designs that are placed within the EIT framework. Carrieri et al [ 29 ] for example, are undertaking a realist review of interventions to tackle doctors’ mental health, using the EIT to convene a stakeholder group with experts in which research can inform practical decision-making and dissemination of messages. Also Resnik et al [ 30 ] are testing the EIT for implementation of interventions to manage behavioural and psychological symptoms associated with dementia, incorporating a pragmatic trial.

Similarly, case study design with its inherent flexibility provided a sound basis for harmonising the disparities between sites and provided a platform to test our core indicators. Yin [ 21 ] and Cresswell [ 31 ] promote the usefulness of this embedded multi-method design for its ability to add or remove data sources without detriment to the overall analysis. Case study design also gave us a solid data analysis strategy that could accommodate and make comparable and meaningful discrepancies across our partner countries. Case studies have been used in clinical practice and research for a number of decades in complex settings including integrated care [ 27 , 32 ], as well as within an implementation science approach [ 33 ] and in EU studies [ 34 ].

In addition, the incorporation of case study design was a significant addition to theory building opportunities (which is somewhat lacking in EIT – see later discussion), going some way towards assembling a deeper theoretical understanding of integrated care. Eisenhardt & Graebner (2007) [ 35 ] suggest that theory is emergent, situated in and developed by recognising patterns of relationships among constructs within and across cases. The use of replication logic assisted by pattern matching assists with theory building, in that multiple cases serve as replications, contrasts and extension to the emerging theories. Within SUSTAIN, case study design supported the development of our propositions and consequent explanatory models. Theories embodied within the propositions could be tested and expounded, ultimately leading to theory building, in relation to our central concepts of person-centredness, prevention orientation, safety and efficiency in care delivery, and what seems to ‘work’ in integrated care improvements (see SUSTAIN final report De Bruin et al. 2018) [ 36 ].

However our implementation research approach could be described as overly simplistic and lacking clear steps to achieve certain EIT goals, which leaves it open to interpretation. In addition, more guidance is needed regarding how each triangle component relate to each other, as well as how the evidence and stakeholders’ input connect to the triangle individually and as a whole. Importantly, it does not describe sufficiently well how the different context levels should be situated within the triangle and how, within a constantly changing environment, it misses out consideration of sustainability of the intervention. But, the simplicity of the EIT could be described as a strength, in that it is understandable and accessible by participants in the real world, vital in the highly participative stance of the framework. Given the variability within our projects and integrated care interventions generally, the lack of clear process information generated better ‘bottom up’ plans about how the components would work together and relate to the intervention as a whole. Indeed, Glasgow et al. (2012) see other knowledge translation models as too complicated, academic or time consuming for those who wish to use the evidence. In contrast, they purport the EIT to be applicable and usable in a variety of situations, as we found.

With respect to the three key elements within the EIT framework, the practical measures aspect will now be discussed in more detail as it affected the methodological approach. The two other elements, namely evidence-based interventions and the participatory implementation process (including stakeholder involvement), are more concerned with the intervention development and roll out and are reported elsewhere [ 16 , 17 , 37 ]. It is worth, however, mentioning briefly here some experiences with stakeholder involvement as they affected the methodology. In the face of universal health and social care resource constraints, considerable commitment was required for stakeholders not only to develop and implement the improvements with research teams at sites, but also to take part in interviews and assist with obtaining quantitative indicators. Our partnership approach fostered through the EIT approach enabled sustained buy-in to a large extent. However during the course of the implementation plan roll-out, two sites withdrew due to competing priorities and a diversion of resources away from the SUSTAIN initiative. We were able to gather valuable data on the context and reasons for this withdrawal to supplement out analysis. Again, the adoption of case study design overcame these flexes during the data collection period and, overall, helped to create useful and transferable results [ 31 , 38 ].

Moving now to the practical measures, the extent to which we were able to develop a core set of applicable measures needs consideration. Given the difficulties with integrated care evaluation, we made efforts in our design to select meaningful and pragmatic instruments through a wide literature search, particularly with respect to measuring service user impact. A number of considerations resulted in a contraction of suitable instruments; for example, they had to be applicable to each of the very different integrated care improvements set up within the 14 SUSTAIN sites; they had to be suitable for administration to frail older people; and our central concepts of person-centredness, prevention orientation, safety needed to be reflected in the instruments. Authors have usefully illumined on the evaluation of integrated care and the utility of associated instruments, many of which were considered during the selection process [ 39 , 40 ]. However, it became clear early on that several existing and validated indicators for frail older people with multimorbidity would be unsuitable. With quality of life measures for example, this was due to the high possibility that relatively short interventions would have little impact; and recommended instruments such as PACIC (Vrijhoef et al. 2009) were not ‘hitting’ all of our considerations sufficiently closely. We therefore narrowed our focus onto an examination of improvements to care and the personal impact of care delivery, which included degrees of person-centredness, experiences of co-ordination, and perceived control and independence.

With this in mind and after much deliberation within the SUSTAIN consortium, we selected the P3CEQ [ 41 , 42 ], and the PCHC [ 43 ], the latter validated for our population group. At the time of selection, the P3CEQ was relatively new but seemed suitable for administration across all sites and intervention types. We did experience, however, some repetition between these two questionnaires, and in some sites there were significant problems with recruitment and fatigue of older people. In response to this, the PCHC was withdrawn, as the P3CEQ seemed more tuned to the SUSTAIN themes and also included items on control and independence in health and social care. Case study design accommodated this adaptation. The data collection and analysis relating to the P3CEQ was not without its challenges during the course of SUSTAIN however. We found it needed essential preconditions (eg. face-to-face administration, collections of reasons for non-response) and administration and coding guidelines (eg where informal carers support service users to answer questions) [ 44 ]. We conclude through our experiences, that establishing a solid and standard cross-country measure of older service user experiences for integrated care still remains fraught with complexity and somewhat elusive.

Obstacles were more apparent with obtaining quantitative indicators due to the availability, accessibility and reliability of appropriate health and social care data across partner countries. This is due for example to differences in what and how data is collected, variations in the geographical representation of data, and the general lack of social care data, and these problems are persistent. For example, across Europe, data is scattered across systems, is not interoperable, and there are privacy concerns and technical challenges that block effective data recording and sharing at local, national and European levels [ 18 ]. In addressing this somewhat ‘hostile’ environment, we co-created a core list with professionals and managers at the sites of what could be obtained from either routine service level data, clinical notes, care plans or other sources (see Efficiency data in Table 1 for indicators that were deemed common across sites).

Collecting directly from clinical data and care plans had the potential to be a rich source of data [ 45 ] and could overcome the problem of aggregated measures, such as hospital admission data, and their sensitivity to projects where the population group is small and widely dispersed. Similarly, with the cost data, very few sites were able to extract specific costs related directly to the improvement interventions, but an estimate of staff hours was deemed possible, to give some indication of resource use. However using both clinical notes, care plans and staff hours were dependent upon the accurate recording of these events by busy practitioners and managers, which could not be assured, an aspect also acknowledged by Jefferies et al [ 46 ] With clinical notes, this recording was variable and unless prompted, did not always yield the information required. Care plans were not always completed or available; other researchers have had similar experiences and list causes as staff time pressures, poor document construction and communication difficulties with service users, recognised in other studies [ 7 ]. With staff hours, although diaries/templates were made available at sites, staff worked across initiatives and were not always able to separate and accurately record specific hours dedicated to the improvement initiatives. So, in most cases this was estimated, and thus the ability to give a sound cost analysis was greatly reduced.

With this last point, difficulties with the measurement of cost in integrated care is the subject of much debate within the literature. Lack of standardised outcomes and continuous changes in care delivery, for example, render the employment of traditional economic models unusable [ 47 ]. While SUSTAIN was keen to avoid health economic methods that have a poor fit with the nature of integrated care, it was clear that our more pragmatic approach was also not optimal, and the search for a more reliable and attributable method should continue.

Any deficits within quantitative data were however compensated by the richness of our qualitative data sources. As well as service user and carer interviews, we obtained professional, managerial and other stakeholder viewpoints, alongside documentary evidence from care plans (where available), steering group meetings and field notes. These perspectives provided valuable insights into personal impacts of the intervention, contextual influences and more nuanced information about if, how and why improvements made a difference (see De Bruin et al. [ 36 ]).

Having reviewed the relative mertis of the EIT framework, the discussion moves on to a critique of case study design. One of the most commonly cited disadvantages of case studies is that findings can lack generalisability and scientific credibility because replication is difficult [ 37 ]. However, external validity can be stronger in multiple case study designs, which was the choice in SUSTAIN, and can be weak in more highly ranked randomised control trials. Such weaknesses in RCT design have been exposed in a number of systematic reviews and secondary analyses [ 48 ].

In practical terms, there are further difficulties that researchers can encounter. For example, there can be a tendency to become overwhelmed with data and the process can be very time consuming, particularly with regard to developing and blending thematic statements from the analysed data sources. This occurs particularly when propositions are lacking and there has been no attempt to link the data collection with the aims of the study in a focused way, or create some boundaries to data collection [ 26 ]. In SUSTAIN, we established clear objectives and propositions, protocols for every aspect of data collection and management, analytical templates for ensuring consistency with data analysis, and a shared quality-controlled database. Difficulties still arose however, so to supplement this and optimise uniformity of our evaluation approach, we arranged regular one-to-one progress and ‘trouble-shooting’ calls with research teams and devoted space at six monthly consortium meetings to deal with methods issues.

The discussion now moves finally to a consideration of why we selected implementation research over other methods such as realist evaluation. For SUSTAIN, the importance of gaining a consistent and understandable method across different institutions and contexts, as well as involving stakeholders not wholly conversant with research, was paramount. While our approach was not fault-free, realist methods also has its challenges regarding its complexity. For example, Greenhalgh et al. (2009) [ 10 ] noted that a set of more or less well-defined ‘mechanisms of change’ in reality can prove difficult to nail, and the process of developing CMO configurations is an interpretive task, achieved through much negotiation and dispute. In addition, the authors add that while realist evaluation can draw useful lessons about how particular preconditions make certain outcomes more likely, it cannot produce a simple recipe for success. Given that this latter aspect was a significant factor for our aim of promoting good knowledge transfer, the applicability of realist approaches to our design was limited, with implementation research seemingly more suitable.

Nevertheless, similarities are evident between these different evaluation approaches. While realist uses the development of CMO configurations, implementation research also investigates equally important factors affecting implementation (geographical, cultural beliefs, poverty), the processes of implementation themselves (multi-disciplinary working, local resource distribution) and the end product or outcome of the implementation [ 14 ]. Implementation research does not however link the components so strongly, circumnavigating the lengthy interpretation tendency of realist approaches. Nor does it, particularly in the case of EIT, lend itself to so readily to theory generation, unlike realist approaches. Hence combining the EIT framework with case study design as we did in SUSTAIN offered stronger opportunities for theory testing and development, as previously outlined.

Conclusions: lessons learned and recommendations

Overall, in the strive to seek out the answers to ‘what works’ in integrated care provision, SUSTAIN has enabled the identification of different ways to advance integrated care evaluation locally, nationally and across Europe, that fundamentally recognises its status as a complex intervention, and as a process. Operating within this conceptual and theoretical understanding, we were able to apply pre-emptive consideration to the challenges in the evaluation design, obtaining a good pragmatic fit with the objectives of evaluating improvements. It is clear that difficulties with health data continue, which impacted on our ability to provide a robust transferable set of core indicators, highlighting the continuing challenges. However, instruments within this set still are anticipated to be of value and more meaningful to what integrated care should aim to achieve. Integrated care evaluation continues to challenge, and our approach in SUSTAIN was not without its own challenges. However out intention with this paper is to support researchers by adding to the international methodological repertoire of evaluation approaches that encourage a goodness of contextual fit.

The following are key lessons learned and recommendations:

Without doubt, we would advocate a participatory approach to evaluation designs and one set within implementation research. This recognises the dynamic nature of integrated care implementation and keeps pace with its ebbs and flows, thereby strengthening the evaluation approach and potential for knowledge transfer.

Case study design also proved to be highly useful and adaptable to the changes in evaluation requirements, variations between sites, and is pertinent to cross-European comparative research.

With respect to the target group of older people, there is a clear need to employ more innovative data collection techniques that step aside from traditional survey and interview approaches, towards methods that are interactive, engaging and experiential and take account of ageing. Talking Mats, a tested and validated vehicle to support older people to communicate about things that matter to them, is gathering momentum as a research tool [ 49 ] and may be a way forward.

Further research is needed to better understand and measure the relationship between resource and cost changes and integrated care. In keeping with growing opinion, the focus must move away from traditional health economic models towards a more realistic and pragmatic perspective of what can be measured. Rephrasing of cost objectives towards investigating a ‘better use of resources’ within the integrated care environment may be a start.

Availability of data and materials

Researchers can apply for data by submitting a proposal to [email protected] . After agreement of the proposal analysis by the SUSTAIN steering committee, and after ethics approval and a data transfer agreement, collaborative researchers can receive data for a specific research question. Fees will be dependent upon the amount of work needed for data extraction.

Abbreviations

Evidence Integration Triangle

European Union

Person-centred experiences of co-ordinated care questionnaire

Perceived control of health care

Team climate inventory short version

Billings J. Improving the evidence base. In: Leichsenring K, Billings J, Nies H, editors. Long-term care in Europe: improving policy and practice. Basingstoke: Palgrave Macmillan; 2013. p. 299–324.

Chapter   Google Scholar  

Nuño Solinís R, Stein KV. Measuring integrated care – the quest for disentangling a Gordian knot. Int J Integr Care. 2016;16(3):18. https://doi.org/10.5334/ijic.2525 .

Article   PubMed   PubMed Central   Google Scholar  

Tsasis P, Evans JM, Owen S. Reframing the challenges to integrated care: a complex-adaptive systems perspective. International Journal of Integrated Care. 2012;12(5). https://doi.org/10.5334/ijic.843 .

Shaw S, Rosen R, Rumbold B. What is integrated care? Nuffield trust; 2011. https://www.nuffieldtrust.org.uk/files/2017-01/what-is-integrated-care-report-web-final.pdf Accessed 15th Nov 2018.

De Bruin SR, Versnel N, Lemmens LC, Molema CCM, Schellevis FG, Nijpels G, et al. Comprehensive care programs for patients with multiple chronic conditions: a systematic literature review. Health Policy. 2012;107:108–45.

Article   Google Scholar  

Billings JR, Leichsenring K. Methodological development of the interactive INTERLINKS Framework for Long Term Care. International Journal of Integrated Care. 2014;14(2). https://doi.org/10.5334/ijic.1173 .

Curry N, Harris M, Gunn L, Pappas Y, Blunt I, Soljak M, et al. Integrated care pilot in north West London: a mixed method evaluation. Int J Integr Care . 2013; 13 25th July https://www.ijic.org/articles/10.5334/ijic.1149/ Accessed 15th Nov 2018.

Billings J. The INTERLINKS Framework for Long-Term Care of Older People in Europe. Journal of Integrated Care. 2013;21(3):126–38. https://doi.org/10.1108/JICA-02-2013-0007 .

Pawson R, Tilley N. Realistic evaluation. London: Sage Publications; 1997.

Google Scholar  

Greenhalgh T, Humphrey C, Hughes J, Macfarlane F, Butler C, Pawson R. How do you modernise a health service? A realist evaluation of whole-scale transformation in London. Milbank Q. 2009;87(2):391–416. https://doi.org/10.1111/j.1468-0009.2009.00562.x .

Middleton L, Rea H, Pledger M, Cumming J. A realist evaluation of local networks designed to achieve more integrated care. Int J Integr Care. 2019;19(2):4. https://doi.org/10.5334/ijic.4183 .

Jagosh J, Bush PL, Salsberg J, et al. A realist evaluation of community-based participatory research: partnership synergy, trust building and related ripple effects. BMC Public Health. 2015;15:725. https://doi.org/10.1186/s12889-015-1949-1 .

Busetto L, Luijkx K, Vrijhoef B. Development of the COMIC model for the comprehensive evaluation of integrated care interventions. Int J Integr Care. 2017;17(5):A282. https://doi.org/10.5334/ijic.3595 .

Peters DH, Adam T, Alonge O, Akua Agyepong I, Tran N. Implementation research: what it is and how to do it. BMJ. 2013;347:f6753. https://doi.org/10.1136/bmj.f6753 .

Article   PubMed   Google Scholar  

Eccles MP, Mittman BS. Welcome to Implementation Science . Implement Sci. 2006;1:1. https://doi.org/10.1186/1748-5908-1-1 .

Article   PubMed Central   Google Scholar  

De Bruin SR, Stoop A, Billings J, Leichsenring K, Ruppe G, Tram N, et al. The SUSTAIN project: a European study on improving integrated Care for Older People Living at home. Int J Integr Care. 2018;18(1):61–12. https://doi.org/10.5334/ijic.3090 .

Stoop A, De Bruin SR, Wistow G, Billings J, Ruppe G, Leichsenring K, et al. Exploring improvement plans of fourteen European integrated care sites for older people with complex needs. Health Policy . 2019;123(12):1135–54. https://doi.org/10.1016/j.healthpol.2019.09.009 .

Politico. Europe chases promise of big health data; 2017. https://www.politico.eu/article/europe-chases-promise-of-big-health-data/ Accessed 15 Nov 2018.

Glasgow RE, Green LW, Taylor MV, Stange KC. An evidence integration triangle for aligning science with policy and practice. Am J Prev Med. 2012;42(6):646–54.

Glasgow R. What does it mean to be pragmatic? Pragmatic methods, measures and models to facilitate research translation. Health Educ Behav. 2013;40(3):257–65.

Yin RK. Case study research: design and methods. 4th ed. Thousand Oaks: Sage; 2009.

Swanborn P. Case study research: what, why and how? Thousand Oaks: Sage Publications; 2010.

Book   Google Scholar  

Church C, Rogers MM. Indicators. Chapter 4 in Designing for Results: Integrating Monitoring and Evaluation in Conflict Transformation Programs . Washington USA: Search For Common Ground; 2006. p. 43–60.

Kivimäki M, Elovainio M. A short version of the team climate inventory: development and psychometric properties. J Occup Organ Psychol. 1999;72:241–6. https://doi.org/10.1348/096317999166644 .

Hancock DR, Algozzine B. Doing case study research: a practical guide for beginning researchers. New York: Teachers College Press; 2006.

Billings J. Towards rigour in qualitative research across European partnerships. Eur J Ageing. 2004;1(1):73–8.

Donnelly C, Brenchley C, Crawford C, Letts L. The integration of occupational therapy into primary care: a multiple case study design. BMC Fam Pract. 2013;14:60 www.biomedcentral.com/1471-2296/14/60 accessed 12/2/16 .

Baxter, Jacks. Qualitative case study methodology: study design and implementation for novice researchers. Qual Rep. 2008;13(4):544–59.

Carrieri D, Briscoe S, Jackson M, et al. Care under pressure’: a realist review of interventions to tackle doctors’ mental ill-health and its impacts on the clinical workforce and patient care. BMJ Open. 2018:e021273. https://doi.org/10.1136/bmjopen-2017-021273 Accessed 14th Oct 2018.

Kolanowski A, Van Haitsma K, Galik E, Boltz M, Ellis J, Behrens L, et al. Testing the evidence integration triangle for implementation of interventions to manage behavioral and psychological symptoms associated with dementia: protocol for a pragmatic trial. Res Nurs Health. 2018;41(3):228–42. https://doi.org/10.1002/nur.21866 https://www.ncbi.nlm.nih.gov/pubmed/29485197 Accessed 14th Nov 2018.

Creswell J. Research design; qualitative and quantitative and mixed methods approaches. London and Thousand Oaks: Sage Publications; 2009.

Bergen A, While A. A case for case studies: exploring the use of case study design in community nursing research. J Adv Nurs. 2000;31(4):926–34.

Article   CAS   Google Scholar  

Chambers A, Mustard CA, Breslin C, Holness L, Nichol K. Evaluating the implementation of health and safety innovations under regulatory context: a collective case study of Ontario’s safer needle regulation. Implement Sci. 2013:8–9 www.implementationscience.com/contents/8/1/9 Accessed 12th Dec 2018.

Eerden M, Csikos A, Busa C, Hughes S, Radbruch L, et al. Experiences of patients, family and professional caregivers with integrated palliative care in Europe: protocol for an international, multicentre, prospective, mixed method study. BMC Palliat Care. 2014;13:52. https://doi.org/10.1186/1472-684X-13-52 Accessed 12/2/16.

Eisenhardt KM, Graebner ME. Theory building from cases: opportunities and challenges. Acad Manage J. 2007;50(1):25–32.

De Bruin SR, Stoop A, Baan CA, Billings JR, Nijpels G, on behalf of the SUSTAIN consortium. Sustainable tailored integrated care for older people in Europe (SUSTAIN-project). Bilthoven/Amsterdam/Canterbury: Lessons learned from improving integrated care in Europe. National Institute for Public Health and the Environment/VU University Medical Centre/University of Kent; 2018.

MacInnes J, Baldwin J, Billings J. The over 75 service: continuity of integrated Care for Older People in a United Kingdom primary care setting. Int J Integr Care. 2020;20(3):2. https://doi.org/10.5334/ijic.5457 .

Raeburn T, Schmied V, Hunderford C, Cleary M. The contribution of case study design to supporting research on clubhouse psychosocial rehabilitation. BMC Res Notes. 2015;8:521. https://doi.org/10.1186/s13104-015-1521-1 Accessed 12 Feb 2016.

Vrijhoef HJM, Berbee R, Wagner EH, Steuten LMG. Quality of integrated chronic care measured by patient survey: identification, selection and application of most appropriate instruments. Health Expect. 2009;12(4):417–29. https://doi.org/10.1111/j.1369-7625.2009.00557.x .

Bautista MAC, Nurjono M, Lim YW, Dessers E, Vrijhoef HJ. Instruments measuring integrated care: a systematic review of measurement properties. Milbank Q. 2016;94(4):862–917. https://doi.org/10.1111/1468-0009.12233 .

Lloyd H, Fosh B, Whalley B, Byng R, Close J. Validation of the person-centred coordinated care experience questionnaire (P3CEQ). Int J Qual Health Care . 2019;31(7):506–12. https://doi.org/10.1093/intqhc/mzy212 .

Sugavanam T, Fosh B, Close J, Phil D, Byng R, Horrell J, et al. Co-designing a measure of person-centred coordinated care to capture the experience of the patient. J Patient Experience. 2018;5(3):201–11.

Claassens L, Terwee CB, Deeg DJ, Broese van Groenou MI, Widdershoven GA, Huisman M. Development and validation of a questionnaire assessing the perceived control in health care among older adults with care needs in the Netherlands. Qual Life Res. 2016;25(4):859–70. https://doi.org/10.1007/s11136-015-1124-2 .

Article   CAS   PubMed   Google Scholar  

Reynolds J, Gadsby E, Rijken M, Stoop A, Espallargues M, Lloyd H, et al. Measuring older peoples’ experiences of person-centred coordinated care: the SUSTAIN experience applying a patient reported experience measureIn press. Int J Integr Care. 2020. in press.

Andrews A, St Aubyn B. If it’s not written down it didn’t happen. J Commun Nurs. 2015;29:5 pp20–2 https://www.jcn.co.uk/files/downloads/articles/jcn-v29-is5-08-not-written-down-not-happen.pdf Accessed 15th Nov 2018.

Jefferies D, Johnson M, Griffiths R. A meta-study of the essentials of quality nursing documentation. Int J Nurs Pract. 2010;16:112–24.

Evers SMMA, Paulus ATG. Health economics and integrated care: a growing and challenging relationship. Int J Integr Care. 2015;15 17 June 2015, URN:NBN:NL:UI:10-1-114829 https://ijic.ubiquitypress.com/articles/10.5334/ijic.2201/ Accessed 14th Nov 2018.

Hunt G, Siegfried N, Morley K, Sitharthan T, Cleary M. Psychosocial interventions for people with both severe mental illness and substance misuse (review). Cochrane Database Syst Rev . 2013;2013:10.

Murphy J, Oliver TM. The use of talking Mats to support people with dementia and their carers to make decisions together. Health Soc Care Community. 2013;21(2):171–80.

Download references

Acknowledgements

The content of this article reflects only the SUSTAIN consortium members’ views. The European Union is not liable for any use that may be made of the information contained herein. The authors would like to acknowledge the contributions of the SUSTAIN consortium as a whole to the methods development.

The SUSTAIN project was funded under Horizon 2020 – the Framework Programme for Research and Innovation (2014–2020) from the European Commission under grant agreement No. 634144. The funding body had no role in the design of the study, data collection, analysis, and interpretation of data, nor in the writing of this manuscript.

Author information

Authors and affiliations.

Centre for Health Service Studies, University of Kent, Canterbury, Kent, CT2 7NF, UK

Jenny Billings

National Institute for Public Health and Environment, Antonie van Leeuwenhoeklaan 9, 3721, Bilthoven, MA, Netherlands

Simone R. de Bruin & Caroline Baan

VU University Medical Center, De Boelelaan 1117, 1081, Amsterdam, HV, Netherlands

Giel Nijpels

You can also search for this author in PubMed   Google Scholar

Contributions

The authors have read and approved the manuscript. JB was responsible for and led the selection of the methodological approach, its development and the responses to methods challenges as the project evolved. JB also selected suitable instruments and wrote all the internal study protocols and guidelines associated with conducting the evaluation. This was consistently and strongly supported by SdB, and guided by GN and CB. The SUSTAIN consortium contributed through discussion and feedback. All authors contributed towards drafts of the articles.

Corresponding author

Correspondence to Jenny Billings .

Ethics declarations

Ethics approval and consent to participate.

All seven countries participating in SUSTAIN obtained ethical approval according to their own governance procedures. Regarding consent to participate, written informed consent was obtained from all participants in all countries. The details of the institutions seeking approval and their respective Ethics Committees are as follows:

Austria . Institution: Osterreichishe Plattform fur Interdisziplinare Alternsfragen. Ethics committee of Vienna.

Catalonia, Spain . Institution: Agencia de Qualitat/Avaluacio Sanitaries de Catalunya. Ethics Committee: Comite Etica D’Investigacio Clinica 2015888.

Estonia . Institution: Sihtasutus Poliitikauuringute Keskus Praxis. Ethics committee: Meditsiiniuuringute Eetikakomitee 1187.

Germany . Institution: Stiftung Gesundheit. Ethics committees: Medical Association of Berlin and Medical Association of Brandenburg.

The Netherlands . Institution: Rijksinstituut voor Volksgezondheit en Milieu Centrum voor Voedin. Ethics committee: Medische Ethische Toetsingscommissie VU Medisch Centrum 2016.507.

Norway . Institution: University of Olso. Ethics Committee: REK Regionale Komiteer for Medisinsk og Helsefaglig Forskningsetikk 2016/2004/REK.

UK . Institution: University of Kent. Health Research Authority approval ref. 16/1EC08/0045.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Billings, J., de Bruin, S.R., Baan, C. et al. Advancing integrated care evaluation in shifting contexts: blending implementation research with case study design in project SUSTAIN. BMC Health Serv Res 20 , 971 (2020). https://doi.org/10.1186/s12913-020-05775-5

Download citation

Received : 02 June 2020

Accepted : 29 September 2020

Published : 23 October 2020

DOI : https://doi.org/10.1186/s12913-020-05775-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Integrated care evaluation
• Implementation research
• Process and outcome measures

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Threat Brief: Vulnerability in XZ Utils Data Compression Library Impacting Multiple Linux Distributions (CVE-2024-3094)

• 35,501 people reacted
• 6 min. read

March 30, 2024 at 7:15 PM

Category: Cloud , Threat Brief , Threat Briefs and Assessments , Vulnerability

Tags: Advanced WildFire , Cortex XDR , Cortex XSIAM , CVE-2024-3094 , incident response , Linux , Prisma Cloud , XZ Utils

This post is also available in: 日本語 ( Japanese )

Executive Summary

On March 28, 2024, Red Hat Linux announced CVE-2024-3094 with a critical CVSS score of 10. This vulnerability is a result of a supply chain compromise impacting the versions 5.6.0 and 5.6.1 of XZ Utils. XZ Utils is data compression software included in major Linux distributions. The U.S. Cybersecurity and Infrastructure Security Agency (CISA) has advised people to downgrade to an uncompromised XZ Utils version (earlier than 5.6.0).

The newly disclosed vulnerability has been assigned the following CVE:

Palo Alto Networks customers are better protected from and can implement mitigations for CVE-2024-3094 in the following ways:

The Next-Generation Firewall with cloud-delivered security services including Advanced WildFire detects the compromised versions described in this report as malicious, as well as features known to be associated with the backdoors.

• Cortex XDR and XSIAM help protect against post-exploitation activities using the multi-layer protection approach. Cortex customers using the Host Insights module can detect if the vulnerability exists on protected devices.
• Prisma Cloud has out-of-the-box detection capabilities in place that will help prevent the launch of images with CVE-2024-3094.
• The Unit 42 Managed Threat Hunting team is monitoring attempted malicious activities against our customers. The XQL queries shared in that section below can also be used by Cortex XDR customers to search for affected versions of XZ Utils.
• The Unit 42 Incident Response team can also be engaged to help with a compromise or to provide a proactive assessment to lower your risk.

Table of Contents

Details of CVE-2024-3094 Affected Versions and Mitigation Actions Conclusion Unit 42 Managed Threat Hunting Queries Palo Alto Networks Product Protections for the XZ Util Vulnerability Cortex XDR and XSIAM Prisma Cloud Additional Resources

Details of CVE-2024-3094

On March 28, 2024, Red Hat Linux announced CVE-2024-3094 with a critical CVSS score of 10. This vulnerability is a result of a supply chain compromise impacting the latest versions of XZ tools and libraries. XZ Utils is data compression software included in major Linux distributions.

Versions 5.6.0 and 5.6.1 of the libraries contain malicious code that modifies functions during the liblzma build process. Liblzma is a data compression library.

This malicious code results in a compromised liblzma library, which may modify or intercept data from other applications that leverage the library. Under certain conditions this code may allow unauthorized access to affected systems.

A security researcher, Andres Freund, found the malicious code when he saw failing ssh logins using high CPU loads. When researching the cause of the high CPU utilization he then also noticed slower logins which led to further exploration and discovery of the vulnerability.

Affected Versions and Mitigation Actions

All major Linux distros recommend either reverting back to versions built prior to the inclusion of XZ Utils 5.6.0 and 5.6.1 or migrating to updated releases.

Please check the notification page for your specific distribution for additional updates and guidance.

Additionally, HomeBrew package manager is forcing downgrades to 5.4.6. They do not believe Homebrew’s builds were compromised but are taking this action as a precaution.

Amazon has stated that Amazon Linux customers are not affected by this issue, and no action is required.

Unit 42 will continue to monitor the situation and will update this post as more information becomes available.

Unit 42 Managed Threat Hunting Queries

The Unit 42 Managed Threat Hunting team continues to track any attempted malicious activities across relevant Linux distributions used by our customers, using Cortex XDR and the XQL queries below. Cortex XDR customers can also use these XQL queries to search for affected versions of XZ Utils.

Palo Alto Networks Product Protections for the XZ Util Vulnerability

Based on the information presently known, Palo Alto Networks products and cloud services do not contain affected XZ software packages and are not impacted by these issues. Read our informational bulletin for more details.

Palo Alto Networks customers can leverage a variety of product protections and updates to identify and defend against this threat.

If you think you may have been compromised or have an urgent matter, get in touch with the Unit 42 Incident Response team or call:

• North America Toll-Free: 866.486.4842 (866.4.UNIT42)
• EMEA: +31.20.299.3130
• APAC: +65.6983.8730
• Japan: +81.50.1790.0200

Advanced WildFire

Cortex xdr and xsiam.

Cortex XDR and XSIAM agents help protect against post-exploitation activities using the multi-layer protection approach. Cortex customers using the Host Insights module can detect if the vulnerability exists on protected devices.

XDR customers can find and upgrade software vulnerable to this issue centrally from the XDR console. Our InfoSec SOC team used the following XQL query to find vulnerable versions of XZ on our endpoints:

Once you review the results, if you identify installations of software that need to be updated, you can execute an endpoint script on a particular software package.  For example, to upgrade a vulnerable HomeBrew installation on a list of Macs to the latest version (with the downgraded, safe, version of XZ) you can execute the following commands via XDR on your endpoints.

The menu options to follow are: Incident Response → Action Center → Run Endpoint Script → Execute Commands.

Prisma Cloud

Prisma Cloud has out-of-the-box detection capabilities in place that will help prevent the launch of images with CVE-2024-3094. Prisma Cloud’s agentless approach provides you with a comprehensive lifecycle overview from Code Repository to Cloud and simplified filter options that enable you to identify vulnerable hosts, high privilege access and potential exposure to the internet. Additionally, its defender component or pipeline integration offer real-time insights and protection capabilities, enabling you to prevent the launch of images with the CVE or detect and prevent anomalous behavior. Our researchers validated this capability relative to this CVE by committing a Dockerfile and then triggering a CI/CD pipeline to build and deploy the Docker image.

Additional Resources

• Reported Supply Chain Compromise Affecting XZ Utils Data Compression Library, CVE-2024-3094 – Cybersecurity and Infrastructure Security Agency (CISA)
• NVD - CVE-2024-3094 – NIST 
• Brew install xz installs the outdated version 5.4.6 instead of 5.6.1 – Homebrew, Discussion #5243 on GitHub
• Urgent security alert for Fedora Linux 40 and Fedora Rawhide users — Red Hat
• Debian Security Advisory DSA-5649: 1xz-utils security update – Debian
• Kali Linux: "As of the information we have …" – infosec.exchange on Mastodon
• OpenSUSE - Open Build Service Request 1163302 – Open Build Service 
• CVE-2024-3094 – Alpine Security Tracker
• News: The xz package has been backdoored – Arch Linux

Updated March 31, 2024, at 9:30 a.m. PT to add an additional XQL query and additional details about Cortex XDR protections.

Updated March 31, 2024, at 7:15 p.m. PT to add a link to additional info about Host Insights.

Updated April 1, 2024, at 8:15 a.m. PT to add info on Advanced WildFire protections.

Updated April 2, 2024, at 10 a.m. PT to add a link to the Palo Alto Networks Product Security Assurance team's assessment that PANW products and cloud services are not impacted by these issues.

Updated April 3, 2024, at 1:45 p.m. PT to expand Cortex XDR product protection information on endpoints. 

Get updates from Palo Alto Networks!

Sign up to receive the latest news, cyber threat intelligence and research from us

Please enter your email address!

Please mark, I'm not a robot!

By submitting this form, you agree to our Terms of Use and acknowledge our Privacy Statement .

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Equity Health

Does public service motivation matter in Moroccan public hospitals? A multiple embedded case study

Zakaria belrhiti.

1 National School of Public Health, Rabat, Morocco

2 Department of Public Health, Institute of Tropical Medicine, Antwerp, Belgium

3 Vrije Universiteit Brussel, Brussels, Belgium

Wim Van Damme

Abdelmounim belalia, bruno marchal, associated data.

Not Applicable.

The motivation of health workers is a key concern of policy makers, practitioners and researchers. Public Service Motivation (PSM), defined as the altruistic desire to serve the common interest, to serve others and to help patients and their families regardless of financial or external rewards, has been shown to be key to the performance of public servants. Yet, limited attention has been paid to this kind of motivation in health care settings in low- and middle-income countries. Little is known about PSM and its contextual specificity in the Moroccan health system. We set out to qualitatively explore the meaning of PSM and its expression among health workers in four public hospitals.

We adopted a multiple embedded case study design to explore PSM in two well-performing and two poor-performing hospitals. We carried out 68 individual interviews, eight focus group discussions and 11 group discussions with different cadres (doctors, administrators and nurses). We carried out thematic analysis using NVivo 10.

Our analysis shows that public service motivation is a notion that seems natural to the health workers we interviewed. Daily interactions with patients catalysed health providers’ affective motives (compassion and self- sacrifice), a central element of PSM. It also provided them with job satisfaction aligned with their intrinsic motivation. Managers and administrative personnel express other PSM components: attraction to public policy making and commitment to public values. A striking result is that health workers expressed strong religious beliefs about expected rewards from God when properly serving patients.

This study highlights the presence of PSM as a driver of motivation among health workers in four Moroccon hospitals, and the prominence of intrinsic motivation and compassion in the motivation of frontline health workers. Religious beliefs were found to shape the expression of PSM in Morocco.

Introduction

The motivation of health workers is a key concern of policy makers, researchers and practitioners [ 1 , 2 ], as it is widely regarded as a key determinant of health worker’s performance. Most research on motivation carried out in the field of human resources for health focuses on extrinsic motivation,, performance-based financing and contracting out [ 3 – 5 ]. These latter strategies, inspired by New Public Management principles, have been questioned [ 6 – 8 ] and more specifically in terms of goal-displacement, the risk of crowding out intrinsic motivation and inducing mistrust [ 9 – 11 ]. In response, a number of public management scholars have been developing the notion of Public Service Motivation (PSM). PSM is defined in public management as “ the beliefs, values and attitudes that go beyond self-interest and organizational interest, that concern the interest of a larger political entity, and that motivate individuals to act accordingly whenever appropriate” [ 12 ].

At the core of PSM are motives and actions that are intended to serve the public interest, to serve others and to improve the well-being of society [ 13 ]. According to Perry and Wise (1990), three kinds of motives are associated with public service [ 14 ]. Rational motives refer to the aspiration of workers to participate in good public decision-making, because they consider this as a social duty or because it reinforces their image of self-importance and self-esteem. Norm-based motives refer to the desire to altruistically serve the public interest or to serve the nation in order to contribute to social equity [ 14 , 15 ]. A ffective motives involve identification with the public service, affective bonding with service users and compassion, contributing to self-sacrifice [ 14 – 16 ].

The interest in PSM is due to a number of positive effects that have been attributed to it. Research exploring these effects was mainly carried out among public servants in governmental agencies in the USA [ 17 – 19 ], Belgium [ 20 ], Switzerland [ 21 ], the Netherlands [ 22 ], Danemark [ 23 ], Malta [ 24 ], South Korea [ 25 ] and China [ 26 ]. The studies indicated significant statistical relationships between PSM on one hand, and individual performance, job satisfaction and reduced turnover on the other. These findings were supported by meta-analysis [ 27 – 30 ] and research designs using quasi-experimental designs [ 31 , 32 ].

The research on public service motivation started with substantial exploratory research about 20 years ago [ 14 , 18 , 33 – 35 ]. Later, frameworks were developed that linked policy and management [ 12 ] or leadership [ 36 ] to PSM. The latter is now considered to be a complex psychological state that is influenced by leadership, organisational and contextual factors [ 24 , 33 , 37 – 40 ]. Indeed, research has shown that transformational leadership may contribute to PSM [ 41 ], but that this is contingent on organisational and contextual characteristics [ 42 – 45 ]. Organisational factors include organisational culture [ 37 , 46 ] and job characteristics that favour increased contact with public service beneficiaries [ 31 , 32 ]. Contextual factors include the broader societal culture [ 47 , 48 ], religion, family education and professionalism [ 39 , 40 ].

The current state of the art points to the importance of developing robust research methodologies to explore the underlying mechanisms through which leadership, context and organisational attributes may influence PSM [ 18 , 29 , 33 , 36 , 49 , 50 ]. No research, to our knowledge, has explored the concept of PSM in Morocco, nor the contextual factors and mechanisms underlying the emergence of PSM in healthcare settings in Morocco. Yet, cultural differences may explain differences in meaning, antecedents and consequences across countries [ 46 , 51 , 52 ].

The study we present here aimed at exploring the concept of public service motivation in Morocco and how it is expressed by health professionals and managers/administrators working in public hospitals. More specifically, we set out to identify how the notion of public service motivation is being defined by health workers, to explore the differences in the definition by cadre and to explore the factors that may influence PSM.

This study is part of a larger research project that adopted the realist evaluation approach [ 53 ] to examine the links between leadership, staff motivation and performance. This study is exploratory in nature [ 54 ] in a sense that it explores the motivation of health workers using insights from PSM theory in the context of Morocco.

Morocco is a lower middle-income country with a population of 35.6 millions [ 55 ]. Islam is the religion of most Moroccans and this has an impact on daily life and work place practices [ 56 – 58 ]. The Moroccan society is multi-cultural, collectivistic and strongly attached to family relationships and filial piety [ 56 , 57 ].

In health, there has been a significant progress in many health indicators (e.g. 35% reduction of the maternal mortality rate between 2010 and 2016), the extension of the coverage to the poor and vulnerable populations, the decentralisation and the introduction of public private partnerships [ 59 , 60 ]. However, the Moroccan health system remains weak, ranked 134th out 195 countries in terms of health access and quality of care [ 61 ] with an inequitable access to care and a poorly regulated private sector [ 62 , 63 ].

The Moroccan health system is constrained by an acute health workforce shortage, for instance having 0.7 doctors per 1.000 inhabitants and 0.92 nurses and midwives per 1.000 inhabitants) [ 64 , 65 ]. Studies indicated a lack of staff commitment, poor motivation, low job satisfaction and poor working conditions [ 58 , 66 ] that have hampered the implementation of many health system reforms, for instance quality assurance programmes [ 67 , 68 ], fee exemption policies [ 69 ]. Other studies found that these health workforce issues affected the quality of patient-provider interactions [ 70 , 71 ].

Study design

In this study, we adopted the case study design. The case here is PSM as experienced by health personnel in hospitals. The multiple embedded case study design is appropriate to the exploration of complex phenomena, such as PSM in real world settings and allows for comparison between sites [ 72 ]. We purposefully selected two high-performing and two low-performing hospitals. We used the results of the national quality assurance programme called concours qualité to select hospitals as our study sites. This programme assessed hospital performance in eight dimensions: (1) accessibility/availability/continuity; (2) patient security and responsiveness; (3) satisfaction; (4) ethics; (5) quality assurance; (6) resource utilisation; (7) technical competencies and (8) leadership. The overall performance score index is measured by the ratio between the actual score assessed by the external audit and the maximum obtainable score. We refer to [ 67 , 68 ] for an overview of the programme. Using data from the quality assurance report 2011 [ 73 ] and 2016 [ 74 ], we identified hospitals with a significant increase or decrease of performance between 2011 and 2016.

Conceptual framework

In the field of public management, the definition of PSM has evolved since 1990, broadening from ‘individual predisposition’ to a more detailed description (Table  1 ).

Definitions of PSM

In the last two decades, most PSM research focused on developing measurement scales. These studies were carried out in the USA, Europe, Asia and South America [ 78 ]. Wright noted a high degree of variability in operational definitions and a diversity of PSM scales [ 79 ]. To overcome this diversity, Kim and colleagues [ 80 ] refined the multidimensional scale developed by Perry and Wise [ 35 ] and validated it across 12 industrialised countries [ 81 – 83 ]. It comprises:

• attraction to public service, which means a “disposition to serve the public, to work for the common good, and to participate in public policy processes” [ 15 ].
• commitment to public values, understood as “a personal disposition to pursue public values” [ 15 ].
• compassion, defined as “an affective bonding with identified objects, such as other members of a social category or of a political system” [ 15 ].
• self-sacrifice, meaning “the willingness of public servants to forego financial rewards for the intangible rewards they receive from serving the public” [ 35 ].

In this study, we consider individual motivation as mix of motives including a continuum that ranges from extrinsic motivation to more autonomous forms of motivation (intrinsic motivation and PSM). We mean by intrinsic motivation ‘ the doing of an activity for its inherent satisfaction rather than for some separable consequences’ [ 84 ]. We adopt Ryan and Deci’s definition of extrinsic motivation: “ a construct that pertains whenever an activity is done in order to attain some separable outcome ” (e.g. tangible and verbal rewards) [ 84 ]. We adopted the definition of Vandenabeele of PSM. As explained below, we used the 4 components of PSM as defined by Kim et al. [ 83 ] and the concepts of intrinsic and extrinsic motivation as defined by Ryan and Deci [ 84 ] to analyse the data.

Data collection and analysis

We started with interviews and then conducted group discussions. A document review was carried out all along the study. All data were collected during the period January–June 2018 (see Tables ​ Tables2 2 and ​ and3 3 ).

Data collection

Respondents characteristics

We started the data collection by interviewing hospital staff. In total, we carried out 68 in-depth interviews (IDI) with senior-, middle-, operational-level managers and health professionals [ 85 , 86 ]. We explored the views of respondents on public service motivation, its definition and its expression, as well as the factors that may influence PSM. We used adapted open-ended interview guides for each category of respondents (Additional file 1 ). These were tested in a pilot study with professionals. We carried out the interviews until saturation was attained.

(Focus) group discussions

In a second stage, we carried out eight focus group discussions and 11 group discussions with different cadres (administrators, nurses and doctors) to further explore the key constructs mentioned by interviewees in relation to motivation. This allowed us to deepen the analysis across different categories of health workers (Table  3 ). The FGD and group discussions were conducted by the first author, who used a facilitator guide (Additional file 2 ). We conducted the FGD following standardised procedures described by Krueger and Morgan and used probes, asked follow up questions, summarised key themes and sought verification from participants [ 87 , 88 ].

We used qualitative purposive sampling [ 86 ] in order to identify respondents for the in-depth interviews and the focus group discussions. Interviews were carried out in Moroccan dialect. All interviews, FGDs and group discussions were audio-recorded with the exception of two interviews. In these specific cases, we took notes and transcribed the unrecorded interviews using memory recall [ 85 ]. All transcripts were checked for accuracy by two co-authors (ZB and BM).

Table  2 presents the breakdown per hospital. We used codes to identify respondents anonymously, referring to the hospital EJMH, NHMH, RKMH, SMBA.

At the end of each contact with research participants, we wrote a brief contact summary that included major themes and ideas arising from the interaction following guidance provided by Miles and Huberman, and Krueger [ 87 , 89 ]. Table  3 represents the summary of data collection tools and respondents’ profiles (Additional files 3 , 4 , 5 and 6 provide detailed descriptions of the respondent characteristics).

Document review

In order to identify key elements in the broader health policy context and to describe the general context, we collected documents all along the study. Key informants at the four hospitals and at the Ministry of Health contributed to identifiying relevant documents. We focused on human resources availability and skill mix data, strategic plans, audit documents and quality assurance reports.

We structured the qualitative data analysis along the analytic phases of compiling data, interpreting data, discussion and drawing conclusions [ 90 ]. During the initial coding cycle, we coded all data sources (transcripts, contact summaries and field notes) using different coding techniques (concept, hypothesis and “in vivo” coding) [ 91 ] and used NVivo 10 v11. 4.3 software to manage the data [ 92 ]. During the second coding cycle or pattern coding, [ 89 ], we used the four PSM components (attraction to public service, commitment to public values, compassion, self-sacrifice) described by Kim and colleagues [ 83 ] and the intrinsic/extrinsic types of motivation defined by Ryan and Deci [ 84 ]. As we will present below, we also identified other categories of motives that emerged from the inductive analysis of data (religious based motives). The coding was discussed during research team meetings. These meetings were conducted at different moments during the analysis, focusing on the initial coding, the second coding cycle, the in-case analysis and the cross-case analysis.

Ethical considerations

The research protocol was approved by the Moroccan Institutional Review Board in Rabat (n°90/16) and the Institutional Review Board of the Institute of Tropical Medicine, Antwerp (N° 1204/17). All interviewees were informed before the start of data collection about the study objectives, the topics, the type of questions and their right to refuse being interviewed or interrupt the interview at any time. The same information was included in an information sheet that was given to candidate interviewees and reiterated when the written consent form was discussed before the start of the interview. The informed consent forms were signed by the participants and co-signed by the researcher. A copy of the signed consent form was given to research participants.

In this section, we present how health workers belonging to different cadres define ‘public service motivation’ and identify which factors may influence the level of PSM. We start with a summary of the intrinsic and extrinsic motives of the respondents.

What motivates health workers?

We found that the respondents were motivated by a mix of intrinsic and extrinsic motives.

Intrinsic motives

Both nurses and doctors expressed the importance of their intrinsic motivation, which is fuelled by the satisfaction derived from applying their professional skills and competencies.

“I love my job. I chose deliberately to work at the emergency unit. I love working at the emergency unit. I am totally engaged. Handling serious medical emergencies is a motivation in itself”. EJMH 38, Doctor.

“I am frustrated because my salary does not compensate my efforts. However, nursing is a noble profession that has nothing to do with financial incentives.” NHMH 30, Nurse.

Performing non-clinical tasks such as participating in quality circles and community volunteering aligned with intrinsic motivation. This enhanced their feeling of self-efficacy.

“As for me, this [participating in the quality contest] was a great pleasure. I enjoyed that. This was not for the sake of doing good for others, but it was mainly for my own satisfaction… My objective was to accomplish this managerial task and to prove to myself that I can do this. This is why I was striving to make that effort.” NHMH 10, Doctor.

“I participated in several ‘medical caravans’. This was for me just pleasure. I just enjoyed it. It was not about the feeling to do good for others. I gained patient recognition and above all I felt self-efficacious toward patients”. NHMH 31, Doctor

Extrinsic motives

More extrinsic motivation-related drivers were reported as well, including the importance of recognition by leaders and patients.

“We need that leaders recognise our performance: 6.000 deliveries a year! We need that they congratulate us”. SMBA 06, Midwife.

“We are satisfied when our effort is acknowledged by others (patients).” NHMH 13, Doctor

“When people came to thank me, because they get well because of me, I feel that I am the happiest man in the world. Because, I hate to walk around in the city and that people bad mouth me: “This obnoxious doctor did not treat me well”. I cannot tolerate this.” SMBA 18, Doctor.

“I feel ashamed, if I did not do the work my superior ask me to do”. SMBA 05, Nurse / senior manager.

Other sources of extrincic motivation mentioned by our respondents include job security, flexibility of working schedules and work-family life balance.

“Here in the public sector, there is a certain liberty, work is fluid. This is why I choose the public sector. I avoided the private sector where I could earn twice or thrice my salary, but I dropped the economic reasons, and chose to get the minimum wages offered in the public sector, because I chose to be free, because there is less hierarchy than in the private sector or in the university teaching hospital, where the doctor in chief could sanction me severely and even stop my salary.” NHMH 2, Nurse.

“Why am I staying at the public sector? I can earn three times my salary in the private sector. Well, I have a six years old daughter, I need to enjoy her company.” RKMH 3, Doctor.

How is PSM defined?

It emerged from our analysis that public service motivation is a notion that seems natural to the health workers we interviewed. When talking about PSM, respondents mentioned several components: affective motives (compassion and self-sacrifice), normative motives (commitment to public values) and rational motives (attraction to public service). In addition, they reported religion-based motives they labelled as ‘seeking divine rewards’.

Affective motives

In terms of affective motives, our respondents talked mainly about compassion and self-sacrifice. To a striking degree, nurses and physicians expressed compassion with patients’ conditions as a major motivational factor. This emotional response was also expressed by administrative personnel with frequent contacts with patients (e.g. cashiers).

“Patients are important for me because I got sick. So, I sense what the patients are feeling. My family members, my daughter and my grandmother got sick. So, I feel the pain patients are suffering from. I can feel their suffering. ” SMBA 35, Nurse.

Staff were expressing compassion with the vulnerable and underprivileged members of the population. They were placing themselves in their situation and showing empathy toward these patients and their families. This even led some of them to help patients pay user fees.

“One day, a citizen came to pay for laboratory tests for his daughter who had fever and he could not afford to pay the fees. I added the missing amount from my own money. Not every day, but often, I bring small change to help citizens who do not have the full amount. My wallet is always opened.” NHMH 29, Revenue officer.

Respondents explained that serving the underprivileged and caring for the poor is one of the reasons that kept them working in the public sector.

“Here, I work a lot with vulnerable citizens. It is a reward in itself to serve poor patients. It is my source of motivation”. RKMH 3, Doctor

The notion of self-sacrifice was mentioned but not often. Some respondents expressed the importance of serving the patient compared with the financial rewards. Some health workers may forgo their own health needs in order to serve patients in what we call an “escalation of commitment”.

“Sometimes, I think I sacrifice a lot in order to serve others. We cannot justify this. It is not reasonable. It is 16h30 and I just ate! I do not eat. I lost weight, I got tuberculosis, I had cervical pain, arthralgia, backpain, a sciatica. I had thoracic pain because I suffered from a pleural effusion. Is it logical to suffer in order to treat patients?” EJMH 38, Doctor Emergency Unit.

“The cardiologist is highly concerned about patients. She is worried about them. Even if her shift ends at 4H30 pm, she keeps coming back to the hospital to check on them, even during weekends. She is continuously in contact with us by phone to check on her patients, their test results, their condition. She is omnipresent. If a patient is in a bad condition, she returns to the hospital. She does not have to, but she keeps coming back even during weekends. She is so consciencious and keeps checking on her patients. This is why she got sick!” EJMH 24 nurse.

Norm-based motivation

In terms of norm-based motivation, our analysis shows that many respondents emphasised their desire to serve the public interest and their high sense of civic duty. They referred to these motives as feelings of citizenship, the desire to serve the nation, the citizens and the general interest, and the need to be rightful and equitable. This was expressed by all categories of staff (doctors, nurses and administrative staff).

“A love for our country, love of our territory, love of the land of our ancestors, love of neighbourhoods, we love to leave a suitable environment for our siblings.” EJMH 8, Administrator.

“We are Moroccans serving Moroccans. I do not feel proud if I could not serve adequately our citizens. We are an integral part of this institution. The reputation of this institution is our reputation. We are a small cell within a large cell that is the Ministry of Health. We work with devotion”. NHMH 16, Doctor.

“We care for the public service. We organized a community action in the former hospital location on a voluntary basis, without any instruction from the hospital administration. We noticed that the hospital garden was deteriorating. So, with other friends and union representatives, we refurbished the garden.” EJMH 23, Nurse.

“I worked for non governemental organisations, doing medical caravans with doctors in rural areas in Walidia. I started working for NGOs not affiliated to any political party in 1979-1980. I believe that working for NGOs educates young people. We grew up doing medical caravans, serving the population, we worked with doctors till 8 or 9 pm.” EJMH 41, Nurse.

Rational motives

Finally, we examined in how far respondents mention rational motives for working in the public service. We found that nurses who occupy managerial positions emphasised the importance of participating in decision-making and the social importance of their function. They feel proud when they are consulted and involved in decision-making. For some, this contributes to their motivation, as it allows them to more effectively serve the public.

“As the chief of this department, the most important thing for me is when I am involved in decision-making. I do not only report on problems. As a leader, I suggest solutions that get always approved by the hierarchy.” EJMH, 12 Nurse intermediate manager.

“This title [Chief Nursing Officer] allows me to commit to my job. As a chief nursing officer, I have an authority on all nurses in the hospital. Sincerely, this title motivates me because I am solicited by staff to provide them with the necessary support. This is true for both doctors and nurses. I am fully satisfied when my opinion is listened to and taken into consideration. This makes me more motivated to fill this position that allows me to serve people.” EJMH1, Senior manager, Nurse.

“We lose patients stupidly because of a lack of material. There is no material to work with. You see patients die in front of you and you do not have necessary tools to save them. These conditions are beyond our control.” SMBA 43, Doctor

This influenced negatively their well-being.

“When you do not have necessary material to work with, you are in trouble! It is not only a constraint, but a source of suffering. Instead of relieving patients’ distress, it is us who get stressed.” SMBA 45, Doctor

What contributes to public service motivation?

The respondents in both high- and poor-performing hospitals identified a number of factors that contribute to high PSM: family education, military service, volunteering, professionalism and religion.

Family education

Health workers with a high level of PSM explained how family education and childhood experiences contributed to their high sense of civic duty and a high orientation to civic participation. Parent modelling and education led them to serve others and to act for the common good from a non-self-interested perspective.

“I do not take bribes. I have an ideal about the role of doctors in society. Their role is not limited to being a care provider at the hospital. They should get out to the community, sensitize the population during health education sessions. I feel satisfied when we organize a round table with practitioners and local representatives, when we organize medical caravans, when we circumcise children for the sake of God. I learned these principles through my parents’ education. When we were young, we were educated to help people, to help others, neighbours, friends, siblings and family members. I cannot explain these things, I do not know if it is genetics, but we learn that we do not live alone but in a society. We depend on each other, we belong to a society, we live with neighbours and people. Everybody is leaning on others, like dominos.” SMBA 40, Surgeon

Military service

Some health workers explained that their commitment to public values is reinforced by their former experience in the mandatory military service. These experiences provided them with high sense of civic duty and citizenship, which subsequently contributed to their feelings of public service motivation.

“Every health worker, specifically males, should do military service, as I did. At that time, I went to Al Farssia, a remote area in the Moroccan territory in the middle of nowhere. I used to stay there between two to three months and then would go back to town for a short period of time. Every time I went back to that region, I recognized the true meaning of life and the true value of things, when I compared urban life to these remote areas.” NHMH 32, Doctor/Senior Manager.

“I was asked to join the military service. … I believe that in my country, we are more effective if we serve the poor, if we work for the interest of the most vulnerable citizens. The experience I had in the medical service in the military affected me a lot. It has shown me that Morocco needs more faithful and serious health workers.” SMBA 18, Doctor

Volunteering

The experience with volunteering in remote areas during medical outreach caravans shaped the feelins of PSM of some respondents. They identified this as a catalyser of self-sacrifice and civic participation.

“From 1970 to 1980, I was part of an NGO doing medical caravans with physicians in rural areas. Volunteering educates people. When you grow up volunteering, you will be loving to serve the population. I worked for 8 to 9 hours a day without being paid. I did it for the sake of God without waiting for external rewards. EJMH 41, Nurse.

Professionalism

Some respondents value the comfort of the patients and expressed a strong sense of ethical responsibility towards them. They related this to their professional ethics and believe this attitude is embedded in their professional identity. They asserted that their behaviour is less dependent on the supervision of their superiors than it is relying on their professional conscientiousness.

“I love my job. I cannot neglect my job. This is how I was educated and taught in the first place. I cannot let down a patient, even though I know that this (quality of care) does not depend only on me. I cannot. I try to help patients even if they are from other departments. I call the surgeon to deal with a patient with a suspicion of appendicitis, even if it is the job of the physician at the emergency department.” SMBA 24, Nurse,

Their commitment to serve the patients is integrated within their professional identity but also with their religion.

“ ‘Citizens’ means for me ‘professional conscientiousness’ [in Arabic, damir ]. I have to serve them conscientiously. It is important part of my personality. I do not like to fail to properly perform my duty. My money will be then halal [Compliant with what is permitted in Islam]. NHMH 13, Nurse

Religious beliefs

Our analysis shows that quite some interviewees expressed strong religious beliefs about expected rewards from “God” when properly serving patients. They explained their altruistic behaviour by their expectance of divine rewards (called ajre in Arabic) when serving patients. This was expressed by nurses, administrators and doctors alike.

“He [the patient] praises God for you, he says nice prayers! “May Allah [God] be pleased with you. May Allah be merciful to you. May you be covered by the grace of God”. At such moments, I feel reassured and relieved during my night shift. Then I go home fully satisfied.” SMBA 35, Nurse.

“When serving people, you earn ajre (divine rewards) from God, which is far better than money.” EJMH 18, Administrative staff.

‘This is between us and God, Glory be to Allah. It is between me and God who created me. We do good deeds when serving patients because we need ajre [divine rewards]. We do not know what might occur to us in the future. We seek ajre from God. He is the only one who will reward us for our effort”. SMBA 7, Nurse.

“There is a religious element that plays a role in my motivation. We have to make our money halal [meaning in compliance with Islamic rules]. This means I have to work, to serve people. We are equal to them. We are all Muslims. They [patients] are our fellow countrymen, our fellow citizens. We are obliged to give them back what we owe to our country.” NHMH 11, Surgeon, Intermediate-level manager.

“I am pleased when I treat patients decently and they pray God for you. I am pleased when I see that patients are happy and in a good state.” SMBA 35, Nurse

“Have mercy on those on earth, the one in Heaven will have mercy on you ... This is our real profession, to love others. I am a human, I can also become sick. We are all patients”. SMBA 17, Nurse

“In Arabic terms, we do this because we need to make our salary “ Halal .” [ Halal means compliant with Islamic rules]. “Thank God.” We said this because it is very important! We must make sure that our salary is halal before God and our own conscience. There is nobody watching over us, whether we came at 8 am in the morning or not”. EJMH 9 doctor

“[In the Prophet Muhammed sayings and teachings], the prophet said “Who served other muslims, is rewarded like someone who stayed a month praying in this mosque.” This explains why I came early at 8h30 am. Sometimes, when I come late to work, I stay late in compensation, hoping that this way, God will help my siblings.” NHMH 11, surgeon

“We are muslims. Then, satisfying the need of others is essential [referring to ‘ Hadith ’ ]. I feel satisfied and I enjoy that. We have an ideology that dominates our behaviour as muslims. We are not compensated directly but we get rewards later in other circumstances. Sometimes, I might serve and help this old lady we have just seen. By doing so, I might be helped in the future when I will experience the same situation.” NHMH 7, Doctor.

In this study, we explored how health workers from four Moroccan hospitals describe public service motivation. We found that most respondents expressed some form of public service motivation, both in the high- and the poor-performing hospitals. Our respondents referred to the main elements identified by Kim et al.: compassion, self-sacrifice, commitment to public values, and attraction to public service.

In our four sites, we could clearly distinguish between administrators on one hand and health professionals (nurses and doctors) on the other. We found, in line with other study findings [ 32 , 93 , 94 ], that health professionals identified compassion and self-sacrifice as major components of PSM, while administrative staff (except those in direct contact with patients) and managers tended to indicate commitment to public values, besides feeling attracted to policy-making. Our analysis suggests, similarly to other PSM studies [ 32 , 93 – 96 ], that the nature of professional work and the daily interaction with patients catalyses affective motives among healthcare providers. This is conditioned by the ability of health workers to help and ease the suffering of patients. In our study, similarly with other empirical findings [ 16 , 97 , 98 ], we found that health workers experiencing unnecessary deaths or harm to patients expressed high levels of psychological distress because of these events but also because of the conflict with their public service motivation.

Our study showed that managers, and nurses in management positions in particular, similar to previous studies [ 46 , 99 ], valued the importance of social status and respect from other staff. This in line with the dominant organisational culture in the Moroccan public sector, which emphasises the importance of social status [ 100 – 102 ].

Although PSM scholars usually do not consider that intrinsic motivation is an essential part of PSM [ 103 ], our study showed that health professionals are motivated by both PSM and intrinsic motivation and that both are sometimes difficult to disentangle. We found that doctors and nurses were highly intrinsically motivated by the task of caring and saving patients’ lives. They enjoyed the perceived self-efficacy and the esteem from patients and their relatives. We tend to agree with Grant and Berry [ 104 , 105 ] that intrinsic motivation is essential in maintaining the persistence of PSM in medical and nursing professions: the day-to-day interactions of these providers with patients shape and reinforce their helping role identity and this contributes to PSM [ 32 , 106 , 107 ]. Our analysis showed that PSM and intrinsic motivation are related to personal volition: the locus of causality is internal. In that sense we agree with Perry (1990) that health workers have a mix of motives: PSM, intrinsic motivation and extrinsic motives (supporting their families, job security and stability, work family balance). The relative importance at any given time in one’s career depends on contextual conditions and personal factors. This is line with findings from other studies in North African countries [ 101 , 108 , 109 ].

We also found that a number of respondents framed their public service motivation in religion-based motives, such as divine rewards (and Islam in general) and with roots in family education. This is similar to findings of studies in Morocco [ 56 , 110 – 112 ], Egypt [ 102 ], and Tunisia [ 101 ]. In as far as religion and family education are social institutions that shape the identity of health workers before they enter the public health sector, these can be considered as contextual elements. Future research could focus on the underlying mechanisms by which religion and education influence the formation of health workers’ identity and how that translates (or not) in PSM. Indications about this relationship is found in studies carried out in industrialised countries [ 38 – 40 , 113 – 117 ].

As mentioned in the introduction, this study on PSM is part of a larger project looking into the relationship between leadership, motivation and performance. Future research needs to examine the link between PSM and individual and hospital performance. We think managers of public hospitals should be aware of the differences in PSM among health workers and adapt their leadership practices accordingly. In the Moroccan context, they may need to emphasise spiritual and public values when communicating about organisational mission and objectives, and reinforce the interaction of their early-carrier health personnel with the underprivileged parts of the population during, for instance, medical outreach and other community-level initiatives. Finally, they also may ensure that adequate resources are provided for frontline healthworkers to allow them to experience valued outcomes such as saving patients’ lifes.

This study has a number of limitations. First, we did not measure actual levels of PSM, largely because of the time constraints which precluded validation of existing scales in Morocco. This study was thus necessarily exploratory in nature. Further studies should indeed validate, and adapt if necessary, the scales of for instance Kim et al. [ 83 ] in order to substantiate our results. Second, it would have been interesting to interview health workers who left the public service to explore their views on PSM. We also acknowledge that in hospital NHMH and RKMH, four planned FGDs were in practice carried out as group discussions because the necessary number of participants (6 to 8) was not reached.

While the notion of PSM is not part of the management discourse in Morocco, we found that PSM seems to be a ‘natural’ concept to health workers in Moroccan public hospitals. We found that hospital staff are motivated by different drivers of PSM and that religious-based beliefs infuse the notion of public service. Hospital managers should pay more attention to the nature of staff motivation (PSM, intrinsic and extrinsic) and adapt their leadership practices accordingly.

Supplementary information

Acknowledgement.

We would like to thank all participants from the four hospitals who participated in the study.

Declarations

This research is part of PhD research frameworks exploring the relationship between leadership and motivation of health workers.

Abbreviations

Authors’ contributions.

All the four authors contributed to the original design and writing of the manuscript. ZB carried out the data collection. BM cross-checked the transcripts. Initial coding was done by ZB and discussed between the research team members. ZB edited the final draft. All authors read and approved the final manuscript.

This work was funded through a PhD framework agreement between the Belgian Directorate-General for Development Cooperation and Humanitarian Aid, and the Institute of Tropical Medicine, Antwerp. The sponsors had no role in the study or in the writing of the paper.

Availability of data and materials

Ethics approval and consent to participate.

The research protocol was approved by the Moroccan Institutional Review Board (n°90/16) of the Faculty of Medicine of Pharmacy, Rabat and the Institutional Review Board of the Institute of Tropical Medicine, Antwerp (n° 1204/17). All participants have been informed prior to the conduct of the research and written consent forms were signed by the respondents and countersigned by the researcher. A signed copy was given to each respondents.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Zakaria Belrhiti, Email: moc.liamg@itihrlebrd .

Wim Van Damme, Email: eb.gti@emmadvw .

Abdelmounim Belalia, Email: moc.oohay@minuom_b .

Bruno Marchal, Email: eb.gti@lahcramb .

Supplementary information accompanies this paper at 10.1186/s12939-019-1053-8.