planning considerations in research health and social care

Patient and Public Involvement in Health and Social Care Research

An Introduction to Theory and Practice

• © 2020
• Jurgen Grotz 0 ,
• Mary Ledgard 1 ,
• Fiona Poland 2

Faculty of Medicine and Health Sciences, University of East Anglia, Norwich, UK

You can also search for this author in PubMed   Google Scholar

Norwich, UK

School of health sciences, university of east anglia, norwich, uk.

• Provides an overview of the historical and conceptual background and of new developments in Patient and Public Involvement (PPI) in research
• Includes a step-by-step guide to planning PPI in research
• Offers exercises and examples, including on ethics and critical perspectives

1672 Accesses

17 Citations

1 Altmetric

This is a preview of subscription content, log in via an institution to check access.

Access this book

• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Other ways to access

Licence this eBook for your library

Institutional subscriptions

Table of contents (7 chapters)

Front matter, introduction.

• Jurgen Grotz, Mary Ledgard, Fiona Poland

Historical and Conceptual Background of Public Involvement

Ppi in research practice, ppi in research ethics and ethics in ppi, critical perspectives on patient and public involvement in research, the coherence model, back matter.

• Health Studies
• Public Health
• Health Policy
• Social Care
• Participation
• Public Engagement
• Patient Involvement
• Community Research
• Health Research

About this book

‘This book is an inspiring reflection on public and patient involvement. The authors have captured the critical considerations for planning involvement in research, relating how meaningful, relevant and beneficial involvement is only possible when we have considered the reasons for doing it and tackled the structures that support unhelpful power imbalances.’

—Becki Meakin, General Manager of Shaping Our Lives: A National Network of Service Users and Disabled People, UK

Authors and Affiliations

Jurgen Grotz

Mary Ledgard

Fiona Poland

About the authors

Jurgen Grotz , PhD, is the Director of the Institute for Volunteering Research (IVR) at the University of East Anglia, UK. He has co-edited the Palgrave Handbook of Volunteering, Civic Participation, and Nonprofit Associations (2016).

Mary Ledgard is a retired researcher who has served as a trustee of Healthwatch Norfolk, UK, and is currently a PPI representative in the fields of health and social care.

Fiona Poland is Professor of Social Research Methodology at the University of East Anglia, UK and has worked for many years to promote inclusive community research. She is journal editor of Quality in Ageing and Older Adults .

Bibliographic Information

Book Title : Patient and Public Involvement in Health and Social Care Research

Book Subtitle : An Introduction to Theory and Practice

Authors : Jurgen Grotz, Mary Ledgard, Fiona Poland

DOI : https://doi.org/10.1007/978-3-030-55289-3

Publisher : Palgrave Macmillan Cham

eBook Packages : Social Sciences , Social Sciences (R0)

Copyright Information : The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2020

Hardcover ISBN : 978-3-030-55288-6 Published: 05 November 2020

Softcover ISBN : 978-3-030-55291-6 Published: 06 November 2021

eBook ISBN : 978-3-030-55289-3 Published: 04 November 2020

Edition Number : 1

Number of Pages : XXV, 163

Number of Illustrations : 6 illustrations in colour

Topics : Medical Sociology , Health Policy , Health Promotion and Disease Prevention , Microeconomics

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Open access
• Published: 17 April 2017

Understanding relevance of health research: considerations in the context of research impact assessment

• Mark J. Dobrow 1 ,
• Fiona A. Miller 1 ,
• Cy Frank 2 &
• Adalsteinn D. Brown 1  

Health Research Policy and Systems volume  15 , Article number:  31 ( 2017 ) Cite this article

14k Accesses

24 Citations

9 Altmetric

Metrics details

With massive investment in health-related research, above and beyond investments in the management and delivery of healthcare and public health services, there has been increasing focus on the impact of health research to explore and explain the consequences of these investments and inform strategic planning. Relevance is reflected by increased attention to the usability and impact of health research, with research funders increasingly engaging in relevance assessment as an input to decision processes. Yet, it is unclear whether relevance is a synonym for or predictor of impact, a necessary condition or stage in achieving it, or a distinct aim of the research enterprise. The main aim of this paper is to improve our understanding of research relevance, with specific objectives to (1) unpack research relevance from both theoretical and practical perspectives, and (2) outline key considerations for its assessment.

Our approach involved the scholarly strategy of review and reflection. We prepared a draft paper based on an exploratory review of literature from various fields, and gained from detailed and insightful analysis and critique at a roundtable discussion with a group of key health research stakeholders. We also solicited review and feedback from a small sample of expert reviewers.

Conclusions

Research relevance seems increasingly important in justifying research investments and guiding strategic research planning. However, consideration of relevance has been largely tacit in the health research community, often depending on unexplained interpretations of value, fit and potential for impact. While research relevance seems a necessary condition for impact – a process or component of efforts to make rigorous research usable – ultimately, relevance stands apart from research impact. Careful and explicit consideration of research relevance is vital to gauge the overall value and impact of a wide range of individual and collective research efforts and investments. To improve understanding, this paper outlines four key considerations, including how research relevance assessments (1) orientate to, capture and compare research versus non-research sources, (2) consider both instrumental versus non-instrumental uses of research, (3) accommodate dynamic temporal-shifting perspectives on research, and (4) align with an intersubjective understanding of relevance.

Peer Review reports

Various levels of government in Canada collectively invest multiple billions of dollars in health-related research per annum, above and beyond investments in the management and delivery of healthcare and public health services. In recognition of this sizeable collective commitment, much work has focused on the impact of health research to explore and explain the consequences of these investments and inform strategic planning. Relevance is tacit in the increased attention to the usability and impact of health research. Additionally, research funders increasingly engage in relevance assessment as an input to decision processes; yet, it is unclear whether relevance is a synonym for or predictor of impact, a necessary condition or stage in achieving it, or a distinct aim of the research enterprise. Therefore, the main aim of this paper is to improve our understanding of research relevance as it relates to research quality and research impact, with specific objectives to (1) unpack research relevance from both theoretical and practical perspectives, and (2) outline key considerations for the assessment of research relevance.

Globally, there has been increasing critical assessment of the value of health research investments [ 1 – 3 ], with growing interest in research impact assessment (RIA) in the health sector [ 4 – 6 ]. RIA focuses on understanding how research activity can directly and indirectly advance knowledge, influence decision-making, and effect health and socio-economic outcomes, with a small but growing body of work seeking to develop better measures to evaluate (and ideally attribute) the returns on health research investments [ 6 ]. The Canadian Academy of Health Sciences (CAHS) released a comprehensive report on the subject in 2009 that presented a call for action, with a number of recommendations including establishing collaborative efforts among Canadian research funders to advance frameworks and sets of indicators and metrics for health research impact [ 4 ]. The CAHS impact framework [ 4 ], which drew on the Buxton and Hanney [ 7 ] ‘payback model’, among others, has provided a thoughtful starting point for considering the impact of health research in Canada. Subsequent work by Alberta Innovates – Health Solutions (AIHS) on a Research to Impact Framework (described in Graham et al. [ 8 ]) provides further insights on operationalising RIA frameworks for health research in Canada.

These initiatives are part of a broadly discussed shift in approaches to knowledge production, from an emphasis on investigator-initiated, curiosity driven work judged and guided by scientists, to expanded approaches to knowledge production, drawing on a wider set of actors and approaches, and emphasising relevance and usability. This shift from science produced by and for scientists to knowledge production that is “ socially distributed, application-oriented, trans-disciplinary, and subject to multiple accountabilities ” [ 9 ] has been characterised as a shift from ‘mode 1’ to ‘mode 2’ knowledge regimes. In the language of mode 2, interest in research ‘impact’ expresses a concern for application or consequence, and – in the economic language of return on investment – a concern that the yield is at least equal to the investment in the research itself. Extending this reasoning, interest in research ‘relevance’ may reflect a concern for accountability – linking research to the actor(s) for whom the research is performed and who will, ideally, put it to use.

In Canada, interest in research impact and relevance appears to have been felt most forcefully in the context of health services and policy research, which has long been encouraged to orient to the needs of policymakers, health system planners and related decision makers. More recently, there has been increased attention to ensuring that all forms of health research are ‘patient oriented’ – that is, that the research is prioritised, conducted and applied in ways that are accountable to this important end user. This call has been picked up on several fronts, including by the Canadian Institutes of Health Research (CIHR), which released its Strategy for Patient-Oriented Research (SPOR) in 2011. The SPOR vision “…is to demonstrably improve health outcomes and enhance patients’ health care experience through integration of evidence at all levels in the health care system ” [ 10 ]. In some respects, it represents a fundamental re-orientation for the primary funder of health research in Canada.

Though relevance is tacit in attention to research impact and the wider concern with mode 2 knowledge production, explicit attention to the meaning or measurement of research relevance is limited. The CAHS and AIHS frameworks, for example, acknowledge ‘relevance’ of health research but do not clearly define the term nor describe approaches for assessing it [ 4 , 8 ]. Rather, these frameworks emphasise the role of broad stakeholder engagement approaches and feedback mechanisms as methods for addressing relevance. For example, the AIHS framework notes the challenge of, and need to, move “ …beyond the collection of traditional scientific indicators […] to include measures of greater interest to the broader stakeholder community… ” [ 8 ] without stating explicitly how “ greater interest ” or related concepts such as relevance should be judged. As currently constructed, these RIA frameworks provide important advances in how we think about the impact of health research, but they were not intended to provide guidance specifically to the assessment of the relevance of health research.

Despite this lack of specific guidance on research relevance from a scholarly or measurement perspective, attention to it as a practical component of health research funding and organisation is evolving. There is, for example, growing use of ‘relevance assessment’ by research funders. The Canadian Health Services Research Foundation, in particular, was an innovator in incorporating relevance review into its applied research funding programmes, including promoting partnerships and knowledge translation (KT) with health system stakeholders [ 11 ]. Current applications for funding from the Institute of Gender and Health at CIHR go through ‘relevance review’ ( http://www.cihr-irsc.gc.ca/e/45212.html ). Similarly, applications for Ontario’s Health System Research Fund are judged based on ‘internal review of relevance and impact’ ( http://www.health.gov.on.ca/en/pro/ministry/research/cihr.aspx ). However, given the lack of conceptual clarity on research relevance, and in particular, how relevance assessment aligns with and differs from impact assessment, there is a critical gap in our understanding that has implications for both its contemporary and ongoing application and our ability to make sound research investment decisions.

This work was commissioned by the Ontario SPOR SUPPORT (Support for People and Patient-Oriented Research and Trials) Unit (OSSU) – one of several units established at provincial and regional levels across Canada to work with CIHR in pursuing the SPOR. Like other research organisations, OSSU saw the need to consider the relevance of the research it supported, and it established both scientific and relevance advisory committees as part of its original governance structure [ 12 ], tasking the latter to “ …develop a measure, or small set of strategic measures, that serves to inspire the Ontario research, implementation, provider and patient communities to come together to make a difference for patients ” [ 12 ]. In the spirit of research and scholarship, OSSU then asked what exactly this commitment to research ‘relevance’ entailed.

Our approach to answering this question involved the scholarly strategy of review and reflection. As with the early investigations into research impact assessment, we were surprised to find so little reflexive attention to the topic within the health research community [ 13 ]. We prepared a draft paper based on an exploratory review of literature from various fields, and gained from detailed and insightful analysis and critique at a roundtable discussion with a small group of key health research stakeholders. We also solicited review and feedback from a small sample of expert reviewers.

The structure of our paper is as follows. First, to ‘unpack’ the concept of relevance, we review theoretical literature and then consider practical work both from within and outside the health sector, to ask what has been argued and concluded about the nature of relevance and its appropriate assessment. Next, we outline a series of forward-looking considerations for assessing research relevance and conclude with reflections on how research relevance assessment fits with evolving interest in RIA.

Unpacking relevance

Theoretical perspectives.

Before considering the relevance of health research, we need to step back and consider what we mean by the term ‘relevance’. A range of descriptors is often used to define relevance, including ‘pertinent to…’, ‘bearing upon…’, ‘connected with…’, or ‘appropriate to…’, ‘…the matter at hand’, as well as ‘germane’, ‘apropos’, ‘material’, ‘applicable’ and ‘satisfactory’. A large body of dedicated theoretical work on relevance, drawn from many fields and perspectives, such as computer science, information science, statistics/probability theory, artificial intelligence, cognitive science, epistemology, linguistics and jurisprudence [ 14 ], reflects its importance but also the challenge for establishing a common understanding of the term [ 14 , 15 ]. For example, Gärdenfors [ 16 ], in his discussion on the logic of relevance, noted that “ …relevance ought to be a central concept in the philosophy of science… ” given the position that “ …it is only relevant information that is of any importance… ” (p. 351). However, from a ‘research’ relevance perspective, the theoretical work on relevance has been linked to ‘information’, ‘evidence’, ‘reasoning’, ‘argument’ and ‘decision’ [ 15 – 18 ], each presenting variable framing that impedes practical definition or consistent comprehension of the term. Floridi [ 14 ] recently suggested that existing theories are “ …utterly useless when it comes to establish the actual relevance of some specific piece of information ” (p. 69), and goes on to advance a ‘subjectivist’ interpretation, with relevance judged by the questioner. While a subjectivist approach to relevance is intuitively appealing, its contribution to the assessment of research relevance presents particular challenges that we will discuss later in the paper.

Another approach to unpacking relevance is to consider the theoretical model behind the broad-based research strategies that have governed research investments and policies in high-income countries since the end of the Second World War. For the better part of the 20th century, a linear model was the dominant conceptual framework, whereby basic research was viewed as a necessary input for applied research, which then led to development and production [ 19 , 20 ]. In the late 1990s, an alternate thesis was introduced when Stokes proposed a new model for broad-based research strategy – known as Pasteur’s Quadrant – that highlighted the conceptual relationship between the ‘quest to understand’ and ‘practical needs’ [ 21 ]. While some research is clearly focused on advances in basic research (e.g. Niels Bohr’s foundational research on atomic structure and quantum theory), and some research is clearly focused on applied problems (e.g. Thomas Edison’s practical inventions), Stokes emphasised the potential for use-inspired basic research (e.g. Louis Pasteur’s foundational research on microbiology that addressed contemporaneous population health challenges). Pasteur’s Quadrant invokes consideration of ‘relevance’ with some commentators framing the two-by-two relationship as the relevance for advancement of basic knowledge and the relevance for immediate application [ 22 ]. Stokes’ model adds conceptual insight on the role of relevance when considering the value of research to society, however, it was not intended to specifically conceptualise the term and does not distinguish it from other related concepts such as research impact or value. Therefore, to provide further insights, we next consider relevance in practical settings.

Health sector perspectives

In the health sector, the idea that research should be ‘relevant’ is commonplace. Commitments to ‘knowledge translation’ and the ‘knowledge to action cycle’ [ 23 ] emphasise issues of relevance and provide considerable insight into approaches to ensuring research usability and use. At the same time, the health research community has given disproportionate attention to issues of research quality, with an emphasis on internal validity that may downplay external validity and suggest some tension between rigour and relevance. Thus, though the concept of relevance is of central importance to the health research enterprise, the failure to unpack it or explore it both theoretically and practically leaves room for misunderstanding and misapplication.

In the health sector, research relevance often arises as a practical question of the ‘fit’ between a body of knowledge or research approach and a specific field or issue (e.g. public health, primary healthcare, healthcare access, genomics, alternative healthcare, healthcare reform in rural areas). The results of two recent International Society for Pharmacoeconomics and Outcomes Research task forces take this approach. The task forces developed questionnaires to assess the relevance and credibility of research other than randomised controlled trials (e.g. observational research, meta-network analysis) to inform healthcare decision-making [ 24 , 25 ]. Both make similar observations about relevance, reinforcing the subjectivist approach noted earlier, and can be summarised by the following statement by Berger et al. [ 24 ]:

“ Relevance addresses whether the results of the study/apply [sic] to the setting of interest to the decision maker. It addresses issues of external validity similar to the population, interventions, comparators, outcomes, and setting framework from evidence based medicine. There is no correct answer for relevance. Relevance is determined by each decision maker, and the relevance assessment determined by one decision maker will not necessarily apply to other decision makers. Individual studies may be designed with the perspective of particular decision makers in mind (e.g. payer or provider) ” (p.148, emphasis added).

Research relevance in health is also noted in discussion and debate regarding the value of qualitative research relative to the more established forms of quantitative health research. For example, Mays and Pope [ 26 ] suggest that qualitative research can be assessed “… by two broad criteria: validity and relevance ”. Their further discussion provides some insight into the several ways that research might be relevant, suggesting that:

“[r] esearch can be relevant when it either adds to knowledge or increases the confidence with which existing knowledge is regarded. Another important dimension of relevance is the extent to which findings can be generalised beyond the setting in which they were generated ” [ 27 ].

The work of Mays and Pope positions research relevance amidst the longstanding tension between internal and external validity. This tension reflects opposing foci on internal validity as the quality/rigour of research methodology and external validity as the applicability/transferability of research to other settings or contexts. While external validity is not the only measure of relevance – as research may remain relevant to some contexts even when not generalisable to others – it is an important component, and one that has not always attracted sufficient attention. For example, the Canadian health research community has focused considerable practical attention on internal validity as a critical component of evidence for clinical and health policy decisions. Evidence-based medicine, the Cochrane Collaboration, the Canadian and United States task forces on preventive healthcare/services and a long list of aligned groups have developed and established many tools to assess the quality of research evidence (e.g. GRADE [ 28 ]), with a predominant focus on issues of internal validity, and an emphasis on evidence hierarchies that is sometimes seen to be incompatible with ‘real world’ relevance. The relative lack of similar approaches or tools that focus on external validity in health research is notable, though movements to marshal evidence in support of sound public policy, such as the Campbell Collaboration, have attended to issues of external validity in other areas of health and social policy [ 29 ]. Further, there are emerging approaches and tools for documenting the external validity of health research and facilitating its use [ 30 ]. For example, WHO has supported the development of workbooks to contextualise health systems guidance for different contexts [ 31 ] and the field of local applicability and transferability of research has emerged to facilitate the adaptation of interventions from one setting to another, including the development of some well-documented tools like RE-AIM [ 32 ].

Alongside these emerging approaches and tools sits the established field of KT. KT has a strong history in Canada with a distinctive feature being a reliance on stakeholder engagement to support a commitment to improve research relevance. For example, the AIHS framework relies heavily on KT and stakeholder engagement approaches as part of its RIA, describing the mobilisation of knowledge through “ …a process of interactions, feedback, and engagement using a variety of mechanisms (e.g. collaborations, partnerships, networks, knowledge brokering) with relevant target audiences (i.e. actors and performers) across the health sector ” ([ 8 ] p. 362). Experience in stakeholder engagement, particularly with clinical, management and policy decision-makers, has become fairly extensive and there is now increased attention on engaging patients as core stakeholders in health research. If relevance is truly subjective, then KT efforts (including engagement, dissemination, promotion, communication) would appear to represent reasonable approaches for articulating, conveying and improving research relevance. However, if there are underlying elements of relevance that are more universal, then there is a risk that KT efforts – and subjectivist approaches to ensuring relevance – are akin to commercial marketing or communication strategies where the aim is to ‘sell’ more product and/or generate more influence that may not align with a more objective lens.

In sum, the health research community in Canada has a longstanding history of critically appraising research quality based on study design and research methodology, with greater emphasis on internal rather than external validity. As the same time, there is established expertise in KT, emphasising engagement with research users and adaptation to settings or contexts of use – approaches that may imply a subjectivist interpretation of relevance. Thus, while relevance is an important concept for the health research enterprise, its use is largely tacit and taken for granted.

Non-health sector perspectives

To unpack relevance further we consider some non-health sector perspectives that give attention to the term, often with formal definitions or taxonomies established. Examples include the legal, financial accounting, education and web search (information retrieval) sectors, each of which are briefly described below.

From a legal perspective, relevance has a specific meaning that relates to the admissibility of evidence in terms of its probative value (i.e. the extent to which evidence contributes to proving an important matter of fact) [ 33 ]. For example, a common objection to legal testimony or evidence is that it is ‘irrelevant’ [ 34 ]. Legal processes for considering the admissibility or legal-relevance of evidence are firmly established, requiring explicit declaration of evidentiary sources and direct consideration of that evidence as it relates both to a specific case and related historical precedents, something that is undeveloped in the health sector [ 35 ]. It is the formality, explicitness and retrospective nature of this process, which is directly associated with a specific case (or decision), that is characteristic of the consideration of relevance in the legal context.

Financial accounting provides another perspective on relevance. In this field, relevance is viewed as a fundamental component of generally accepted accounting principles. Relevance and materiality are emphasised such that accountants and auditors focus on financial information that meets the decision-making needs of users and is expected to affect their decisions. In financial accounting, ‘value relevance’ provides a more focused perspective on relevance, defined as “ …the ability of information disclosed by financial statements to capture and summarise firm value. Value relevance can be measured through the statistical relations between information presented by financial statements and stock market values or returns ” [ 36 ]. Similar to the legal perspective, the financial accounting perspective on relevance is set with a formal context, where the focal point (i.e. financial performance) is clear and principles (i.e. generally accepted accounting principles) and processes (i.e. financial reporting and auditing) are clearly established and monitored.

Education provides a slightly more expansive approach to operationalising relevance, given the more general aim of the enterprise. In the United States, the Glossary of Education Reform [ 37 ] notes that “ …the term relevance typically refers to learning experiences that are either directly applicable to the personal aspirations, interests, or cultural experiences of students (personal relevance) or that are connected in some way to real-world issues, problems, and contexts (life relevance) ”. They further state that “ personal relevance occurs when learning is connected to an individual student’s interests, aspirations, and life experiences ”, while “ life relevance occurs when learning is connected in some way to real-world issues, problems, and contexts outside of school ”. A similar framing of relevance in this context suggests that it “…extends the learning beyond the classroom by teaching students to apply what they are learning to real world situations ” [ 38 ]. While the education sector also makes numerous references to a ‘rigour and relevance’ dyad [ 39 ] in contrast to the dominance of the internal validity focus in healthcare, it is the prominent dual focus on ‘personal’ relevance (with its subjectivist orientation) and ‘life’ or ‘real world’ relevance (with its more universal orientation) that seems to most clearly define the education sector’s perspective on relevance.

One of the most intensive and competitive sectors focusing on relevance is the web search (or information retrieval) field. This includes dominant search engines such as Google and Bing, as well as a wide range of commercial and social media sites such as Amazon, eBay, Facebook and LinkedIn, that compete either directly or indirectly on their ability to identify relevant information in response to user queries. Therefore, the ability of these organisations to advance the theory and practice related to relevance is fundamental to their success. For example, Google was built upon the effectiveness of its search algorithm, which is in a constant state of evolution. Both explicit and implicit approaches to assess relevance are used to contribute to search algorithm refinements [ 40 ]. The explicit approach focuses on ‘relevance ratings’, whereby evaluators (e.g. human raters) are contracted to assess the degree of ‘helpfulness’ of search results paired to specific search queries [ 41 ]. The implicit approach to assess relevance monitors and aggregates search behaviour of millions of users who are likely unaware that their behaviour is being assessed. Google has more recently advanced ‘personalised relevance’, which uses past individual search behaviour to personalise/tailor future search results for the same individual. Pariser has critiqued this concept as “ the filter bubble ” [ 42 ], warning that Google’s intent to optimise search algorithms for personal relevance creates a “ …personal ecosystem of information… ” that limits the diversity of search results and promotes insularity. This personal relevance is situated within the pervasiveness of social media, which facilitates the advancement of ‘social relevance’. Personal and social relevance highlight two important orientations towards relevance – one built on increasingly detailed understanding of individual preferences and the other reflecting the growing power and increasing accessibility of crowd-sourced perspectives. Overall, web search has made important contributions to how we understand and operationalise relevance, including the use of increasingly sophisticated explicit and implicit feedback mechanisms and the ability to draw upon and analyse big data sets. Web search has also exposed the contrasting orientations of personal and social relevance that underscore the challenges of combining or integrating different relevance assessments.

These non-health sector perspectives on relevance highlight several considerations. First, they reinforce general findings that point to perspective, decision context, timeliness and precision of focus or ‘fit’ as key elements of relevance. Additionally, they highlight a few distinctive considerations. The formalistic contexts of financial accounting and law emphasise issues such as precedent and legitimacy, implying that relevance in a research sense might require the demonstration of some legitimate or credible association between research and its use or user, among other considerations. Further, the complex consumerist world of social media highlights some of the challenges of a purely subjectivist definition of relevance. Whereas the International Society for Pharmacoeconomics and Outcomes Research guidance takes a subjectivist stance in suggesting that, “[t] here is no correct answer for relevance ” [ 24 ], the “ filter bubble ” criticised by Pariser [ 42 ] suggests otherwise. Relevance solely to the personally-perceived interests of a research user is unlikely to adequately serve the collective commitments to health and health equity that are especially germane to the health research enterprise.

Forward-looking considerations for assessing the relevance of health research

To this point, we have endeavoured to unpack relevance from theoretical and practical perspectives. In light of these insights and in the context of persistent interest in research impact assessment and evolving interest in research relevance, we now turn to some specific forward-looking considerations for research relevance assessment (RRA).

Relevance of research versus everything else

The first consideration for RRA is the acknowledgement that research is only one of many sources of insight to inform the needs or actions of research users. A research user is influenced by a wide range of political, legal, media, economic and other contextual information, interactions and experiences, as well as prevailing organisational governance, leadership, culture and values that all serve to complement (and often dominate) any insights that might be derived from research [ 43 ]. This reality implies that ‘relevance’ has a different meaning for researchers and research users. Researchers are typically interested in the relevance of a specific research product or activity for identifiable actions of (potentially) multiple research users; relevance is here judged relative to both the perceived needs of research users, and the extent and content of other related research. In contrast, research users are typically focused on identifying multiple relevant inputs to guide a specific action, only some of which may be research; relevance is here judged relative to both the research user’s needs and the form and content of the other inputs.

Given these distinct orientations to research relevance, RRA needs to be explicit about its comparative lens. Clear distinctions should be made between relevance based on the merits of the research product or activity (researcher lens) and relevance based on the relative value of research compared to other research and non-research sources (research user lens). RRA provides an opportunity to build more robust ways to characterise and assess the contribution of research to research users, including a more systematic and transparent articulation of anticipated research uses (akin to the Research Councils UK’s ‘Pathways to Impact’ [ 44 ] or descriptions of planned study design and methodological approach published in study protocols/registrations for randomised controlled trials or systematic reviews).

Beyond instrumental uses of research

The considerations noted above rely heavily on instrumental uses of research. Theoretically derived definitions of relevance, such as Floridi’s [ 14 ], tend to focus on the response to a specified question. This suggests a direct and tangible connection between research and its ‘use’. However, as Weiss [ 43 ] and others have observed, most types of research use are not instrumental, where use is documented and explicitly addresses a specific query or challenge for a research user. Rather, research use tends to be more conceptual, where use is indirect and evolves over time, or symbolic, where use may be politically or tactically motivated [ 43 ]. Research may also create externalities or unintended effects. For example, general research activity might support an engaged learning environment, interactive research relationships, and additional research-related discourse that provides benefits that are not attributable to any specific research product or activity. This has important ramifications for how research is funded and the role that relevance can play in that assessment. Ultimately, RRA needs to go beyond a singular understanding of research use as instrumental use, to develop better methods for capturing and assessing the relevance of the many non-instrumental uses of research.

The temporal factor

Another closely related consideration for RRA is the temporal context. Almost all research is conducted in a temporally defined period. Yet, while the quality of research is typically characterised by its methodology, which is a static feature typically not subject to temporal variation (e.g. the assessed quality of a randomised controlled trial should be consistent over time), relevance of research can be considered at any time (e.g. prior to the initiation of a research study or at different points in time post-completion) and is therefore subject to dynamic perceptions as they pertain to evolving action or decision contexts. Cohen [ 15 ] suggests that “ …relevance, like reasoning, has a prospective dimension as well as a retrospective one. It helps prediction as well as explanation ” (p. 182). The important insight is that, in contrast to research quality, the relevance of a specific research product can change over time, making assessment of research relevance more challenging.

This requires RRA to acknowledge the temporal factor and its associated implications for research relevance. At minimum, RRA should specify the temporal context as either pre-research (e.g. proposal/funding stage) or post-research (e.g. after research results have been produced). RRA at the pre-research stage focuses on proposed inputs and hypothetical outputs and outcomes, and may be more likely to overestimate instrumental research use and underestimate non-instrumental use. RRA at the post-research stage focuses mainly on the importance and value of actual outputs and tangible results, and may capture more non-instrumental research use. The pre-research stage is clearly aligned with research funding/investment processes, while the post-research stage can contribute to retrospective return-on-investment calculations and more general research impact assessment. However, employing this simple temporal categorisation should not lead us to lose sight of the dynamic, iterative nature of research relevance and the opportunity to assess it at interim and ongoing stages that captures re-interpretations or re-applications of research findings over time.

Moving from a subjective to an intersubjective understanding of relevance

An underlying theme in our review of relevance is subjectivity. Consider the broad scientific paradigms of positivism and interpretivism that are typically respectively aligned with research quality and research relevance. Research quality can be viewed as relating to characteristics or features that are assessed objectively, while research relevance may be seen as subjectively adjudicated. The subjective focus emphasises the variability of different perspectives and contexts and the suggestion that anyone can have a different take on the relevance of a specific research product or activity. For RRA, this reinforces a user-centred orientation to relevance assessment that privileges the judgment of the interrogator and raises the key question regarding who is positioned as the main arbiter of research relevance.

However, while relevance may never be characterised as universal, it could be argued that it is not purely subjective either. Rather, relevance may be more consistent with an intersubjective understanding that emphasises the extent of agreement or shared understanding among individual subjective perspectives representing a way to bridge the personal and the universal. The intersubjective view, while not presenting an objective approach to measuring relevance, does provide a road towards a meaningful and structured assessment of research relevance. It also emphasises the importance of representation in forging the intersubjective judgments that guide the research enterprise.

This paper has unpacked research relevance from different perspectives and outlined key considerations for its assessment. Alongside research impact assessment, research relevance seems increasingly important in justifying research investments and guiding strategic research planning. Indeed, judgments of ‘relevance’ are becoming a key component of the health research enterprise. However, consideration of relevance has been largely tacit in the health research community, often depending on unexplained interpretations of value, fit and potential for impact. Reviewing the various uses of relevance in health research, the concept is sometimes used as a synonym for research impact or positioned as a reliable predictor of later consequence. In many ways, research relevance seems a necessary condition for impact – a process or component of efforts to make rigorous research usable. However, relevance is not a necessary or sufficient condition to achieve impact. We expect that research that is relevant, and thus accountable to specific and legitimate users, will be impactful, but this may not necessarily be the case where other factors intervene. Additionally, we may expect that research that is impactful will be appropriately accountable – but again, this is not necessarily the case. Ultimately, relevance stands apart from research impact. Like rigour, relevance is a complementary but distinctive dimension of what it is that ensures ‘the good’ in health research.

While ‘relevance’ is ever-present, understanding of the concept in terms of health research is emergent and not well codified. To improve our understanding, this paper outlines four key considerations, including how research relevance assessments (1) orientate to, capture and compare research versus non-research sources, (2) consider both instrumental versus non-instrumental uses of research, (3) accommodate dynamic temporal-shifting perspectives on research, and (4) align with an intersubjective understanding of relevance. We believe careful and explicit consideration of research relevance, guided by transparent principles and processes is vital to gauge the overall value and impact of a wide range of individual and collective research efforts and investments. We hope this paper generates more discussion and debate to facilitate progress.

Abbreviations

Alberta Innovates – Health Solutions

Canadian Academy of Health Sciences

Canadian Health Services Research Foundation

Canadian Institutes of Health Research

knowledge translation

Ontario SPOR SUPPORT Unit

research impact assessment

research relevance assessment

Strategy for Patient-Oriented Research

Kleinert S, Horton R. How should medical science change? Lancet. 2014;383(9913):197–8.

Article   PubMed   Google Scholar  

Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, Howells DW, Ioannidis JPA, Oliver S. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156–65.

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.

Panel on the Return on Investments in Health Research. Making an impact: a preferred framework and indicators to measure returns on investment in health research. Ottawa: Canadian Academy of Health Sciences; 2009.

Google Scholar  

Banzi R, Moja L, Pistotti V, Facchini A, Liberati A. Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews. Health Res Policy Syst. 2011;9:26.

Article   PubMed   PubMed Central   Google Scholar  

Milat AJ, Bauman AE, Redman S. A narrative review of research impact assessment models and methods. Health Res Policy Syst. 2015;13:18.

Buxton M, Hanney S. How can payback from health services research be assessed? J Health Serv Res Policy. 1996;1:35–43.

CAS   PubMed   Google Scholar  

Graham KER, Chorzempa HL, Valentine PA, J M. Evaluating health research impact: Development and implementation of the Alberta Innovates – Health Solutions impact framework. Res Eval. 2012;21:354–67

Nowotny H, Scott P, Gibbons M. Introduction: ‘Mode 2’ Revisited: The New Production of Knowledge. Minerva. 2003;41(3):179–94.

Article   Google Scholar  

Canadian Institutes of Health Research. Strategy for Patient-Oriented Research. 2013. http://www.cihr-irsc.gc.ca/e/41204.html . Accessed 27 Jul 2014.

Lomas J. Preface: The first ones over the barricade. In: Potvin L, Armstrong P, editors. Shaping Academia for the Public Good: Critical Reflections on the CHSRF/CIHR Chair Program. Toronto: University of Toronto Press; 2013.

Ontario Ministry of Health and Long-Term Care – Community and Health Promotion Branch. Ontario Support for People and Patient-Oriented Research and Trials (SUPPORT) Unit: Business Plan. Toronto, ON: MOHLTC; 2013.

Buxton M. The payback of ‘Payback’: challenges in assessing research impact. Res Eval. 2011;20(3):259–60.

Floridi L. Understanding epistemic relevance. Erkenntnis. 2008;69(1):69–92.

Cohen LJ. Some steps towards a general theory of relevance. Synthese. 1994;101(2):171–85.

Gärdenfors P. On the logic of relevance. Synthese. 1978;37(3):351–67.

Schlesinger GN. Relevance Theoria. 1986;57(1):57–67.

Keynes JM. A treatise on probability. London: MacMillan and Co. Limited; 1921.

Bush V. Science - the endless frontier. A report to the President on a program of postwar scientific research. Washington: National Science Foundation; 1945.

Dudley JM. Defending basic research. Nat Photonics. 2013;7:338–9.

Article   CAS   Google Scholar  

Stokes DE. Pasteur's quadrant - basic science and technological innovation. Washington: Brookings Institution Press; 1997.

Tushman M, O'Reilly C. Research and relevance: implications of Pasteur's Quadrant for doctoral programs and faculty development. Acad Manag J. 2007;50(4):769–74.

Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13–24.

Berger ML, Martin BC, Husereau D, Worley K, Allen JD, Yang W, Quon NC, Mullins CD, Kahler KH, Crown W. A questionnaire to assess the relevance and credibility of observational studies to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health. 2014;17(2):143–56.

Jansen JP, Trikalinos T, Cappelleri JC, Daw J, Andes S, Eldessouki R, Salanti G. Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health. 2014;17(2):157–73.

Mays N, Pope C. Qualitative research in health care: assessing quality in qualitative research. BMJ. 2000;320(7226):50–2.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Pope C, Ziebland S, Mays N. Qualitative research in health care: analysing qualitative data. BMJ. 2000;2000(320):114–6.

Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ, for the GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924–6.

Campbell Collaboration. Campbell Collaboration Systematic Reviews: Policies and Guidelines (Version 1.2). 2016. https://www.campbellcollaboration.org/library/campbell-collaboration-systematic-reviews-policies-and-guidelines.html . Accessed 27 Mar 2017.

Burchett H, Umoquit M, Dobrow M. How do we know when research from one setting can be useful in another? A review of external validity, applicability and transferability. J Health Serv Res Policy. 2011;16(4):238–44.

World Health Organization. WHO recommendations: Optimizing health worker roles to improve access to key maternal and newborn health interventions through task shifting. Annex 8 Contextualizing the guidelines – workbook. 2012. http://www.who.int/reproductivehealth/publications/maternal_perinatal_health/Annex_8_Contextualizing_Workbook.pdf?ua=1 . Accessed 27 Sep 2016.

Re-aim.org. What is RE-AIM. 2017. http://re-aim.org/about/what-is-re-aim . Accessed 27 Mar 2017.

Center for the Study of Language and Information, Stanford University. Stanford Encyclopedia of Philosopy: The legal concept of evidence. 2015. http://plato.stanford.edu/entries/evidence-legal/ . Accessed 27 Sep 2016.

Cornell University Law School. Rule 402. General admissibility of relevant evidence. 2016. https://www.law.cornell.edu/rules/fre/rule_402 . Accessed 27 Sep 2016.

Giacomini M. One of these things is not like the others: the idea of precedence in health technology assessment and coverage decisions. Milbank Q. 2005;83(2):193–223.

Karğın S. The Impact of IFRS on the Value Relevance of Accounting Information: Evidence from Turkish Firms. Int J Econ Finance. 2013;5(4):71–80.

Abbott S. Relevance. In: Abbott S, editor. The glossary of education reform. Portland: Great Schools Partnership; 2013.

Pearson. Rigor and relevance: an overview. 2014. http://www.californiareading.com/media/pdf/rigor_and_relevance.pdf . Accessed 12 Oct 2014.

International Center for Leadership in Education. The Rigor Relevance Framework. 2016. http://www.leadered.com/our-philosophy/rigor-relevance-framework.php . Accessed 27 Sep 2016.

Google. Google Inside Search – Algorithms. 2016. http://www.google.ca/insidesearch/howsearchworks/algorithms.html . Accessed 27 Sep 2016.

Google. Search Quality Evaluator Guidelines. 2016. Accessed 28 Mar 2016.

Pariser E. The Filter Bubble: What the Internet is Hiding from You. New York: Penguin Group Inc.; 2011.

Weiss C. The many meanings of research utilization. Public Adm Rev. 1979;39(5):426–31.

Research Councils UK. Pathways to Impact. 2014. http://www.rcuk.ac.uk/innovation/impacts . Accessed 22 Feb 2017.

Download references

Acknowledgements

We acknowledge and appreciate the contributions of participants of a roundtable discussion to gather feedback on an earlier version of this paper. Participants included Simon Denegri, National Director for Public Participation and Engagement in Research, National Institute for Health Research (NIHR) UK, and Chair of INVOLVE, UK; Lee Fairclough, Vice-President, Quality Improvement, Health Quality Ontario; Michael Hillmer, Director, Planning, Research and Analysis Branch, Ontario Ministry of Health and Long-Term Care; John McLaughlin, Chief Science Officer and Senior Scientist, Public Health Ontario; Allison Paprica, Director, Strategic Partnerships, ICES; Michael Schull, President and CEO, ICES; and Vasanthi Srinivasan, Executive Director, Ontario Strategy for Patient-Oriented Research (SPOR) SUPPORT Unit (OSSU). We also want to thank John Lavis of the McMaster Health Forum for his very helpful comments on an earlier draft. Though we owe these individuals and organisations many thanks for their insights and support, we alone are responsible for the final product.

This work was commissioned by the Ontario SPOR Support Unit (OSSU). The executive director of the OSSU was one of the participants in a roundtable discussion to gather feedback on an earlier version of this paper, but beyond that, the OSSU did not have any role in the design of the study, collection, analysis or interpretation of the data, or writing of the manuscript.

Availability of data and materials

Not applicable. No datasets were generated or analysed during the development of the article.

Authors’ contributions

ADB acquired funding for the study. MJD, FAM and ADB conceptualised the study. MJD, FAM, CF and ADB participated in the review and writing of the manuscript. MJD, FAM and ADB participated in the roundtable discussion. MJD, FAM and ADB reviewed and approved the final version of the manuscript (CF passed away prior to submission of the manuscript).

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and affiliations.

Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, 155 College Street, 4th Floor, Toronto, ON, M5T 3M6, Canada

Mark J. Dobrow, Fiona A. Miller & Adalsteinn D. Brown

Alberta Innovates - Health Solutions, Edmonton, Alberta, Canada

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mark J. Dobrow .

Additional information

This article is dedicated to the memory of Dr Cy Frank, our co-author and esteemed colleague, whose untimely death occurred midway through development of this work. Among his many interests, Dr Frank was a champion for improving understanding of research impact assessment and provided many insights on the concept of research relevance, some of which we expand upon in this article. His many contributions to the health sector will live on, but he will be greatly missed.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Dobrow, M.J., Miller, F.A., Frank, C. et al. Understanding relevance of health research: considerations in the context of research impact assessment. Health Res Policy Sys 15 , 31 (2017). https://doi.org/10.1186/s12961-017-0188-6

Download citation

Received : 27 September 2016

Accepted : 07 March 2017

Published : 17 April 2017

DOI : https://doi.org/10.1186/s12961-017-0188-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health research systems
• Research relevance
• Research impact

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Member Organisations
• Research, Implementation and Impact
• Social Care and Social Work
• Social Care - Kent, Surrey & Sussex Region
• Starting Well: Children's Mental Health
• Primary and Community Health Services
• Living Well with Dementia
• Co-production
• Public Health
• Digital Innovation
• Economics of Health and Social Care
• Implementation
• Public and Community Involvement
• What is PCIE?
• Getting Involved
• Support for researchers
• Introduction
• An overview of ARC KSS
• Who can help you with PCIE?
• Terminology and PCIE
• Why and When to do PCIE
• Why involve members of the public in research?
• Why do members of the public want to get involved in research?
• When to get members of the public involved?
• How to do PCIE
• Communication and Accessibility
• Planning for involvement and engagement
• How to recruit members of the public
• Equality, Diversity and Inclusion (EDI)
• Approaches – different options for undertaking PCIE
• Facilitating meetings and Group dynamics
• Impact and Evaluation
• Examples/Case Studies
• Appendix/Useful tools
• Learning and Development

• Learning and Development Finder
• Researchers Journeys

• Research Week
• Previous Research Weeks

• Resource Library

Your Health and Social Care Research Quick Guide

This guide is designed for those in practice to navigate the world of research.

In this section

Produced by  Dr Diana Ramsey , ARC KSS Darzi Fellow (2021-2022) with insights and support from people working to deliver research in the Health and Social Care region, the  Your Health and Social Care Research Quick Guide  is a resource to help those working in health or social care to start or develop their research journey.

Packed with information, links and personal stories from people working across Kent, Surrey and Sussex, this guide has been designed to answer any questions you might have in regards to getting involved in research, building networks, research training and resources.

Whatever your research experience, this guide is for you.

View guide here

Subscribe to our newsletter

Sign up here

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please visit this page . We use e-shot as our marketing platform. Learn more about e-shot's privacy practices here .

Cookies on the NHS England website

We’ve put some small files called cookies on your device to make our site work.

We’d also like to use analytics cookies. These send information about how our site is used to a service called Google Analytics. We use this information to improve our site.

Let us know if this is OK. We’ll use a cookie to save your choice. You can  read more about our cookies before you choose.

Change my preferences I'm OK with analytics cookies

Maximising the benefits of research: Guidance for integrated care systems

England has a vibrant research and development ecosystem, with well-developed research infrastructure and research expertise within our health and care workforce. The value of research in transforming health and care is significant; additionally, staff satisfaction, recruitment and retention is higher among staff who are involved in research. The inception of integrated care systems (ICSs) provides the opportunity for systems to embed research within health and care for the benefit of our population. Supporting this opportunity, a clear research thread runs through ICS strategies and plans, from joint strategic needs assessments and joint health and wellbeing strategies , integrated care strategies , joint forwards plans , integrated care board (ICB) annual reports and the assessment by NHS England of the discharge of duties by ICBs.

The Health and Care Act 2022 (the 2022 Act) sets new legal duties on ICBs around the facilitation and promotion of research in matters relevant to the health service, and the use in the health service of evidence obtained from research. NHS England will assess ICBs for their discharge of these duties. The ICS design framework sets the expectation that in arranging provision of health services, ICBs will facilitate their partners in the health and care system to work together, combining expertise and resources to foster and deploy research and innovations. This guidance supports ICBs in fulfilling their research duties.

ICSs are encouraged to develop a research strategy that aligns to or could be included in their integrated care strategy. This strategy will enable the unification of research across ICS partners, and be consistently embedded to:

• identify and address local research priorities and needs, and work collaboratively to address national research priorities
• improve the quality of health and care and outcomes for all through the evidence generated by research
• increase the quality, quantity and breadth of research undertaken locally
• extend and expand research in settings such as primary care, community care, mental health services, public health and social care
• drive the use of research evidence for quality improvement and evidence-based practice
• influence the national research agenda to better meet local priorities and needs
• improve co-ordination and standardisation within and between localities for the set up and delivery of research
• harness the patient and economic benefits of commercial contract research
• co-ordinate and develop the research workforce across all settings.

1. Introduction

This guidance sets out what good research practice looks like. It supports integrated care systems (ICSs) to maximise the value of their duties around research for the benefit of their population’s health and care and, through co-ordination across ICSs, for national and international impact. It supports integrated care boards (ICBs), integrated care partnerships (ICPs) and their partners to develop a research strategy that aligns to or can be incorporated into their integrated care strategy, and helps them and their workforce to build on existing research initiatives and activities across health and social care to improve sector-wide performance and best practice

• explains the ICB legal duties and other requirements around research and the use of evidence from research, and that research is included in forward planning and reporting
• encourages system leaders to develop a footprint-wide research strategy that aligns to local and national research priorities, develops and supports their workforce, takes the opportunities offered by commercial research and includes plans to embed research in their system’s governance and leadership
• identifies best practice examples and other resources that ICBs may find useful as they develop their research strategies.

This guidance provides comprehensive information for use by:

• those with senior responsibility, including at board level, for research strategy development and/or operationalising research
• managers responsible for developing joint strategic needs assessments, integrated care strategies, joint health and wellbeing strategies, joint forward plans, other linked strategies, or reporting on ICB activities
• research managers
• research and development/innovation leads
• heads of services
• knowledge and library specialists.

It may also be useful to individuals involved in research, education, and partner organisations such as local authorities, social care services, the voluntary, community and social enterprise sector (VCSE) and other providers of healthcare services.

NHS England provides guidance on embedding research in the NHS and secure data environments, and the Office for Life Sciences (OLS ) champions research, innovation and the use of technology to transform health and care service. Other sources of guidance, support and information are signposted in this guidance to support ICSs in aligning to national visions, strategies and plans around research.

1.1 Definition of research

NHS England uses the UK Policy Framework for Health and Social Care Research definition of research:

“… the attempt to derive generalisable or transferable new knowledge to answer or refine relevant questions with scientifically sound methods. This excludes audits of practice and service evaluation. It includes activities that are carried out in preparation for or as a consequence of the interventional part of the research, such as screening potential participants for eligibility, obtaining participants’ consent and publishing results. It also includes non-interventional health and social care research (that is, projects that do not involve any change in standard treatment, care, or other services), projects that aim to generate hypotheses, methodological research and descriptive research”.

This broad definition encompasses the range of types of research:

• clinical trials and other clinical investigations into the safety and effectiveness of medicines, devices and health technologies
• public health research
• observational studies
• discovery science and experimental medicine
• translational research in which results from basic research are developed into results that directly benefit people
• applied research
• research to support policy-making and commissioning
• social care research and research in social care settings
• research into NHS services and care pathways.

1.2 Why research is important

The UK is a world leader for research and invention in healthcare, with around 25% of the world’s top 100 prescription medicines being discovered and developed in the UK ( The impact of collaboration: The value of UK medical research to EU science and health ). Research in the health and care system is important because it underpins all advances in health and care and is the basis for evidence-based practice. Engaging clinicians and healthcare organisations in research is associated with improvements in delivery of healthcare ( Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review) . To benefit service users and the public, the NHS and local government, and achieve return on investment, it is vital that research is disseminated, shared and translated into practice.

The National Institute for Health and Care Research (NIHR) is funded by the Department of Health and Social Care (DHSC) to transform research in the health and social care system, including through support for NHS research. Research led to the first proven treatments for Covid, for example the use of dexamethasone, estimated to have saved over a million lives worldwide . This success was in part due to how research is undertaken in the unique environment of the NHS, innovative trial designs, the support provided by the NIHR, frontline staff enabling research, and the awareness and readiness of the public to support research. We need to learn from these and other successes, and translate this across all health and care settings. ICSs will play a vital role in enabling research to be embedded in evolving patient pathways across their footprints.

Example: PRINCIPLE trial – finding treatments for Covid recovery at home

The Platform Randomised Trial of Treatment in the Community for Epidemic and Pandemic Illnesses (PRINCIPLE) was a UK-wide, clinical study to find Covid treatments for recovery at home without the need to attend hospital. The study was open to all with ongoing Covid symptoms, registration was easy, and the trial was run entirely remotely by delivering ‘participant packs’ to people’s homes. It was one of the first trials in the world to show that azithromycin and doxycycline did not benefit patients with Covid and to identify the effectiveness of a commonly used drug – inhaled budesonide –in reducing time to recovery.

The PRINCIPLE study team demonstrated the integral role that primary, secondary and ambulatory care staff can play in the delivery of studies. Local collaborators were trained in good clinical practice to allow them to assess and confirm the eligibility of potential participants, and were commended specifically for their use of patient data to contact people soon after they received a positive test result. It is this network of local staff contributing to research within their healthcare setting that has enabled over 10,000 people to be recruited onto this study so far – one of the largest at home Covid treatment studies worldwide.

This is an example of a study design that incorporates the vital contributions of healthcare providers across the system.

Policy-makers and commissioners need evidence to support their decision-making around the delivery and system-wide transformation of health and care services, including how health inequalities will be reduced.

There is also evidence that:

• staff involved in research have greater job satisfaction and staff turnover is lower in research active trusts ( Academic factors in medical recruitment: evidence to support improvements in medical recruitment and retention by improving the academic content in medical posts)
• research active hospitals have lower mortality rates, and not just among research participants ( Research activity and the association with mortality )
• 83% of people believe that health research is very important ( Survey of the general public: attitudes towards health research)
• healthcare performance improvements have been seen from the creation of academic research placements ( Experiences of hospital allied health professionals in collaborative student research projects: a qualitative study )
• clinical academic research, and in particular the practice changes resulting from it, is associated with improved patient and carer experiences ( A qualitative systematic review and thematic synthesis exploring the impacts of clinical academic activity by healthcare professionals outside medicine ).

Key to having research embedded in health and care is having staff who can understand, undertake, use and generate new research, and share actionable research finding as part of a pro-research culture. Education and training are therefore critical for research to be sustainably embedded within health and care, and for people to develop careers in research and support it in their clinical or care roles.

DHSC, NHS England, the devolved administrations, NIHR and other partners expect to publish a clinical research workforce strategy in 2023/24 to help the UK realise the national clinical research vision outlined in Saving and Improving Lives: The Future of UK Clinical Research Delivery and deliver the Life Sciences Vision to see research embedded in the NHS as part of health and care pathways.

Research will support ICSs to deliver on their four key aims:

Improving outcomes

The NHS 2023/34 priorities and operational planning guidance emphasises the importance of research in improving patient care, outcomes and experience.

Research evidence will inform commissioning decisions to improve experience and outcomes. Research activities should align with the local health priorities identified through local joint strategic needs assessments, and may be best designed and delivered by collaborating with partners. Research priorities may be best addressed by collaborating with partners nationally to design and deliver research.

Tackling inequalities

Research can give a better understanding of local populations and the wider determinants of health, and with this the steps to maintain health and narrow health inequalities.

Enhancing productivity

The development of ICSs creates the opportunity to consider research delivery within the ICS and across ICS boundaries, increasing flexibility of workforce or recruitment while reducing bureaucracy and improving research productivity and value for money.

Supporting social and economic development

An active research ecosystem working in a co-ordinated way and to national standards brings revenue and jobs to regions. The NIHR Clinical Research Network (CRN) supports service users, the public and health and care organisations across England to participate in high-quality research. The 2019 impact and value report detailed the significant income and cost savings that commercial research generates for NHS trusts. Between 2016/17 and 2018/19 the NHS received on average £9,000 per patient recruited to a commercial clinical trial and saved over £5,800 in drug costs for each of these patients. This equates to income of £355 million and cost savings of £26.8 million in 2018/19.

In 2021 150 members of the Association of Medical Research Charities funded £1.55 billion of medical research, including the salaries of 20,000 researchers. Every £1 million spent by charities on medical research in the UK contributes £1.83 million to the economy.

Example: Research that cut problematic prescribing and generated cost savings in general practice – a local health priority

Analysis of routine patient data identified the need for strategies targeting clinicians and patients to curb rising opioid prescribing. From this, the Campaign to Reduce Opioid Prescription (CROP) was launched in 2016, urging GPs across West Yorkshire to ‘think-twice’ before prescribing opioids. This promoted the NICE guidance on chronic pain , which recommends reducing the use of opioids because there is little or no evidence that they make any difference to people’s quality of life, pain or psychological distress, but they can cause harm, including possible addiction.

Over a year 15,000 fewer people were prescribed opioids (a 5.63% relative reduction), a net saving to the NHS of £700,000. The biggest reduction was in people aged over 75, who are at higher risk of opioid-related falls and death, and there was no compensatory rise in the prescribing of other painkillers or referrals to musculoskeletal services.

The CROP campaign, led by researchers at the University of Leeds, has subsequently been rolled out across all ICBs in Yorkshire and the Humber, and the North East and North Cumbria ICB, and the 1,045 practices to which it has been delivered are reporting results similar to the above.

Foy R, Leaman B, McCrorie C, Petty D, House A, Bennett M, et al (2016) Prescribed opioids in primary care: cross-sectional and longitudinal analyses of influence of patient and practice characteristics | BMJ Open 69(5).

Alderson SL, Faragher TM, Willis TA, Carder P, Johnson S, Foy R (2021) The effects of an evidence- and theory-informed feedback intervention on opioid prescribing for non-cancer pain in primary care: A controlled interrupted time series analysis. PLOS Med .

2. ICS, ICP and ICB responsibilities and requirements

ICBs have legal duties and other requirements that relate to research. These are additional to the duties and responsibilities of individual providers within ICS footprints. This section sets out what these duties mean in practical terms and gives examples of how to meet them.

2.1 Legal duties relating to research in the Health and Care Act 2022

Part 1 of the 2022 Act includes specific legal duties for ICBs and NHS England in respect of research. In the Explanatory Notes to the 2022 Act, government sets out how ICBs could discharge their research duty.

Duty to facilitate or otherwise promote research

The ICB duty builds on the previous clinical commissioning group (CCG) duty to promote research, by requiring each ICB, in the exercise of its functions, to facilitate or otherwise promote research on matters relevant to the health service. This duty is intended to include a range of activities to enable research. Section 3 of this guidance outlines ways in which ICBs can do this.

The NHS Constitution also makes clear that patients should be enabled to take part in research: “the NHS pledges … to inform you of research studies in which you may be eligible to participate”.

The Provider Selection Regime (PSR) will be a new set of rules for arranging healthcare services in England, introduced by regulations made under the 2022 Act. The research component should be referred to once the PSR is published.

Duty to facilitate or otherwise promote the use in the health service of evidence obtained from research

This duty similarly builds on the CCG requirement to promote the use of evidence. ICBs must, in the exercise of their functions, facilitate or otherwise promote the use in the health service of evidence obtained from research. For example, ICBs should facilitate or otherwise promote the use of evidence in care, clinical and commissioning decisions.

Duty for ICSs to include research in their joint forward plans and annual reports

Joint forward plans are five-year plans developed by ICBs and their partner NHS trusts and foundation trusts. Systems are encouraged to use the joint forward plan as a shared delivery plan for the integrated care strategy and joint health and wellbeing strategy, aligned to the NHS’s universal commitments. The plan must explain how the ICB will discharge its duties around research, and the ICB must report on the discharge of its research duties in its annual report. These inclusions will raise the profile of research at board level and help embed research as a business-as-usual activity.

The joint forward plan and NHS Oversight Framework guidance set the minimum requirements for what needs to be included in plans and reports.

NHS England duty to include how each ICB is carrying out its duties relating to research in its annual performance assessment of each ICB

NHS England has a new legal duty to annually assess the performance of each ICB and publish a summary of its findings. For 2022/23 NHS England will complete a narrative assessment, identifying areas of good and/or outstanding performance, areas for improvement and any areas that are particularly challenged, drawing on national expertise as required and having regard to relevant guidance. This assessment will include a section considering how effectively the ICB has discharged its duties to facilitate or otherwise promote research and the use of evidence obtained from research.

This, alongside the implementation of the NHS Long Term Plan commitment to develop research metrics for NHS providers, will increase transparency across the system and enable more targeted support for research. Research metrics from NHS England, the Care Quality Commission (CQC) and NIHR will enable the monitoring of progress over time, and are under development with sector colleagues, including providers.

2.2 Legal requirement to work with people and communities

Working with people and communities is a requirement of ICBs, and statutory guidance is available to support them and their partner providers meet this legal duty. A co-ordinated approach across healthcare delivery and research will make it more likely that research reflects what matters to people and communities.

This will also help ICBs to fulfil their legal duty in the 2022 Act to reduce health inequalities in access to health services and the outcomes achieved. Section 3.9 includes links to resources to help guide engagement with underserved communities around research.

The Public Sector Equality Duty also applies and requires equality of opportunities between persons who share a relevant protected characteristic and persons who do not.

2.3 Research governance

While research can address local priorities, it typically operates across ICS boundaries and at national and international levels. Health and social care research is governed by a range of laws, policies, and international, national and professional standards.

The Health Research Authority (HRA ) is responsible for ensuring such regulation is co-ordinated and standardised across the UK to make it easier to do research that people can trust. The HRA is an executive non-departmental public body created by the Care Act 2014 to protect and promote the interests of patients and the public in health and social care research, including by co-ordinating and standardising the practice of research regulation. Local authorities and the NHS are obliged to have regard to its guidance on the management and conduct of research.

Before a research project can start in the NHS in England it must receive approval from the HRA. This includes research taking place in NHS trusts, NHS foundation trusts, ICBs or primary care providers of NHS commissioned services in England, and all research under an NHS duty of care, including that undertaken by NHS staff working in social care or other non-NHS environments.

The HRA schemes indemnify NHS organisations accepting these assurances against any claim covered by the NHS Litigation Authority arising as a result of incorrect assurances. If an NHS organisation duplicates the HRA assessments, it will be liable for any consequences of the decisions it bases on its own checks.

ICBs and partner organisations should have processes for the set up and delivery of research that comply with national laws and systems, and does not duplicate them. Such national systems include confirmation of capacity, National Contract Value Review (NCVR), management of Excess Treatment Costs (ETCs) and contracting arrangements (see section 2.4).

The UK Policy Framework for Health and Social Care sets out the roles and responsibilities of individuals and organisations involved in research.

2.4 Contractual requirements around research

NHS England mandates commissioner use of the NHS Standard Contract for all contracts for healthcare services other than primary care. The contract is updated annually. References to research in the current NHS Standard Contract and service conditions fall into three main areas.

Recruitment of service users and staff into approved research studies

The NHS Standard Contract obliges every provider of NHS-funded services to assist the recruitment of suitable subjects (whether patients or staff) into approved research studies. This requirement aligns to those in the 2022 Act that require ICBs to facilitate or otherwise promote research (see section 2.1). Section 3 considers how this requirement can best be met. Research involving people or their data requires ethical and potentially other approvals (see section 2.3).

National Directive on Commercial Contract Research Studies

Adherence to the National Directive is mandated as part of the NHS Standard Contract. The directive states that providers must:

• Use the unmodified model agreements for sponsor-to-site contracting; HRA and Health and Care Research Wales (HCRW) approval of studies will be dependent on use of these templates.
• Use the standard costing methodology to set prices for commercial contract research undertaken by NHS providers; this is currently in the NIHR interactive costing tool (NIHR iCT).
• Introduce the National Contract Value Review (NCVR) process in line with national rollout. NCVR is a standardised national approach to costing commercial contract research within the NHS. It currently covers acute, specialist and mental health trusts, but the intention is to roll it out to all NHS providers. The creation of ICSs is the ideal opportunity to explore how commercial study set up can be supported across these footprints, reducing the resource needed and time taken.

Comply with HRA/NIHR research reporting guidance

The provider must comply with HRA/NIHR research reporting guidance, as applicable.

2.5 Excess treatment costs

Patients in a research study may receive healthcare that differs from what is standard in the NHS, requires more clinician time or is delivered in a different location. The associated NHS treatment costs may exceed or be less than those of standard treatment. If greater, the difference is referred to as the NHS Excess Treatment Costs (ETCs).

In the case of commercial contract research, the commercial funder will pay the full cost of the study. In the case of non-commercial research, the commissioner of the service in which the study operates is responsible for funding the ETCs.

ICBs as commissioners of services are responsible for ETCs in services that they commission. Guidance for the management of ETCs is available.

DHSC and NIHR are piloting interim arrangements to support non-NHS ETCs for research in public health and social care (non-NHS intervention costs). Please refer to the further detail on the NIHR website .

2.6 Care Quality Commission

The CQC is currently developing its approach for ICS-level assessments, and its new assessment framework will be introduced towards the end of 2023 .

CQC inspection of NHS providers continue, with research assessed as part of the review of the trust-level Well-led framework. Providers are asked:

• Are divisional staff aware of research undertaken in and through the trust, how it contributes to improvement and the service level needed across departments to support it?
• How do senior leaders support internal investigators initiating and managing clinical studies?
• Does the vision and strategy incorporate plans for supporting clinical research activity as a key contributor to best patient care?
• Does the trust have clear internal reporting systems for its research range, volume, activity, safety and performance?
• How are service users and carers given the opportunity to participate in or become actively involved in clinical research studies in the trust?

3. Developing a research strategy

3.1 why develop a research strategy.

Like the health and care system, the research environment is complex. Developing a research strategy will help bring together the legal and other duties around research in a coherent way, and help the ICS understand its local research capability, workforce, activity and needs, set ambitions around research and maximise the benefits associated with commercial research. It will help demonstrate the benefit of research locally, nationally and internationally, and guide the production of clear plans.

Example: Value of research partnerships and integration with ICSs

Bristol Health Partners (BHP) Academic Health Science Centre (AHSC) has a fully integrated relationship as the new Research and Innovation Steering Group for the Bristol, North Somerset and South Gloucestershire (BNSSG) ICS, and reports directly to ICB chief executives.

The group provides the strategic direction and oversight for all research undertaken and delivered across the system. Membership includes directors of research, clinical strategy, public health, social care, senior innovation and education leaders from its core funding partners. It also includes public contributors and senior representatives from primary care, NIHR Applied Research Collaboration West, NIHR CRN West of England, West of England Academic Health Science Network (WEAHSN), Healthier Together ICS, university research institutes and People in Health West of England.

The group has reviewed ICS programmes, identified current and potential research and innovation connections, and begun to establish new connections. It has also supported work with the ICS Ageing Well programme and secured funding for innovative pilots to improve dementia care and increase physical activity for older adults.

Since 2016 BHP has directly contributed an estimated additional £1.1 million to support ICS priorities through Health Integration Team projects and other activities, and has attracted more than £33 million of external research, service redesign and infrastructure into the region.

3.2 General considerations

In developing its research strategy, the ICS may find it helpful to consider these overarching questions alongside the suggested focused content covered in the sections below:

• What do you hope to achieve within a given timeframe?
• Are all the right organisations involved in developing the research strategy?
• How will the health and care workforce be enabled to deliver the research strategy?
• How can research be embedded in existing health and care delivery and pathways?
• What mechanisms are in place to translate actionable research findings into practice and decision-making?
• What inequalities exist in different areas, communities or groups? How will you ensure planning and delivery of research aligns to CORE20plus5 priorities?
• Are you considering equality, diversity and inclusivity and the Public Sector Equality Duty in facilitating and promoting research opportunities for service users and for health and care staff?
• Is the ICS considering the opportunities of developing their commercial research portfolio?
• Is research informing or being informed by population health management?
• How will you plan and deliver research in a sustainable manner, aligning it to the Greener NHS agenda and the ICB’s duties in relation to climate change ?

Buy-in from NHS staff, patients and the public will be vital if ICBs are to discharge their research duties and deliver on their research plans. An important consideration is how to develop sustainable, routine and accessible information flows to ensure the ICB, partners, staff, patients and public can access up-to-date and appropriate information around local research activity, regional, national and international research opportunities and findings, and contact information.

3.3 Leadership and governance across the ICS

Executive leadership.

The Explanatory Notes to the 2022 Act suggest that ICBs have board-level discussions on research activity, the use of the evidence from research, the research workforce and research culture within the ICS. ICSs should refer to the NHS Leadership Competency Framework for board-level leaders at organisation and ICS level for the competencies relating to the research duties of ICSs, once published.

All ICBs are encouraged to have an executive lead responsible for fulfilling the research duties conferred by the 2022 Act. They should help give the ICB a clear understanding of research across the area, regularly reporting on progress towards agreed aims. An executive lead can take responsibility for ensuring clear research ambitions and a research strategy are developed; oversight of organisational research portfolios, diversity in research, alignment to national priorities; promotion of research skills and the need for research skills training; and succession planning.

Senior leaders could engage, consult and be supported by representatives of each registered health and social care professional group when developing strategic plans, and for oversight of training, succession planning, and equality and inclusivity. They could use the capacity and capability of the research and development leads within provider organisations, although established lead roles across social care settings are rare so extra effort may be needed to garner social care research insight.

Research steering group, board or forum

Some CCGs had research steering groups and some of these have expanded with the widening remit of ICBs. ICSs that do not have a such a group should consider adopting a model similar to one in other ICSs where research is effectively embedded in ICS governance structures.

A dedicated steering research group, board and/or forum can:

• provide dedicated time to plan, oversee and report on research
• bring a range of representatives from research infrastructure organisations, patients and the public together with representation from across the ICS, to develop a common aim and objective
• ensure board-level sight of research
• take a cross-ICS approach to research, increasing participation and diversity in research, and reducing bureaucracy.

Example: A dedicated research and innovation subgroup

East and North Hertfordshire Health Care Partnership established a formal research and innovation subgroup to support its objectives to transform services, reduce health inequalities and improve patient health and wellbeing. This subgroup is dedicated to determining and supporting local research priorities and developing an innovation agenda. With effective patient and public involvement, it is working to ensure the local population has access to more research opportunities.

Bringing together the NIHR, academia, industry and local health and care services, the subgroup develops collaborative work plans that support the design, implementation and evaluation of local transformation needs, sharing resources, staff, expertise and facilities. Its work exemplifies a sustainable approach to partnership working and supports Hertfordshire and West Essex ICS’s developing strategy.

HWE ICS Partnership Board 14 September 2021

3.4 Understanding your research activity and working with local and national research infrastructure

Research in NHS and non-NHS settings across an ICS footprint will be supported by different organisations. In some areas networks or collaboratives already exist to bring these organisations together, but in others the links are not as well formed. ICBs would benefit from having a clear map of the research infrastructure and pre-existing local or national investment into research in their area.

It may be valuable to consider:

• Who are the research leaders in your local health and care system, NIHR, higher education institutions, VCSE sector and businesses?
• Are there any pre-existing local or regional research, researcher or research engagement networks?
• What are the opportunities to inform, participate in, collaborate with or lead national and international research efforts in addition to local opportunities?

A list of organisations involved in research including NIHR-funded infrastructure and programmes is included in Annex 1 .

Much of the research undertaken in NHS and other health and care settings is funded though national calls and grants provided by funders such as NIHR, research charities , UK Research and Innovation (UKRI) , including the Medical Research Council (MRC ) and Economic and Social Research Council (ESRC) , and is aligned to national priorities. Other research may include national or international commercial or non-commercial clinical trials funders.

Partners within ICS systems can use NIHR research portfolio data to monitor and plan research activity; however, not all research is included within the NIHR’s portfolio, so this will not give a full picture of the research within the footprint. Mechanisms to map and monitor research more widely could be incorporated in ICB research strategies.

Some local needs may best be addressed through public health or social care research rather than research in primary, secondary or tertiary healthcare settings. Public health and social care research are described in Annex 2 .

Example: Mapping health and care research activity, expertise, interests and infrastructure

The Nottingham and Nottinghamshire Integrated Care System Research Partners Group meets bi-monthly and is chaired by the ICB Head of Research and Evidence. It brings together senior managers from the NHS providers, ICB, two local authorities, two universities and the NIHR CRN East Midlands, providing a forum for ICS-wide research discussions and the development of a system-wide collaborative approach to health and care research across the ICS. Among its aims, the group seeks to increase participation in research at both the organisational and population level, enable equity of access to research opportunities and generate impact on health and care pathways.

The group have mapped health and care research activity, expertise, interests and infrastructure in the constituent organisations. With this the ICS can see the research capabilities, strengths, expertise, and areas of synergy and opportunities for future collaboration that align to its needs and priorities, and also gaps for future development, recognising that organisations are at different stages of research development.

3.5 Understanding local needs

Universal NHS priorities will be reflected in local research needs, and each ICS footprint is likely to have its own specific local research needs. Joint strategic needs assessments (JSNAs) are undertaken jointly by local authorities and ICBs through health and wellbeing boards (HWBs) to identify current health and social care needs of local communities, where more information is needed to do so or to understand how best to address the need. People and communities should be directly involved in identifying local need, including by working with local charities, specific communities or groups who face inequalities in access to, experience of or outcomes from healthcare, eg to target health research at those areas and populations with greatest need.

ICPs are required to develop an integrated care strategy informed by JSNAs and the joint health and wellbeing strategy (JHWS). The integrated care strategy sets out how the assessed needs can be met through the exercise of the functions of the ICB, partner local authorities or NHS England, and is informed by research and practice-based evidence, as stated in the health and wellbeing guidance. In considering where such evidence is lacking, HWBs should identify in JSNAs those research needs that ICBs, local authorities and NHS England could meet through the exercise of their research functions.

Systems are encouraged to use their joint forward plan to develop a shared delivery plan for the Integrated Care Strategy and the JHWS that is supported by the whole system, including local authorities and VCSE partners. ICBs and trusts must also use their Joint Forward Plan to describe how the ICB will discharge its duty in respect of research.

The Explanatory Notes to the 2022 Act suggest how ICBs can discharge their duties around research. These include the articulating local research needs when assessing local needs and how they will be addressed when preparing strategies and plans, and encouraging partner organisations to play an active and collaborative role in pursuing these.

3.6 Supporting delivery of research

Once an ICS has a clear picture of its local research infrastructure it can consider how best to target and support research and the research workforce across its footprint and how research findings will be used. For this, the ICB should ensure that its approaches reflect national approaches to costing, contracting, approvals and information governance, and that they are also informed by learning from effective practices across equivalent ICBs.

As healthcare shifts into communities, ICSs should support the parallel shift in research by embedding research in health and care. Increasing access to research opportunities will give service users earlier access to new treatments, and faster research set up and delivery may provide the evidence needed to support improvements to local care sooner. Inclusive recruitment practices will be needed to ensure that all groups in society have the opportunity to help shape and take part in research, and benefit from research findings.

In developing its research strategy, an ICS has opportunities to reduce bureaucracy, and make research more efficient and effective across its own and with other ICS footprints, and across NHS and non-NHS boundaries, while meeting national regulatory guidance. ICBs will be expected to work with the HRA to co-develop, build on and implement strategies for further co-ordination and standardisation of study set-up and delivery processes. Any regional systems and processes that ICBs do establish must support consistent national practice in relation to the management and regulation of research, and should not duplicate them. The HRA will work with ICBs to address barriers to efficient and rapid study set-up, including model agreements, information governance and R&D office functions.

Other potential areas for streamlining and cross-organisational working include:

• cross-ICS research proposals to identify research needs
• research delivery – identifying how ICS-wide approaches could accelerate patient recruitment and deployment of research delivery staff
• shared data architecture, including the NHS Secure Data Environment for Research Network and its subnational secure data environments (SDEs). Subnational SDEs cover multiple ICSs to achieve access to multimodal data at a scale of approximately 5 million citizens, and over time will achieve technical and governance interoperability
• a greater focus on translation and implementation of research findings into health and care practice, supporting faster improvements
• sharing access to and funding for knowledge and library services
• shared processes and repositories for research assets.

The Explanatory Notes to the 2022 Act suggest that one way an ICB could discharge its research duty would be to have a dedicated research office or team supporting research.

3.7 Enabling cross-provider research

Health and care priorities can often only be addressed with complex, multiorganisational approaches and as such the research to inform these needs to span organisational boundaries. Organisational policies should promote cross-organisational research and dissemination of research findings, including through participation in collaborative research to address national priorities, joint staff posts, honorary contracts, and administratively easier movement of researchers between health and care organisations and other sector partners, including higher education, industry, charities and local authorities.

The HRA and ICS partners are developing national guidance to support cross-provider research.

The NIHR CRN can offer ICSs opportunities to participate in national and international research studies, including those the NIHR, industry and others commission.

3.8 Commercial research

Commercial contract research is research funded solely by industry, where NHS providers are contracted to carry out the research. Most of these research studies in the NHS are interventional clinical trials, such as the NHS-Galleri trial and Astra Zeneca’s COVID-19 vaccine development . Commercial research can give patients access to a wider range of research opportunities, earlier access to novel therapies and treatments, provide drugs free of charge to patients in trials, accelerate the development of new treatments and devices, generate income for providers, and fund NHS staff. It is vitally important for the benefit of patients, the NHS and the UK economy that we create an environment in the NHS that makes it easy and efficient for the NHS to undertake commercial research. This is particularly important when it comes to international commercial research, where companies can place their studies in a number of different countries and consideration of anticipated set up and recruitment times informs where they place trials.

Data gathered during some commercial research is specific to the study and is the property of the company, as is any Intellectual Property (IP) generated. In other cases, where the NHS contributes to the foreground IP – such as through the use of NHS data for research or where NHS expertise provides important contributions to a commercial product – it is important that the NHS shares in the value of IP generated as a consequence of its contributions.

The establishment of ICSs is an ideal opportunity for their creation of ambitions to enable, grow and benefit from commercial research. ICSs should explore how efficient commercial study set up and delivery could be streamlined across sites within their footprint, and should set ambitions around commercial research.

3.9 Involving patients, service users, carers and the public in research

In developing a research strategy ICSs should set out their approach to diverse public and patient involvement (PPI) in relation to research.

Areas where working with people and communities could add value in the context of research include:

• identification of local research needs, including through JSNAs and JHWSs
• designing research proposals in partnership with local or national experts
• raising awareness of research opportunities and recruitment of participants
• developing research outcome reports and identification of how and when participants will be able to access these
• consideration of how members of the public can access the outputs from publicly-funded research
• how volunteers should be involved and what they should be paid.

The UK Standards for Public Involvement sets out the core components of good public involvement. A guide outlining good practice in engaging underserved communities around research is available from NHS England. Resources about good practice around PPI in designing and delivering research, including around incentivisation , are also available from the HRA and NIHR .

It will be useful to link into established community involvement approaches. NIHR infrastructure organisations may have established networks of expert PPI representatives, and ICSs have extensive VCSE Alliances. A co-ordinated community engagement approach across health and care delivery and research will reduce the risk of overburdening communities with organisations wanting to work with them, and will support the identification of under-served communities.

3.10 Ensuring anyone can participate in research

Making research more visible within communities and increasing the public’s understanding of research can ensure greater diversity in research participation. Research findings will then be more generalisable to a broader range of groups or communities, or can be targeted and specific to relevant communities.

ICSs should seek mechanisms to ensure that opportunities to take part in research are available to all. They should consider encouraging patients and members of the public to register on NIHR Be Part of Research (a national registry where people can express their interest in being contacted about research that is relevant to them), widely disseminate research opportunities and make provision for inclusive access for communities to take part in research. Decentralised or virtual trials are remote access trials recruited to and delivered using electronic tools, making it easier for people to participate in some studies without needing to visit a recruiting hospital or attend appointments in person. ICBs should consider ways in which research delivery can increase access to research opportunities for people within their area. ICBs should also advise the public how they can access research outputs.

NIHR and UK Research and Innovation provide resources that help organisations address issues of equality, diversity and inclusion in research settings.

Example: RELIEVE-IBS decentralised trial

In 2020, Newcastle researchers launched RELIEVE-IBS, one of the first interventional decentralised clinical studies in the UK to trial Enterosgel, a new treatment for irritable bowel syndrome with diarrhoea (IBS-D). Decentralised trials are remote access trials that use electronic tools for trial recruitment and delivery, without the patient needing to visit a recruiting hospital site, which could be miles from their homes – a convenient option for patients with IBS-D. By running the trial remotely, researchers could reach beyond the small proportion of those with this condition who attend specialist clinics, as well as save resource for the sponsor.

Not only did this trial embrace technological developments to deliver research, but it empowered more patients to become involved regardless of where they lived. With in-depth patient input, the research team were able to shape the recruitment approach to be highly accessible to participants and were offered feedback on how to refine the trial design by the sponsors. The resulting patient-centric design ensured a good recruitment response when the trial opened.

NIHR (2020) Virtual trial recruits 67% faster led by NIHR Patient Recruitment Centre in Newcastle in collaboration with Enteromed

NIHR (2021) Pushing virtual boundaries to improve patient engagement and accessibility

NIHR (2022) RELIEVE IBS-D trial case study

3.11 Health data in research

Health data generated through care of service users in the NHS can fuel a revolution in the research and development of new diagnostics and treatments, maximising the potential to improve service user outcomes and experiences, support diversity in research, and minimise health inequalities through research. To do this, researchers need access to high quality and timely data to generate insights. The public expect data to be used legally and efficiently to conduct and support research.

National commitments around data for research can be found in Data saves lives: reshaping health and social care with data . This strategy shows how data will be used to bring benefits to all parts of health and social care. To achieve this vision, the NHS will be making a strategic move away from a system of data dissemination to one of data access when making NHS health and social care data available for research and analysis. This will be facilitated by the implementation of secure data environments (SDEs).

SDEs are data storage and access platforms with features that enable organisations to have greater control and oversight over their data. SDEs allow approved users to view and analyse data without it having to leave the environment. The SDE policy guidelines provide a clear signal to the sector that SDEs will become the default way of accessing NHS data for research.

This change is supported by major investments in digital infrastructure through the Data for Research & Development Programme, which is funding the development of national and subnational SDEs. The subnational SDEs will cover the entirety of England and individual platforms will cover several ICS.

ICBs should seek ways to promote and enable the use of these rich data sources for research and include them in their research strategy.

3.12 Using evidence for planning, commissioning and improving health and care

Evidence-based commissioning has advantages for the commissioner, workforce and service users, as it can:

• lead to innovation in service design and delivery
• enhance the quality of health and care provision
• reduce clinical variation between locations and providers
• improve equity of access to services
• improve patient and population outcomes.

As part of the commissioning process, commissioners are expected to use evidence-based clinical policies, as per the Roadmap for integrating specialised services within integrated care systems . Knowledge and library services can help source and interpret evidence.

The Provider Selection Regime will reflect the research duties of the 2022 Act and should be referred to when commissioning provider services, once it has been published.

NHS knowledge and library services provide access to evidence and support for knowledge management; they train people in searching for, handling and publishing information. The Knowledge for Healthcare strategy encourages and equips NHS knowledge and library services to support NHS organisations with the translation of knowledge for the spread and adoption of research and innovation. To fulfil their obligations under the 2022 Act, ICBs could commit to active knowledge translation.

Evidence for commissioning information is available from a number of sources:

• NHS Library and Knowledge Hub
• Health Libraries and Information Services Directory
• NICE guidance
• NIHR evidence
• NHS evidence works toolkit
• Academy of Medical Royal Colleges: Evidence-based Intervention
• A million decisions

The infographic for the role of research and evidence in commissioning also provides sources for evidence-based commissioning.

Example: Evidence mobilisation, knowledge sharing and improving outcomes

The STEMClub (Sustaining Transformation by Evidence Mobilisation) is a network in the North East and North Cumbria that brings together local policy and decision-makers with NHS knowledge and library specialists to facilitate evidence-based decision-making. The input of knowledge specialists ensures timely access to published research and provides knowledge management expertise to shape how soft intelligence is translated into knowledge assets.

As members within the STEMClub network, knowledge and library specialists are providing ongoing detailed evidence reviews and information management expertise to facilitate system-wide working , eg:

• North East North Cumbria Frailty Framework
• North East and North Cumbria Maternity Clinical Network
• a review of optimal patient transfer times in the North East and North Cumbria
• regular evidence summaries for the ICS Mental Health Evidence and Evaluation subgroup.

3.13 The health and care workforce and research

Staff involved in research have greater job satisfaction and research active trusts have lower staff turnover [3] . Clinical academic roles [7] , having research colleagues within services [8] and taking students on research placements [6] are felt to foster an increase in knowledge and skills across the wider staff workforce. The General Medical Council (GMC) and the Royal College of Physicians (RCP) and NIHR have issued position statements and recommendations around research, with additional signatories including UKRI, UKRD, the Academy of Medical Royal Colleges and the Royal College of Surgeons of England. Learning resources, including programmes for ongoing professional development of the research delivery workforce, are available through NIHR Learn.

In developing a research strategy ICSs could ensure that, as part of their people function and approach to workforce planning :

• Staff roles in leading, delivering or facilitating research and in supervising those developing research skills are recognised, supported and enabled across all staff groups and health and care settings as part of a positive research culture.
• The value of evidence is recognised, and education and training around research are facilitated. Opportunities to develop research careers or in overseeing the development of other researchers are enabled; this may include having protected time, inclusion in job plans and joint appointments across health and care providers and academic institutions.
• Ensuring that there is capacity and systems that support research through services like imaging, pathology and pharmacy, as well as finance and human resources.
• Individual organisations do not always have the necessary skills or services to support effective research and its impact, such as IP management, methodological expertise, regulatory compliance, statistical analysis, knowledge mobilisation expertise, genomics expertise, health informatics and data analytics. Mechanisms are needed to ensure that these can readily and rapidly be accessed across other health and care organisations, including from local authorities and other non-NHS care providers.

A UK Clinical Research Workforce Strategy is under development. ICSs should update their approaches to their research workforce once DHSC publishes this in 2023/24.

Example: Investing in the research workforce – developing capacity for chief investigators

Across the West Midlands NIHR CRN, an investment of approximately £750,000 to develop capacity for chief investigators returned additional research grant income of over £18 million in three years. This was achieved primarily by increasing the programme activity for consultants in areas where chief investigators were underrepresented.

The funding was provided through a competitive process and co-supported by the local NIHR CRN, with several local trusts jointly funded these scholars.

Kirk J, Willcocks J, Boyle P, Brocklehurst P, Morris K, Kearney R, et al (2022) Developing chief investigators within the NHS: the West Midlands clinical trials scholars programme. Clin Med 22(2): 149–52.

Kirk J, Reynolds F, Adey E, Boazman M, Brookes M, Brocklehurst P (2022) Developing paediatric chief investigators within the NHS: the Clinical Trials Scholars programme . Arch Dis Child Educ Pract Published online first: 22 February 2022. doi: 10.1136/archdischild-2021-322186

4. References

• Varnai P, Rentel M, Dave A, De Scalzi M, Timmerman W, Rosemberg-Mantes C, Simmonds P, Technopolis Group (2017) The impact of collaboration: The value of UK medical research to EU science and health .
• Boaz A, Hanney S, Jones T, Soper B (2015) Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review. BMJ Open 5: e009415. doi:10.1136/ bmjopen-2015-009415 .
• Rees MR, Bracewell M (2019) Academic factors in medical recruitment: evidence to support improvements in medical recruitment and retention by improving the academic content in medical posts. Postgrad Med J 95(1124): 323-327. doi:10.1136/postgradmedj-2019-136501 .
• Ozdemir BA, Karthikesalingham A, Singha S, Poloniecki JD, Hinchliffe RJ, Thompson MM, et al (2015) Research activity and the association with mortality. PLoS ONE 10(2): doi.org/10.1371/journal.pone.0118253 .
• Hunn A (2017) Survey of the general public: attitudes towards health research . Health Research Authority.
• Angus RL, Hattingh HL, Weir KA (2022) Experiences of hospital allied health professionals in collaborative student research projects: a qualitative study. BMC Health Services Research 22(1). Available at: https://doi.org/10.1186/s12913-022-08119-7 .
• Newington L, Wells M, Adonis A, Bolten L, Bolton Saghdaoui L, Coffey M, et al (2021) A qualitative systematic review and thematic synthesis exploring the impacts of clinical academic activity by healthcare professionals outside medicine. BMC Health Serv Res 21(1). Available at: https://doi.org/10.1186/s12913-021-06354-y .
• Wenke RJ, Hickman I, Hulcombe J, Phillips R, Mickan S (2017) Allied health research positions: A qualitative evaluation of their impact. Health Res Policy Syst 15(6). Available at: https://doi.org/10.1186/s12961-016-0166-4

Annex 1 – Organisations that may be involved in regional or local research

• Clinical Research Networks (CRNs) , which will be retendered and renamed regional research delivery networks (RRDNs) from April 2024
• Applied Research Collaborations (ARCs)
• Biomedical Research Centres (BRCs)
• Experimental Cancer Medicine Centres (ECMCs) , jointly funded with Cancer Research UK
• Research Design Services (RDSs) and Clinical Trials Units (CTUs) which will be replaced by the NIHR Research Support Service from 1 October 2023
• Patient Recruitment Centres (PRCs)
• MedTech and In vitro diagnostic Co-operatives (MICs) , which will be replaced with HealthTech research centres from April 2024
• School of Public Health Research, School of Primary Care Research and School of Social Care Research
• Health Determinants Research Collaborations (HDRCs)
• Clinical Research Facilities (CRFs)
• Patient Safety Research Collaborations (PSRCs)
• Translational Research Collaborations (TRCs)
• Academic Health Science Centres (AHSCs)
• university teaching hospitals and all trusts that deliver research activity
• primary care organisations, including GP practices, that deliver research activity
• higher education institutions (HEIs)
• local authorities
• social care partners
• Local Government Association
• local and national charities that fund, collaborate in or support participation in research
• research and development offices in providers or CSUs, including primary care providers and ambulance, community and mental health trusts, and those in the VCSE sector
• UKRD members
• NHS subnational secure data environments for research
• NHS R&D Forum
• NHS Genomic Medicines Service Research Collaborative
• NHS Knowledge and Library Services
• Academic Health Science Networks (AHSNs) are often well linked with research organisations and infrastructure as part of their roles in development, adoption and spread of innovation.

Annex 2 – Public health and social care research

Public health research investigates issues that impact at a population rather than an individual level. This can be done within the NHS with system-level studies, such as secondary prevention of cardiovascular disease and examining the impact on health inequalities of changes to the NHS resource allocation formula, and outside the NHS for the wider determinants of health such as air quality, transport systems and housing. There is a substantial body of public health evidence for the clinical and cost effectiveness of prevention, health protection, health service redesign and addressing health inequalities.

Social care research aims to improve the lives of children and adults who need to draw on personal or practical care and support, and family members or other unpaid carers. It can include research around the introduction, use and impact of technologies, and changing social care interventions, policies and practice. Social care research also examines issues pertaining to the safeguarding of adults and children and workforce, commissioning of services, and questions about organisational and professional practice, including decision-making, training and the quality of care.

Publication reference: PR1662

• Research article
• Open access
• Published: 06 June 2020

Measuring the benefits of the integration of health and social care: qualitative interviews with professional stakeholders and patient representatives

• Helen Crocker   ORCID: orcid.org/0000-0002-9786-0749 1 ,
• Laura Kelly 1 , 2 ,
• Jenny Harlock 3 ,
• Ray Fitzpatrick 1 &
• Michele Peters 1  

BMC Health Services Research volume  20 , Article number:  515 ( 2020 ) Cite this article

12k Accesses

11 Citations

5 Altmetric

Metrics details

Integrated care has the potential to ease the increasing pressures faced by health and social care systems, however, challenges around measuring the benefits for providers, patients, and service users remain. This paper explores stakeholders’ views on the benefits of integrated care and approaches to measuring the integration of health and social care.

Twenty-five semi-structured qualitative interviews were conducted with professional stakeholders ( n  = 19) and patient representatives ( n  = 6). Interviews focused on the benefits of integrated care and how it should be evaluated. Data was analysed using framework analysis.

Three overarching themes emerged from the data: (1) integrated care and its benefits, with stakeholders defining it primarily from the patient’s perspective; (2) potential measures for assessing the benefits of integration in terms of system effects, patient experiences, and patient outcomes; and (3) broader considerations around the assessment of integrated care, including the use of qualitative methods.

Conclusions

There was consensus among stakeholders that patient experiences and outcomes are the best measures of integration, and that the main measures currently used to assess integration do not directly assess patient benefits. Validated health status measures are readily available, however, a substantial shift in practices is required before their use becomes commonplace.

Peer Review reports

Transforming the delivery of health and social care is needed to address the growing pressures due to funding constraints and increasing complexity of needs for services [ 1 , 2 ]. In an attempt to address these issues, various programmes to integrate health services have been developed internationally [ 3 , 4 ]. The World Health Organisation (WHO) recognises the importance of integrated care and recently developed a ‘ Framework on integrated people-centred health services ’ calling for fundamental change in the funding, management, and delivery of health services [ 5 ]. In England, many initiatives [ 6 , 7 , 8 , 9 ] have attempted to integrate the health and social care systems, with varying success, and most recently Integrated Care Systems (ICSs) are intended to cover the whole of England, as set out in the UK National Health Service (NHS) Long-Term Plan [ 10 ].

There are many different definitions and concepts of integrated care [ 11 , 12 ]. A commonly referred to definition was developed by NHS England and National Voices, and takes a person-centred approach, “ I can plan my care with people who work together to understand me and my carer(s), allow me control, and bring together services to achieve the outcomes important to me ” [ 13 ]. While the integration of services can take many forms (e.g. horizontal or vertical), the focus of this paper is the integration of health and social care services. As an example, the Better Care Fund initiative pooled budgets to encourage local health and social care services to work more closely together to deliver ‘better, more joined-up services to older and disabled people, to keep them out of hospital and to avoid long hospital stays’ [ 14 ]. Initiatives funded through this scheme included multi-disciplinary teams, seven-day working, and single points of access.

Integrated care is difficult to achieve, especially in England where health and social care systems have a long standing institutional separation, with distinct funding and accountability arrangements. As such, it is important to be able to effectively evaluate the success of integrated care initiatives, and to achieve this, appropriate measures need to be available. However, questions have been raised about the suitability of existing measures, in particular, their breadth. Where instruments are available, there is a general lack of evidence to support their quality [ 15 ]. Furthermore, additional guidance around which measures and measurement instruments should be used is needed to aid comparisons of the success of strategies [ 16 ].

As integrated care arrangements continue to develop, there is a need to better understand their impact for patients, service users, and service providers. This study aimed to investigate how a diverse range of professional stakeholders and patient representatives viewed current approaches to assessing outcomes of the integration of health and social care. The findings are likely to be of international interest as countries attempt to evaluate the effectiveness of diverse integrated care arrangements and their impacts.

Semi-structured interviews were conducted with stakeholders to explore the value of integrated care, identify aspects for evaluation to determine the success of integrated care, and identify appropriate measures for these aspects. Commonly, integrated care is pursued for patients and service users, without adequately acknowledging the potential contributions that they can make [ 17 ]. In an attempt to overcome this, patient representatives were included as active participants in this research, alongside professional stakeholders.

Recruitment

Professional stakeholders were recruited through publically available information of national and regional NHS and social care representative bodies and stakeholder organisations, existing networks, and referral (i.e. snowballing) from those contacted or recruited using the above methods. Potential participants were selected based on their interest in and knowledge of integrated care services, and of policy/strategy and implementation in relation to integrated care. A purposive sampling strategy was adopted to recruit diverse respondents across the health and care sectors.

Patient representatives were recruited through the Quality and Outcomes of Person-Centred Care Policy Research Unit’s (QORU) Public Involvement and Implementation Group, a strategic group which oversees public involvement in QORU projects and includes members of the public.

Forty stakeholders and members of the Public Involvement and Implementation Group were sent an email invitation and participant information sheet about the study. Stakeholders were sent a reminder email where no response was received. We aimed to conduct 25 interviews across patient representatives, health and social care policy makers, system stakeholders and professionals. It was anticipated that this sample size would enable a range of perspectives regarding integrated care and its measurement to be captured thus providing detailed information about the evaluation of integrated care, the challenges and issues.

Ethics approval was granted by the University of Oxford’s Medical Sciences Interdivisional Research Ethics Committee (Reference: R59996/RE001). All participants provided written consent via a secure online consent form prior to taking part.

Participants

Twenty-five people (19 professional stakeholders and 6 patient representatives) took part in a qualitative interview. Nineteen stakeholders responded to the email invitation and were interviewed. The remainder did not respond ( n  = 14), could not be reached (e.g. undeliverable email) ( n  = 6), or were too busy to participate ( n  = 1). Seven patient representatives contacted the research team expressing an interest in participating in the study, before recruitment was stopped due to sufficient numbers being reached. Six patient representatives were interviewed. The average duration of interviews was approximately 30 min. The roles and affiliations of professional stakeholders are summarised below (note that three stakeholders also held clinical roles):

Two senior representatives of NHS England;

Four stakeholders in lead roles within Clinical Commissioning Groups (bodies responsible for the purchase of local health care services);

Three stakeholders in lead roles within other NHS bodies (e.g. Health and Care Partnerships);

Four senior representatives of local authorities (bodies responsible for the provision of local public services, including social care). One of these representatives held a joint role with a Health and Care Partnership;

One senior representative from the Local Government Association;

Four senior representatives of health and/or care focussed charities;

and a senior academic working in the field of integrated care.

A semi-structured interview guide (see Table  1 ) was developed following a review of relevant literature [ 18 ], and focused on the benefits of integrated care and how it should be evaluated. The guide was tested in two mock interviews (led by HC and LK), amended, and further refined after the first few interviews to allow broader exploration of available metrics for integrated care, and their selection. All interviews were conducted over the telephone between November 2018 and January 2019 (by HC, JH, or LK) and audio-recorded with the participants’ consent.

Interviews were transcribed verbatim by a professional transcriber and checked against the audio-recording by a researcher (HC or LK), with corrections made as appropriate. All transcripts were anonymised.

A framework analysis was conducted in five stages: familiarization; identifying a thematic framework; indexing; charting; and mapping and interpretation [ 19 , 20 ]. The thematic coding framework was developed following familiarisation with the data set, and broadly followed the interview guide. The framework was discussed and agreed upon by all authors. Next, the data was indexed using NVivo 12 software. Finally, a chart summarising the relevant data for each case against each qualitative theme was developed to aid interpretation.

Quotations from professional stakeholders are shown as “S” followed by an ID number. The following hierarchy of descriptors can be referred to when interpreting the results:

Most or majority (≥75% of participants)

Many or often (51–74% of participants)

Some or several (26–50% of participants)

A few or limited (≤25% of participants)

Three overarching themes arose from the data, and they are discussed in turn below. The first theme is concerned with how participants viewed integrated care and its benefits. The second theme focuses on potential measures for integrated care. The final theme explores some broader contextual and methodological considerations surrounding assessing benefits of integration initiatives.

Integrated care and its benefits

When asked to define integrated care, the majority of stakeholders first and foremost defined it from the perspective of the patient or service user as an approach that coordinates services around the individual to fulfil their needs. Most went on to state that a key purpose of taking a person-centred approach to integrated care is to achieve better outcomes for patients and service users, for example, keeping people in their homes for longer, maintaining independence, reducing unnecessary admissions, improving health outcomes, and quality of life. In addition to improved outcomes, the potential for integrated care to provide patients with a better experience of care was also raised by stakeholders. Examples given of what a better patient experience may look like included: improved access to services, smoother transitions between services, increased involvement in decision-making, receiving the right care at the right time in the right place, and telling ‘their story’ less frequently.

Secondary to the person-focused definition offered above, some stakeholders provided a systems-focused perspective of integrated care, as an activity that coordinates or brings services together to provide ‘joined-up’ or ‘seamless’ care. Some outlined this vision in greater detail, specifying the services involved in integration (e.g. health, community, voluntary, and social care), as well as detailing the mechanisms by which integrated care can be achieved, for example, vertical or horizontal integration, and multi-disciplinary teams (MDTs). It is important to note that most stakeholders provided a systems-focused perspective in addition to the person-centred definitions given, outlining the structures and processes necessary in order to achieve better outcomes and experiences for patients. For example, one stakeholder spoke about how improvements to the structure of the health and care systems, specifically the integration of IT systems, could benefit patients:

“We don’t have a single joined up system. We don’t yet really have shared information systems which are incredibly important. And they're incredibly important enabler for integration. And so that’s what it's about for me – joining up information, making sure that care professionals have access to the right levels of information that they can use to make joint decisions that are with the patient as well, to improve the patient and service user outcomes.” (S10, Programme Lead, Clinical Commissioning Group)

Although structures and processes were discussed, the benefits of integrated care were generally coached in terms of the patient or service user. A few stakeholders further emphasised a person-centred approach by specifically stating that the end goal of integrated care is not the integration of structures, but rather the outcomes that can be achieved:

“It's [integrated care] about the experience and the improved outcomes that it brings to people, that is what it's all about; it's not about structures … it's about improving people's health and well-being, it's about closing health and inequalities, it's about keeping people well at home, giving them a better, more co-ordinated experience of care, about helping them to help themselves, about using, all of the assets in a place to support the community to live as happily and independently and safely as they can. And so that’s what integrated care is to us. It's not a thing in itself; it's the outcomes that it achieves.” (S9, Senior Advisor, Local Government Association)

The benefits of integrated care to service providers were also discussed, albeit less than the benefits to patients and service users, with two key areas of benefit raised. Firstly, many stated improved service efficiencies as a key benefit, achieved through more efficient pathways, reduced duplication of work (e.g. reduced number of assessments), reduced admissions, and an improved ability to make decisions, plan, and co-ordinate care. Secondly, the potential for integrated care to reduce costs was often discussed, although there were discrepancies between stakeholders regarding whether savings could be realised, with several acknowledging that this was not a given and that current evidence to support this was limited. In addition to these two key areas, a few stakeholders also highlighted the potential for integrated care to improve the experience and retention of staff.

Providing a definition of integrated care was more challenging for patient representatives. However, of those who were able, they provided similar albeit less detailed definitions than professional stakeholders, focused on bringing services together for the benefit of the patient:

“different organisations getting together … clinicians, carers, community people, researchers, research scientists, sharing knowledge and sharing anything that might be relevant to the patient that might improve the outcome to a patient” (Patient Representative 3)

Furthermore, patient representatives also provided similar descriptions of the benefits of integrated care, with a particular focus on streamlining processes to benefit the patient; for example, having one point of contact and being asked fewer times for the same information.

Measures of integrated care

All professional stakeholders discussed how they thought integrated care should be evaluated, both in terms of the aspects to evaluate and how best to measure those aspects. Several professional stakeholders spoke about the difficulty of measuring the impact of integrated care, particularly with regards to the challenges faced when selecting appropriate measures, and assessing the benefits to patients and service users. Under current arrangements, with diverse measures employed across the system, a few stakeholders mentioned that it was impossible to draw general conclusions about the effectiveness of alternative models of integration. As one might expect, patient representatives discussed the measurement of integrated care in much less detail than professional stakeholders. While various system metrics were raised as possible measures of integrated care, patient representatives generally thought evaluation should focus on patients’ experiences of care, and any improvements in patient outcomes.

Three broadly distinct types of measure of the benefit of integration were discussed: system effects, patient experience and patient outcomes.

System effects

System metrics were commonly referred to by stakeholders when discussing measures of integrated care, of which, delayed transfers of care (DTOC), number of emergency admissions, and length of stay, all of which relate to hospital care, were among those most frequently mentioned. While there was great awareness among stakeholders of metrics used to assess system outcomes, and their value acknowledged, many felt that evaluations of integrated care focus too heavily on system effects. Concerns were raised that such an approach is too narrow and does not necessarily effectively evaluate the broader impact of integrated care and the benefits to patients. In particular, some regarded these metrics as ‘blunt instruments’ that do not capture the true impact and benefits of integrated care as defined by stakeholders, i.e. whether system changes are improving patient experiences and outcomes:

“The outcomes often associated with integrated care are hardcore metrics like reductions of emergency admissions, unplanned admissions, length of stay and that sort of thing … these are the metrics that they use to measure integrated care against, don’t actually demonstrate an effective connection, and that’s because those metrics in themselves aren't particularly sensitive to what is the aspiration of integrated care.” (S11, Senior Academic, University)

Vast awareness and use of system metrics is, in part, likely explained by the requirement of regulators to collect data on certain system outcomes, for example, Delayed Transfers of Care (DTOC). An Assistant Director at a local authority questioned this focus:

“The sign of a good performing system is that you have low levels of DTOC … yes, it's nice to have a low level of DTOC, but that still means that the person's been in hospital. They could have been in hospital for three or four weeks. Is that necessarily a good experience for them? … Sometimes we latch onto a particular measurement and then say, "Oh yeah, your system's performing well because you’ve got a really low DTOC." Well, not if you’ve got really high non-elective admissions and really high readmissions and really high residential admissions … that tells me that the system's not operating as well as it could.” (S4, Assistant Director, Local Authority)

Patient experiences

The majority of participants spoke about the need to measure patients’ experiences, with a few specifically extending this to include the experiences of carers and / or family members. Furthermore, several thought it also important to measure the experiences of the workforce.

Patient representatives frequently referred to experiences of care, talking about their own experiences from a patient or carer perspective, as well as referring to the experiences of others (e.g. family, friends and neighbours). One patient representative described the experience of their neighbour:

“She suffered a mental health breakdown with two young children, and the [city] social services department took six months to assess her. And even then they didn’t complete the assessment because the lady was homeless. And they moved her to temporary accommodation into the neighbouring borough of [location], where they had to start the whole process again because they said they'd lost the papers … So, there's no communication between one department and another department.” (Patient Representative 1)

This focus reinforces the stakeholders’ views of the importance of measuring patients’ experiences of care.

Some difficulties around measuring patients’ experiences were raised by stakeholders, for example, often patients and service users provide a positive report of their care despite negative aspects occurring, which often only come to light during subsequent discussions with the patient.

Patient outcomes

In general, stakeholders discussed patient-related outcomes in broader terms than they had discussed system effects. For example, some stakeholders advocated the importance of patient outcomes but generally without specificity or detail. This may be an effect of system outcomes being measured more frequently, and therefore, system metrics being better known to stakeholders than patient outcome measures. This was not true of all, with a few participants able to discuss dimensions to be measured (e.g. wellbeing, functionality, and pain), and tools that could be used to elicit such data, in much greater detail.

Several stakeholders called for a wider “ basket of measures ” to broaden measurement beyond system effects when evaluating the success of integrated care programmes. A few felt that measuring patient outcomes should be central to evaluations, for example, a Senior Associate of a charity stressed that “fundamentally, the best judge of a service or a package of services, is from the perspective of the people experiencing care” (S1, Senior Associate, Charity).

Overall, very few specific health indicators were referred to, with life expectancy, functionality, and dental decay among those mentioned. The majority of discussion was about patient- or carer- reported outcomes. Quality of life, well-being, or specific aspects of quality of life, such as pain, anxiety, and depression were raised by the majority of stakeholders as aspects of integrated care that should be evaluated. The value of measuring aspects such as these was highlighted in an example shared by a Senior Manager, who described an instance where a GP visited an elderly man with multiple conditions nearing end of life:

“And the GP thought, 'Well what can I do for this guy?' and he... so he just asked him. He said, "I'd really like my beard trimmed." So, he kind of got a barber to come round and trim his beard and he said it kind of transformed this guy's kind of, you know, you know, made him feel so much better. And his blood pressure was lower and things like that. We don’t... but we don’t kind of pick that up. I think all you'd pick up was the fact that whether the GP went there and how long he was there for, or she was there for, you know.” (S6, Senior Manager, County Council)

However, a few held some reservations about collecting such data due to its subjective nature, and problems with measurement (e.g. comparability):

“I might say my quality of life's really high and I enjoy my life; I could live in absolute squalor. And other people would say my quality of life is very poor. So, I think it's more difficult around quality of life … I think it is right that we ask the question, but I think it's one of those ones that you need to think about that is subjective.” (S4, Assistant Director, County Council)

Patient-reported measurement instruments

When asked about available patient-reported measures for assessing integrated care, broadly speaking, stakeholders working at a national level (e.g. national charities) and in academia were far more familiar with specific measures than those working at a local level (e.g. County Councils and Clinical Commissioning Groups). Measures named, included, the Patient-Centred Coordinated Care Experience Questionnaire (P3CEQ), the Friends and Family Test, the Patient Activation Measure (PAM-13), the Warwick-Edinburgh Mental Well-being Scale (WEMWBS), and the EuroQol 5 Dimensions (EQ-5D). None of the measures named were strongly advocated by the professional stakeholders. Although patient representatives did not specifically name any patient-reported measurement instruments, many had previously seen or completed patient-reported outcome or experience questionnaires as part of their care. One patient representative described their experience:

“I have endless review questionnaires following interventions about, you know, pain scores and all that sort of stuff. Can we do this, can we do that more, you know, was it better, was it worse, was it just the same. But no, I personally, have a bit of trouble with those as a patient because ticking boxes and being a human being don’t seem very compatible to me, but I do understand that it's the way we measure things. And I know you get something from it which translates into something which will help me.” (Patient Representative 4)

Contextual considerations

In addition to discussing potential measures for integrated care, stakeholders also raised some additional factors for consideration when evaluating integrated care programmes. Both professional stakeholders and patient representatives raised queries around the purpose of data collection in health and care settings. One patient representative discussed the lack of transparency with patients regarding the purpose for which data is collected, referring to “ endless surveys ” that go “ into a dark hole and you assume that nobody ever reads” (Patient Representative 4). Several professional stakeholders called for greater clarity around why data is being collected and how that data will be used:

“The other thing that worries me is are we using that data? … it's very fashionable to talk about, you know, people's mental health and all this sort of stuff, so people feel that they ought to collect it, but people should only be collecting it if they're going to use it, you know. If, you should only be asking these people these questions if they’ve got a plan for that data. If they just want to kind of want to put it on a database somewhere and forget about it then it’s unethical.” (S6, Senior Manager, Council)

A further issue, raised by a few stakeholders, was around the tensions between collecting data for use at a local or national level. While stakeholders recognised the need to collect data at a national level to allow comparisons to be drawn between providers, there were concerns that national level data lacked local relevance, and necessitated a loss of local data ownership. An Assistant Director at a Council questioned the value of nationally imposed metrics:

“We are supposed to work in an integrated way and cooperate, but I think the culture that underpins that has been one of intimidation from the top. An insistence of further solutions by certain providers being delivered, and whether you’ve got a top driven set of metrics that have come from NHS England and don’t reflect the reality of local government funding. Those metrics become either as just a stick to beat people with. And it doesn’t help the integration of human beings working together on the ground; it gets in the way.” (S15, Assistant Director, Council)

Qualitative methods

Qualitative methods of evaluating integrated care were increasingly being used or advocated by many stakeholders in order to overcome difficulties they had experienced with quantitative evaluation of integrated care initiatives. A senior academic discussed their qualitative approach to evaluation:

“We get people to define what is it you wanted to get out of it and did integrated care actually help you? So, it's impossible to plonk it on a politician's desk and say, 'You know, with this integrated care initiative, X number of outcomes were achieved,' because it's not like that. Integrated care is much more evaluation of a process. And individually, did the processes actually facilitate some kind of improvement?” (S11, Senior Academic, University)

Stakeholders often discussed the use of qualitative methods to capture data about the benefits of integrated care from the patients’ perspective. Some advocate these methods as a way of hearing the stories of patients. Despite discussing the benefits of a qualitative approach, it was acknowledged that there is still a need for good metrics with which to evaluate and report on integrated care.

“Asking the person at the end of it how integrated their experience has been and then acting on what they tell us. It's experiential, it's qualitative and it's not going to be easily captured and quantified and reported upon.” (S15, Assistant Director, Council)

It has to be noted that while many stakeholders referred to “ qualitative ” approaches, it was evident that a few were in fact referring to patient experience surveys rather than qualitative methodological approaches, such as interviews and focus groups.

In this diverse sample of professional stakeholders and patient representatives with a range of interests in the process of integrating health and social care, there was striking optimism that increased integration would eventually yield substantial benefits to patients and users of services. There was consensus that currently health and social care services are a very long way from being integrated and that the main measures used to assess progress, such as delayed transfer of care, while important, do not directly address patient benefits.

When invited to consider how benefits of integration should be assessed, participants commonly referred to broadly expressed dimensions such as well-being and health-related quality of life. However such constructs were generally seen as difficult to operationalise and little specific evidence was cited of integration being assessed against outcomes such as well-being. Reference was also made by some participants to the potential value of capturing patients’ experiences of services, i.e. how services were experienced in terms of being joined up and coordinated. For some, such responses to integrated care would best be captured by qualitative and narrative methods.

Although there was broad agreement that progress in integration would be best captured through the measurement of experiences and outcomes (e.g. dimensions such as well-being), no single measure was widely cited. However, individuals were able to cite specific measures such as the Patient Activation Measure or EQ-5D as potentially relevant. Furthermore, stakeholders generally discussed the applicability of measures in broader terms rather than considering their relevance to specific groups.

Related to the lack of agreement about specific measures of outcomes of integration was widespread uncertainty as to where responsibility for outcomes assessment was best located, at local or national level. This uncertainty reinforced the sense that no shared metric could easily emerge that would inform the system as a whole of benefits of initiatives to develop integrated services.

The development of measures of health status, often referred to as patient-reported outcome measures, has been hailed as the most important scientific break-through in fifty years, permitting the evaluation of services by means of broad measures of health as viewed and valued by patients and the public [ 21 ]. There are now readily available validated measures of health intended to capture the benefits of system innovations in health and social care [ 22 , 23 ]. There are, in addition, attempts to define sets of indicators to assess and monitor integrated care [ 24 ], and logic models that delineate relationships between systems and outcomes [ 25 ]. It is clear that there still remains a substantial gulf between health services research and the world of everyday service providers, managers and commissioners, which is holding back the use of evaluative methods to inform innovation [ 26 ]. One aspect of the gulf is limited awareness of potentially relevant patient-reported outcome measures and their application.

One way in which this gulf could be narrowed is through the development and promotion of a core measurement set for integrated care, i.e. a standardised set of measures that represent the minimum group of processes, structures, and outcomes that should be collected and reported for all integrated care initiatives. This type of approach follows that set out by initiatives such as the International Consortium for Health Outcomes Measurement (ICHOM), which focuses on the development of Standard Sets for use by health care providers [ 27 ], or the Core Outcome Measures in Effectiveness Trials (COMET) initiative, which aims to standardise measurement across clinical trials for a specific area of health or health condition [ 28 ]. At the heart of this type of approach is the identification and agreement of what matters most to patients and service users in terms of outcomes from services, together with the identification and agreement about best available specific measures of the most important outcomes.

A potential weakness of these conclusions is the sample size of respondents recruited to the study. An enormous array of professional, managerial and patient groups have an interest in the broad issue of integration of health and social care and the current study cannot claim to have fully sampled this diversity of views. The recruitment of professional participants with in-depth knowledge of integrated care led to a sample of professionals in more senior positions. However, further work would benefit from the inclusion of professionals of varying levels of seniority to capture a wider range of views. Patient representatives were less familiar with the term ‘integrated care’ and its definition, with some finding it difficult to answer some of the questions as a result. However, it is essential to include patient representatives, and further work is needed to explore how best to overcome terminology and conceptual barriers to participation. Interviews were also pragmatic and restricted in time to accommodate respondents’ time constraints.

This study makes an important contribution to understanding professional and lay stakeholders’ views on capturing the benefits of integrated care, but further work is needed to better understand the challenges of measuring health outcomes and patient, informal carers, and staff experiences as part of integrated care evaluations. Patients and informal carers should be central to the development or selection of measures and indicators to ensure that they truly reflect any differences that integrated care has made.

The challenges of measuring integrated care are well documented. In this paper, we sought the views of professional stakeholders and patient representatives about how best to measure the integration of health and social care, and the benefits of integrated care approaches. Stakeholders primarily defined integrated care and its benefits from the patients’ perspective, however, there was consensus among stakeholders that there is not yet enough focus on patient benefit. There is potential for this to be overcome as validated patient-reported measures of health are available. However, there is currently limited awareness of measures of key constructs such as wellbeing among stakeholders, and therefore considerable scope for developing consensus on optimal measures. The study provides clear evidence that the prospect of directly assessing patient benefits of the integration of health and social care services is feasible, although still quite distant.

Availability of data and materials

Due to ethical concerns, the interview transcripts cannot be made openly available.

Abbreviations

Core outcome measures in effectiveness trials

Delayed transfers of care

EuroQol 5 dimensions

International consortium for health outcomes measurement

Integrated care systems

Multi-disciplinary team

National health service

Patient-centred coordinated care experience questionnaire

Patient activation measure

Quality and outcomes of person-centred care policy research unit

Warwick-Edinburgh mental well-being scale

World health organisation

Baxter S, Johnson M, Chambers D, Sutton A, Goyder E, Booth A. The effects of integrated care: a systematic review of UK and international evidence. BMC Health Serv Res. 2018;18(1). doi: https://doi.org/10.1186/s12913-018-3161-3 .

Ham C. Making sense of integrated care in the NHS in England. London: The King’s Fund; 2018 June.

Google Scholar  

Rutten-van Mölken M, Leijten F, Hoedemakers M, Tsiachristas A, Verbeek N, Karimi M, Bal R, de Bont A, Islam K, Askildsen JE, Czypionka T, Kraus M, Huic M, Pitter JG, Vogt V, Stokes S, Baltaxe E. Strengthening the evidence-base of integrated care for people with multi-morbidity in Europe using Multi-Criteria Decision Analysis (MCDA). BMC Health Serv Res. 2018;18(576). doi: https://doi.org/10.1186/s12913-018-3367-4 .

van der Heide I, Snoeijs S, Melchiorre MG, Quattrini S, Boerma W, Schellevis F, Rijken M. Innovating care for people with multiple chronic conditions in Europe: an overview. 2015. http://www.icare4eu.org/pdf/Innovating-care-for-people-with-multiple-chronic-conditions-in-Europe.pdf . Accessed 04 June 2019.

World Health Organization. Framework on integrated people-centred health services. 2016. https://www.who.int/servicedeliverysafety/areas/people-centred-care/framework/en/ . Accessed 04 June 2019.

SQW. Summative evaluation of the Integrated Personal Commissioning (IPC) Programme: interim report – process evaluation. 2017. http://www.sqw.co.uk/files/8215/1818/6800/SQW_first_process_evaluation_report.pdf . Accessed 04 June 2019.

Erens B, Wistow G, Mounier-Jack S, Douglas N, Jones L, Manacorda T, Mays N. Early evaluation of the integrated care and support pioneers Programme: final report. Policy Innov Res Unit. 2016:1–175. https://piru.lshtm.ac.uk/assets/files/Early_evaluation_of_IC_Pioneers_Final_Report.pdf . Accessed 29 May 2019..

Ling T, Brereton L, Conklin A, Newbould J, Roland M. Barriers and facilitators to integrating care: experiences from the English Integrated Care Pilots. Int J Integr Care. 2012;12(5). https://doi.org/10.5334/ijic.982 .

Forder J, Caiels J, Harlock J, Wistow G, Malisauskaite G, Peters M, Marczak J, D’Amico F, Fernandez J-L, Fitzpatrick R, Jones K. A system-level evaluation of the better care fund: final report. Qual Outcomes Person-Centred Care Policy Res Unit. 2018:i–130. https://www.pssru.ac.uk/pub/5424.pdf . Accessed 31 May 2019.

NHS. The NHS long term plan. NHS England. 2019. https://www.longtermplan.nhs.uk/publication/nhs-long-term-plan/ . Accessed 04 June 2019.

Armitage GD, Suter E, Oelke ND, Adair CE. Health systems integration: state of the evidence. Int J Integr Care. 2009;9(2). doi: https://doi.org/10.5334/ijic.316 .

National Collaboration for Integrated Care and Support. Integrated care and support: our shared commitment. Department of Health and Social Care. 2013. https://www.gov.uk/government/publications/integrated-care . Accessed 31 May 2019.

National Voices. A narrative for person-centred coordinated care. NHS England. 2013:12.  https://www.nationalvoices.org.uk/sites/default/files/public/publications/narrative-for-person-centred-coordinated-care.pdf . Accessed 31 May 2019.

National Audit Office. Planning for the Better Care Fund, HC 781, Session 2014–15. 2014. www.nao.org.uk/wp-content/uploads/2014/11/Planning-for-the-better-care-fund.pdf . Accessed 08 May 2020.

Bautista MA, Nurjono M, Lim YW, Dessers E, Vrijhoef HJ. Instruments measuring integrated care: a systematic review of measurement properties. Milbank Q. 2016;94(4):862–917.

Article   Google Scholar  

Suter E, Oelke ND, da Silva Lima MAD, Stiphout M, Janke R, Witt RR, Van Vliet-Brown C, Schill K, Rostami M, Hepp S, Birney A, Al-Roubaiai F, Marques GQ. Indicators and measurement tools for health systems integration: a knowledge synthesis. Int J Integr Care. 2017:17(6). doi: https://doi.org/10.5334/ijic.3931 .

Glimmerveen L, Nies H, Ybema S. Citizens as active participants in integrated care: challenging the field’s dominant paradigms. Int J Integr Care. 2019;19(1). doi: https://doi.org/10.5334/ijic.4202 .

Kelly L, Harlock J, Peters M, Fitzpatrick R, Crocker H. Measures for the integration of health and social care services for long-term health conditions: a systematic review of reviews. BMC Health Serv Res. 2020;20(358). doi: https://doi.org/10.1186/s12913-020-05206-5 .

Ritchie J, Spencer L. Qualitative data analysis for applied policy research. In: Bryman A, Burgess RG, editors. Analyzing qualitative data. London: Routledge; 1994. p. 173–94.

Chapter   Google Scholar  

Srivastava A, Thomson SB. Framework analysis: a qualitative methodology for applied policy research. J Adm Governance. 2009;4(2):72–79.

Brook RH, Vaiana ME. Using the knowledge base of health services research to redefine health care systems. J Gen Intern Med. 2015;30(10):1547–56.

Malley JN, Towers A-M, Netten AP, Brazier JE, Forder JE, Flynn T. An assessment of the construct validity of the ASCOT measure of social care-related quality of life with older people. Health Qual Life Out. 2012;10(21). doi: https://doi.org/10.1186/1477-7525-10-21 .

Potter CM, Batchelder L, A'Court C, Geneen L, Kelly L, Fox D, Baker M, Bostock J, Coulter A, Fitzpatrick R, Forder JE, Gibbons E, Jenkinson C, Jones K, Peters M. Long-Term Conditions Questionnaire (LTCQ): initial validation survey among primary care patients and social care recipients in England. BMJ Open. 2017;7(11). doi: https://doi.org/10.1136/bmjopen-2017-019235 .

Scobie S. Trusted measures: analytical resources for integrated care. Nuffield Trust. 2018. https://www.nuffieldtrust.org.uk/resource/trusted-measures-analytical-resources-for-integrated-care . Accessed 06 June 2019.

Social Care Institute for Excellence. Logic model for integrated care. 2017. https://www.scie.org.uk/integrated-health-social-care/measuring-progress/logic-model . Accessed 6 June 2019.

Lamont T, Barber N, de Pury J, Fulop N, Garfield-Birkbeck S, Lilford R, Mear L, Raine R, Fitzpatrick R. New approaches to evaluating complex health and care systems. BMJ. 2016;352:i154. doi: http://doi.org/ https://doi.org/10.1136/bmj.i154 .

ICHOM. Standard sets. https://www.ichom.org/standard-sets/ . Accessed 14 May 2020.

Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, Clarke M, Gargon E, Gorst S, Harman N, Kirkham JJ, McNair A, CAC P, Schmitt J, Terwee CB, Young B. The COMET Handbook: version 1.0. Trials. 2017;18(280). https://doi.org/10.1186/s13063-017-1978-4 .

Download references

Acknowledgements

We would like to thank the professional stakeholders and patient representatives for participating in the qualitative interviews.

This paper reports independent research funded by the Policy Research Programme (PRP) in the Department of Health and Social Care England, firstly through the Quality and Outcomes of Person-centred Care Policy Research Unit (QORU), and subsequently the National Institute for Health Research (NIHR) Policy Research Unit (PRU) on Quality, Safety, and Outcomes (QSO) (PR-PRU-1217-20702). This study was also funded by the NIHR Collaboration for Leadership in Applied Health Research and Care (CLAHRC) Oxford at Oxford Health NHS Foundation Trust. The funders set only broad areas for long term programmes of research, such as quality and outcomes of health and social care services. They were not involved in the research question, design, data collection and analysis, manuscript preparation or the decision to submit the manuscript for publication. The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

Author information

Authors and affiliations.

Nuffield Department of Population Health, University of Oxford, Richard Doll Building, Old Road Campus, Headington, Oxford, OX3 7LF, UK

Helen Crocker, Laura Kelly, Ray Fitzpatrick & Michele Peters

Harris Manchester College, Oxford, OX1 3TD, UK

Laura Kelly

Health Sciences, University of Warwick, Coventry, CV4 7AL, UK

Jenny Harlock

You can also search for this author in PubMed   Google Scholar

Contributions

RF and MP conceived of the study with all authors contributing to its design. HC, JH, and LK collected the interview data. HC led the analysis of interview data, with the support of RF and MP. HC led on drafting the manuscript with input from RF, JH, LK and MP. All authors have critically reviewed and approved the final version of the manuscript.

Corresponding author

Correspondence to Helen Crocker .

Ethics declarations

Ethics approval and consent to participate, consent for publication.

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Crocker, H., Kelly, L., Harlock, J. et al. Measuring the benefits of the integration of health and social care: qualitative interviews with professional stakeholders and patient representatives. BMC Health Serv Res 20 , 515 (2020). https://doi.org/10.1186/s12913-020-05374-4

Download citation

Received : 27 September 2019

Accepted : 27 May 2020

Published : 06 June 2020

DOI : https://doi.org/10.1186/s12913-020-05374-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Integrated care
• Qualitative interviews
• Outcome measures
• Health and social care
• Measurement

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

How to Conduct Research in Healthcare in 9 Steps

• March 13, 2023

Table of Contents

Conducting research is a complex and multi-step process. This guide will provide an overview of how to conduct healthcare research, including the steps involved and key considerations for each stage.

1: Define the Objectives of Your Research Study

The objectives of the research study should include both short-term and long-term goals . It is essential to have a solid plan with well-defined objectives and the right methods to ask the right questions to the right people. 

Research should also consider ethical issues and the steps needed to minimize harm to those who take part. Furthermore, potential benefits to users must be considered, and it’s important to include time frames, milestones, and communication strategies.

Analyze and interpret data collected from surveys or interviews to identify trends related to the impact on desired outcomes, such as changes in attitudes or behaviors over time and changes in policies or practices.

2: Identify Relevant Studies and Sources

Searching the literature helps ensure that the research will address an appropriate gap in the evidence and help refine the research question and choice of methods. Critiquing the methodology of other people’s work is key to building up a better understanding and justification for a research approach.

The search strategy can be developed and enriched using relevant terms and literature from MeSH , Medline Plus , Ovid MEDLINE, EMBASE , CINAHL , HMIC , and Oxford Academic’s Research Evaluation databases. 

Screen all retrieved citations for eligibility based on your inclusion criteria and discard any that do not meet them (e.g., non-English language articles). Review the remaining articles for quality using appropriate assessment tools (e.g., QUADAS ).

Lastly, apply systematic review methodology (e.g., PRISMA ) to ensure transparency in reporting findings from each study reviewed in your review article or paper report.

? Learn more: Common Data Sources in HEOR

3: Consider Ethical Approval and Consent Requirements

Research teams should follow ethical guidelines and seek approval from a research ethics committee for their plan. The research should not involve any unnecessary harm or risk to the participants, and all possible measures should be taken to ensure their welfare and safety during the study period.

In addition, studies involving new medicines should have gone through a process to ensure they are ready for use with people before being tested on humans or animals in clinical trials or laboratory experiments, respectively.

Consent should also be obtained from all participants before they take part in the study, outlining what they can expect from it as well as any potential risks involved in taking part in it (if any).

4: Create a Study Plan and Timeline

After identifying the topic and research question, you would like to explore and review the eligibility criteria for each registry or study , as well as any additional requirements, such as consent forms or surveys, to determine which ones are suitable for your project goals and needs. 

Select one or more registries or studies that fit your needs best, then register with them if necessary by following their instructions on how to participate in their study.

Don’t forget to create a timeline for how long you will be involved in each study so you can keep track of progress toward completion of the project goals.

5: Construct a Questionnaire or Survey

Identify the purpose of the research and the target population and determine what information you are trying to gather, as well as who will be most affected by your research.

Create a survey or questionnaire that includes questions that are relevant to your purpose and can be answered by respondents in a way that is meaningful to them. Make sure it is clear, concise, and easy to understand for all ages and levels of education and language abilities.

Pilot-test your survey with a small group of people from your target population before rolling it out across the board for full use in research activities (e.g., interviews).

Finally, analyze responses using descriptive statistics (e.g., mean scores) or inferential statistics (for example, t-tests) as needed depending on what questions are being asked of participants in order to understand their experiences better or make comparisons between groups if applicable.

? Related: Biostatistics in HEOR

6: Write Your Research Proposal

The process of writing a research proposal can be broken down into six manageable steps.

• Introduce an educated idea arising from a clinical practice issue . The research topic should be rooted in a problem that affects medical practice; it should also provide motivation for completing the research and relevance for affecting medical practice changes and improvements.
• Conduct a systematic literature review before starting to write . This will help to inform the research idea and provide evidence to support the project from the beginning.
• Develop a conceptual framework . This will help to clarify the research question and ensure the proposed project is feasible and efficient.
• Seek advice from clinical experts, experienced researchers, relevant stakeholders , and even patients. This will enhance the research question’s relevance, feasibility, and efficiency in your document.
• Structure the research proposal . It should include background, objectives, methodology, outcomes, and dissemination. Additionally, include a section on ethical issues, service user involvement, and project management.
• Proofread the proposal carefully before submitting it to a funding body or ethical review panel. Consider having the proposal read by a consumer of health, colleague, or friend to ensure clarity and that jargon would be understood by a wider audience.

7: Collect Data Manually or Electronically

Determine the end goal of your research project and decide whether qualitative or quantitative data collection methods would be most beneficial for achieving your goal.

Select appropriate data collection methods, such as interviews, focus groups, observation for qualitative research, multiple choice surveys, or structured data analysis for quantitative research.

Analyze the collected data using specific tools and methods relevant to each data collection method (e.g., qualitative analysis software). This step should be done thoroughly to draw valid conclusions from the findings.

8: Elaborate on Conclusions and Report Results

First, identify the purpose of the research study and determine which outcomes are most relevant . Next, plan how to communicate results to study participants , including what information to provide, when it should be provided, and how it should be presented.

When possible, involve patients or other individuals who have participated in the study in developing feedback mechanisms that are meaningful to them.

At a minimum, provide participants with a summary of findings from their records that are clinically relevant (as determined by researchers) as well as an explanation of these findings in an understandable format.

In addition, you can consider providing additional information, such as data tables or graphs showing trends over time if applicable; this may help individuals better understand their healthcare situation more clearly. You can also provide opportunities for further discussion if needed.

9. Disseminate Your Findings

Depending on the size and scope of your research, disseminating your research could include media releases and interviews, conference presentations and posters, white papers, policy briefs, infographics, or other formats.

Follow best practices when communicating your findings. Consider the National Institutes of Health Guide, A Checklist for Communicating Science and Health Research to the Public , for tips on making research results accessible to a broad audience. 

Additionally, the Rural Health Research Gateway’s Dissemination Toolkit provides examples of fact sheets, policy briefs, and other dissemination formats, as well as general guidelines for sharing rural health research findings.

By taking the previous steps, you can ensure that your research is accurate and comprehensive and can be communicated thoroughly to all stakeholders. This will give you the confidence to present your findings in a meaningful way and disseminate them to the appropriate audiences.

Health Analytics bridges the gap between sound scientific data and market access. Phone: (410) 997-3314 Email: [email protected]

• Health Care
• Evidence Generation Planning
• Evidence Generation
• Translating to Market Access

This site uses session cookies and persistent cookies to improve the content and structure of the site.

By clicking “ Accept All Cookies ”, you agree to the storing of cookies on this device to enhance site navigation and content, analyse site usage, and assist in our marketing efforts.

By clicking ' See cookie policy ' you can review and change your cookie preferences and enable the ones you agree to.

By dismissing this banner , you are rejecting all cookies and therefore we will not store any cookies on this device.

UK Policy Framework for Health and Social Care Research

This policy framework sets out principles of good practice in the management and conduct of health and social care research in the UK. 

These principles protect and promote the interests of patients, service users and the public in health and social care research, by describing ethical conduct and proportionate, assurance-based management of health and social care research, so as to support and facilitate high-quality research in the UK that has the confidence of patients, service users and the public.

It is for organisations and individuals that have responsibilities for health and social care research. This includes funders, sponsors, researchers and their employers, research sites and care providers.

The policy framework applies to health and social care research involving patients, service users or their relatives or carers. This includes research involving them indirectly, for example using information that the NHS or social care services have collected about them.

The Health Research Authority and the health departments in Northern Ireland, Scotland and Wales have developed the policy framework following  public consultation . It replaces the separate Research Governance Frameworks in each UK country with a single, modern set of principles for the whole UK.

Read the policy framework

You can read the UK Framework for Health and Social Care Research in full on our website or download the UK Framework for Health and Social Care Research (PDF) .

We welcome your comments on the framework, which will help us improve future versions; and any examples of where you have used the framework as part of your work.

"I welcome publication of the UK Policy Framework for Health and Social Care Research which sets out the principles of good practice in the management and conduct of health and social care research across the UK. The Framework will ensure that the public will  continue to feel safe when they take part in research and it will enable researchers to develop innovations which will help to improve the quality of health and care in the UK."

Related links

• Replacing the Research Governance Framework
• New framework for UK health and social care research launched
• UK Policy Framework for Health and Social Care Research: consultation
• Privacy notice
• Terms & conditions
• Accessibility statement
• Feedback or concerns

• Search Menu
• Ageing - Other
• Bladder and Bowel Health
• Cardiovascular
• Community Geriatrics
• Dementia and Related Disorders
• End of Life Care
• Ethics and Law
• Falls and Bone Health
• Frailty in Urgent Care Settings
• Gastroenterology and Clinical Nutrition
• Movement Disorders
• Perioperative Care of Older People Undergoing Surgery
• Pharmacology and therapeutics
• Respiratory
• Sarcopenia and Frailty Research
• Telemedicine
• Advance articles
• Editor's Choice
• Supplements
• Themed collections
• The Dhole Eddlestone Memorial Prize
• 50th Anniversary Collection
• Author Guidelines
• Submission Site
• Open Access
• Reasons to Publish
• Advertising and Corporate Services
• Journals Career Network
• Advertising
• Reprints and ePrints
• Sponsored Supplements
• Branded Books
• About Age and Ageing
• About the British Geriatrics Society
• Editorial Board
• Self-Archiving Policy
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Introduction, acknowledgements, declaration of conflicts of interest, declaration of sources of funding, data availability statement.

• < Previous

Including older people in health and social care research: best practice recommendations based on the INCLUDE framework

• Article contents
• Figures & tables
• Supplementary Data

Victoria A Goodwin, Mikaela S A Low, Terence J Quinn, Emma J Cockcroft, Victoria Shepherd, Philip H Evans, Emily J Henderson, Farhat Mahmood, Mary Ni Lochlainn, Catherine Needham, Benjamin R Underwood, Amit Arora, Miles D Witham, Including older people in health and social care research: best practice recommendations based on the INCLUDE framework, Age and Ageing , Volume 52, Issue 6, June 2023, afad082, https://doi.org/10.1093/ageing/afad082

• Permissions Icon Permissions

Older people are often explicitly or implicitly excluded from research, in particular clinical trials. This means that study findings may not be applicable to them, or that older people may not be offered treatments due to an absence of evidence.

The aim of this work was to develop recommendations to guide all research relevant to older people.

A diverse stakeholder group identified barriers and solutions to including older people in research. In parallel, a rapid literature review of published papers was undertaken to identify existing papers on the inclusion of older people in research. The findings were synthesised and mapped onto a socio-ecological model. From the synthesis we identified themes that were developed into initial recommendations that were iteratively refined with the stakeholder group.

A range of individual, interpersonal, organisational, community and policy factors impact on the inclusion of older people in research. A total of 14 recommendations were developed such as removing upper age limits and comorbidity exclusions, involving older people, advocates and health and social care professionals with expertise in ageing in designing the research, and considering flexible or alternative approaches to data collection to maximise opportunities for participation. We also developed four questions that may guide those developing, reviewing and funding research that is inclusive of older people.

Our recommendations provide up to date, practical advice on ways to improve the inclusion of older people in health and care research.

Individual, interpersonal, organisational, community and policy factors impact on the inclusion of older people in research.

Researchers, reviewers and funders should consider engagement, inclusive study design, adequate resources and communication.

Our 14 recommendations offer practical guidance on ways to improve the inclusion of older people in research.

Evidence from health and social care research helps members of the public, patients, practitioners, commissioners and policy makers make decisions about treatment, care and services. However, those recruited into studies, in particular, randomised controlled trials (RCTs), frequently do not represent the population affected by the condition of interest [ 1 , 2 ]. This impacts people on several levels. Firstly, an inequity in opportunities to participate in and benefit from research. Secondly, arbitrary exclusion (direct or indirect) based on a characteristic such as age is a form of discrimination. Thirdly, study findings may not apply to the breadth of people with a condition, and finally, therapies may not be offered due to absence of evidence or may be ineffective or unsafe. In order to address the lack of representativeness of populations recruited to research studies, the National Institute for Health and Care Research (NIHR) Innovations in Clinical Trials Design and Delivery for the Underserved (INCLUDE) project sought to address the barriers to the inclusion of under-served groups in health and care research [ 1 ]. Although there is no single definition for ‘under-served group’, commonly people in such groups have lower rates of inclusion than would be seen in the population of interest, they may engage or respond differently to interventions, or they have a high health and care burden unmatched by the amount of research conducted in that field. Factors to consider in under-served groups include, but are not limited to, demographics (e.g. age, ethnicity, sexual orientation, gender identity), socioeconomic factors (e.g. carers, digitally excluded), health status (e.g. multiple health conditions, hearing impaired) or living circumstances (e.g. care home, homeless).

Heterogeneity is a hallmark of ageing, and older people constitute a broad group, from the robust and healthy through to those with severe frailty and dependency. Older people and those with multiple long terms conditions or frailty are the highest users of health and care resources [ 3 ]. Older people are frequently excluded in many areas of healthcare research, through restrictions in upper age limits [ 4–7 ], the presence of comorbidities [ 6 , 7 ] and cognitive impairment [ 5 ]. In 2011, McMurdo and colleagues published good practice guidance on improving recruitment of older people to research [ 8 ]. They made recommendations on planning and logistic considerations in different settings, recruiting those with impaired capacity and those from ethnic minorities. Nonetheless the problem remains with evidence from a recent review finding that a third of COVID-19 trials had upper age limits that were unjustified, despite COVID-19 disproportionately affecting older people [ 9 ].

New recommendations on the inclusion of older people in research are needed for several reasons. Firstly, our understanding of under-served groups and how to improve inclusion has evolved in the last decade. For some older people, several factors may impact their inclusion in research and this intersectionality is important ( Figure 1 ). Secondly, the clinical research landscape has evolved with new opportunities and challenges (e.g. digital outcomes, virtual trials) that warrant consideration. Finally, previous recommendations focused on those working in ageing research thus there is an opportunity to refine and extend previous guidance to a broader audience including other specialities.

Intersectionality of factors that may impact on the inclusion of older people in research.

The aim of our work was to develop system-level principles to guide all research relevant to older people. The objective was to identify factors that impacted the inclusion of older people in research through stakeholder engagement and evidence synthesis. This paper presents the NIHR INCLUDE Older People best practice recommendations.

Developing the INCLUDE older people best practice recommendations

The INCLUDE Older People project began in January 2021 and continued until September 2022. A diverse stakeholder group was established including multi-disciplinary health and care professionals representing geriatrics, general practice, nursing, physiotherapy and psychiatry, charity representatives (Age UK) and those with expertise in health and care research design and delivery (NIHR Clinical Research Network (CRN), Clinical Trials Units, Patient and Public Involvement), and ethics committee membership. The group also included three older people with experience of being involved in research. To maximise their engagement, alongside the stakeholder meetings, one of the academic team met with them separately to discuss key aspects of the work including barriers and solutions to including older people in research and the proposed theory of change model. Whilst the work focused on how to improve participation of older people in research, we also considered principles that may aid retention.

A multi-disciplinary core group (Goodwin, Quinn and Witham) met online in January 2021 and identified key stakeholders needed. The NIHR INCLUDE roadmap ( Figure 2 ) as an initial guide [ 13 ] to prompt discussion. Areas for particular focus were non-hospital and non-NHS settings, as well as intersectionality with other under-served groups. This group met throughout the project to synthesise discussions and contributions and to lead development of the recommendations.

An initial online stakeholder meeting was held (April 2021). The group discussed and agreed that:

emphasis of recommendations should be on study design and ethics. Regulatory issues would be considered but would not be a primary focus

cross-speciality research that is relevant to older people should be considered, not just ‘ageing’ research

Tools were needed to help researchers or signpost them to existing resources e.g. www.capacityconsent.com or www.trialforge.org

Prior to the next online meeting (June 2021), stakeholders were invited to identify barriers and solutions to including older people in research at key stages of the INCLUDE roadmap [ 13 ] and add to a Google Jamboard (real-time digital whiteboard). At the meeting, two parallel groups further discussed factors around inclusion and developed a Theory of Change model ( Figure 3 ) to describe inputs (what was needed), activities (what needed to be done), outputs (what we planned to produce) and outcomes in order to achieve impact . Further suggestions were added in the subsequent months via email or directly onto Jamboard.

In parallel, we conducted a broad rapid scoping search for published English language literature on inclusion of older people in research (March 2022). The aim was to supplement stakeholder activities and provide an overview of relevant literature in this area such as key barriers to including older people in research and potential strategies to improve recruitment and retention. PubMed (Medline), Google Scholar, ScienceDirect and the Age and Ageing journal were searched. We included ‘older adults’, ‘elderly’, ‘recruitment’, ‘retention’, ‘inclusion’ and ‘research’ as search terms using Boolean operators AND and OR.

We conducted the initial synthesis of findings from the literature review with outputs from the stakeholder discussions. Issues relating to the inclusion of older people in research are multi-faceted and complex, thus requiring a systems-level approach. We therefore mapped findings from the review and stakeholder discussions onto a socio-ecological model (SEM) [ 14 ]. The SEM ( Figure 4 ) enables mapping of multi-level and interacting factors to help understand how micro, meso and macro-level factors influence the research ecosystem. This approach has been used to better understand recruitment and retention in trials of health interventions [ 15–17 ]. From the descriptive synthesis we identified themes that were developed into initial recommendations.

Initial recommendations were presented to the NIHR CRN Ageing National Speciality Group (June 2022) and shared with the stakeholder group (July 2022). Both groups provided feedback and recommendations were revised accordingly.

Revised recommendations were shared with the groups described above (September–October 2022) along with a list of questions to assist researchers, reviewers and funders.

NIHR INCLUDE Roadmap (reproduced from Witham et al. [ 13 ] under CC-BY-4.0 licence).

Theory of Change for improving the inclusion of older people in research.

SEM representing the multi-level factors that may impact on the inclusion of older people in research.

A total of 14 recommendations are presented within the SEM and summarised in Table 1 . Below we provide rationale for each recommendation with supporting evidence. Examples of good practice and relevant resources are in Appendix 1 .

Summary of best practice recommendations for including older people in research.

Individual factors

Recommendation 1a: studies should not have upper age limits in inclusion criteria.

Some studies still explicitly exclude participants based on chronological age [ 4 , 7 ]. This approach is ageist and hence a morally indefensible form of discrimination. The heterogeneity of health and function in old age makes chronological age a weak proxy for other factors (e.g. multimorbidity, mobility) that may be more relevant to study design.

Recommendation 1B: Studies should be designed to be inclusive of those with cognitive impairment

Informed consent is a core tenet of research participation. However, blanket exclusion of those with cognitive impairment is discriminatory [ 18 , 19 ]. The Mental Capacity Act (2005) makes adequate provision for alternative means of participation using consultees [ 20 ]. Decisions about capacity to consent need to be taken on an individual basis, and such cut-offs impose a false dichotomy on the concept of capacity for research participation.

Recommendation 1C: Studies should not have comorbidity exclusions (except where there is an intervention contraindication)

Blanket exclusions because of comorbidity not only directly discriminate against participants with multiple conditions but constitute covert age discrimination [ 7 ]. Many older people who are the target treatment population for an intervention will have multiple conditions, and excluding such patients risks the production of biassed, unrepresentative research and therefore misleading results. This remains the case even for early phase studies where ‘clean designs’ to minimise heterogeneity may be sought.

Recommendation 1D: Use of clear explanations of why research is relevant to older people’s health and wellbeing

Research should be designed to maximise engagement of the entire population of interest. Some older people may have low interest due to a lack of perceived benefits or relevance of a study to themselves [ 18 ].

Some older people may have misconceptions about research, leading to distrust and unwillingness to engage. The issue of trust in health care providers and researchers has been highlighted particularly amongst older people from ethnic minority groups [ 15 ]. Engagement with people with varying cultural backgrounds who speak different languages presents further challenges which must be surmounted.

Interpersonal factors

Recommendation 2a: advocates and peers should be involved in designing research that meets the needs of older people.

Personal characteristics may impact on individuals’ perceived and actual ability to take part in research such as general poor health, multiple long-term conditions, fatigue or mobility issues [ 24 , 29 , 31 ]. This means older people may believe they are not able to take part or may have to withdraw from the study. Others may be reluctant to commit to taking part in research due to other priorities e.g. family responsibilities [ 7 , 17 ].

Recommendation 2B: Health and social care professionals with expertise in ageing should be involved in designing research that meets the needs of older people

Some researchers lack knowledge and understanding of ageing and involving older people in research resulting in inflexible study protocols that don’t accommodate age-related issues [ 32 ]. This can result in a general apprehension about recruiting older adults due to a higher likelihood of adverse events and attrition [ 24 , 27 ].

Recommendation 2C: Communication about research should be tailored to the needs of older people

How research is communicated can have a significant impact on engagement [ 25 , 27 , 31 ]. Some may have difficulty reading and understanding promotional materials to take part in research.

Recommendation 2D: All older people who meet the eligibility criteria should be offered the opportunity to participate in research

Gatekeeping by those involved in recruiting older people can occur based on a misconception that older adults are vulnerable, due to e.g. comorbidities or a lack of mental capacity, and therefore, need to be protected from research [ 8 ]. It is important that all those eligible to take part in research be offered the opportunity to make an informed decision themselves. Researchers/clinicians should facilitate access to research and not inhibit it.

Recommendation 2E: Consider the involvement of carers or family members and the impact involvement can have on them and the older person

Lack of support from carers or family members can influence the decision to participate [ 25–27 ] and involvement in research can be an additional carer burden. Building a rapport with carers and family members may be important to aid recruitment and retention [ 8 , 26 , 33 ]. For individuals with cognitive impairment, involving carers can aid consent processes, and provide reassurance and encouragement [ 27 , 32 , 33 ].

Organisational factors

Recommendation 3a: those commissioning, funding or approving research should ensure the inclusion of older people and intersectional factors such as demographics and health status have been considered.

Research tends to focus on individual condition processes due to fears that the effects of ageing would impair the ability to report on effects related to a single condition [ 32 ]. This focus on single condition at the expense of those living with multiple long-term conditions has resulted in research and guidelines that do not reflect the patients seen in clinical practice.

Recommendation 3B: Those commissioning, funding and approving research should include older people and those with expertise in ageing

Research funders, ethics committees, regulators, sponsors and Research and Development departments may have an overly risk averse approach to involving older people in research that fails to recognise the autonomy of older people [ 37 ]. There may also be apprehension to allow inclusion of those who are considered vulnerable. Reviewers and decision makers may lack an understanding of ageing, older people and those living with multiple long-term conditions resulting in ill-informed decision making.

Community factors

Recommendation 4a: researchers should consider different and flexible approaches to promote accessibility of the research e.g. financial support to attend research centres or offering home visits for data collection.

The inaccessibility of research facilities due to the need to travel long distances and/or a lack of available transport are significant barriers to older people taking part in research, especially for those with reduced mobility or fatigue [ 24 , 25 , 27 ] or where costs are prohibitive [ 25 , 27 ].

Recommendation 4B: Support, or alternatives should be offered for digital data collection and interventions

Whilst there is increasing interest in remote research delivery that may be beneficial for some e.g. those with mobility impairment, caution must be taken not to exacerbate inequalities through digital exclusion [ 38 ]. Some may not have access to suitable equipment, or the internet [ 38 ]. Older people with equipment may be hesitant due to lack confidence to use it or privacy concerns. Remote trial delivery requires infrastructure and resources [ 39 ] and whilst NIHR have an ambition to work towards hybrid and fully remote trials, consideration should be given to those who may be excluded.

Policy factors

Recommendation 5a: policymakers should consider the impact of policies on health inequalities in research participation.

Policy decisions, such as NHS England’s Internet First [ 40 ] and Digital First Primary Care [ 41 ], are biassed towards those who are digitally enabled. However, older people, and those with disabilities, whose first language is not English, on lower incomes and living in rural areas are considered more likely to be digitally excluded [ 42 ]. These types of healthcare directives can lead to unintended consequences and risk worsening of health inequalities,

The following questions may guide those developing, reviewing and funding research that is inclusive of older people:

Engagement : Are representative older people, their family members, advocates and those with experience of working with older people involved in designing the research?

Inclusive study design : Has consideration been given to where the research takes place, how researchers interact with older people and family members and what happens during interactions to maximise the accessibility of research to older people and family members? This may involve piloting study procedures to ensure they are feasible and acceptable.

Resources: Have adequate infrastructure and financial resources been costed into funding applications to maximise participation, minimise withdrawals and ensure recruitment targets are met? There is a need to recognise that studies involving older people and family members may require additional resources, such as staff time and travel to conduct face-to-face recruitment and data collection at the person’s home.

Communication: Have a range of approaches to communicating with older people and family members about research been considered in terms of promoting the study, recruitment, data collection and dissemination.

The exclusion of older people in research relevant to their health and care is not a new problem [ 43 , 44 ]. The aim of our project was to develop best practice recommendations to promote the inclusion of older people in research. These were developed iteratively by identifying barriers and potential solutions in collaboration with a diverse group of stakeholders and from existing evidence. We identified 14 recommendations and four questions for researchers, reviewers and funders to consider: engagement; inclusive study design; adequate resources; communication.

Whilst there are many well established public and patient involvement and engagement (PPIE) groups they tend to be white, middle-class, have background in health or research [ 45 ], and are already research active. Older people are the most heterogeneous of any age group [ 46 ] with a broad range of intersectional characteristics. Gaining wider perspectives from e.g. communities and community groups would give greater insight into what is important to them and how to overcome some of the perceived barriers. It is a myth that older people and those with complex needs are too risky to include in research. They can and should contribute to identifying important research questions, priority setting, designing studies, participation in studies and disseminating findings [ 47 , 48 ]. Troya and colleagues evaluated the impact of PPIE in the development of research on self-harm amongst older adults highlighting both the added value PPIE can bring alongside challenges such as support, training and ongoing communication that requires both time and resources [ 49 ].

Our findings align with those of the recently developed frameworks for promoting inclusion of people from ethnic minority groups [ 11 , 12 ] and those with impaired capacity to consent [ 10 , 22 ]. These frameworks consider four questions that should be asked by those developing RCTs: (i) who should trial apply to; (ii) are these groups likely to respond in different ways; (iii) will intervention make it harder for some to engage and (iv) will the study design make it harder for some to engage? There are some similarities to the key considerations we identified through the literature and stakeholders.

Whilst involving and engaging with older people living with complex needs can be logistically challenging efforts should be made to design research to be more inclusive. For example, a trial of home-based extended rehabilitation for older people living with frailty follow-up data collection is postal with the option for in-person data collection at the person’s home if needed [ 47 ]. Whilst having a more person-centred approach to recruitment, data collection and intervention delivery is clearly of benefit to the individual, this will come at a cost, both in staff time and resource [ 11 , 50 ].

Communication about research needs to take place before, during and after a study. In the USA, the National Institute on Ageing has developed a toolkit and resources to encourage older people to take part in research addressing concerns and worries that older people may have [ 23 ]. Researchers need to identify a range of different communication strategies that incorporate people’s preferences, cultural and communication needs. The push towards supporting everyone to be digitally included [ 41 ] may create barriers and can threaten some peoples’ autonomy. We must ensure that those that don’t want to or are unable to engage are offered alternatives.

Strengths and limitations of our work

Whilst we conducted a rapid search for existing publications on the inclusion of older people in research, the search strategy may not have been fully comprehensive, and we may have missed relevant studies, which could have been addressed by having broader search terms and by searching the Applied Social Sciences Index and Abstracts database. This said, by integrating the perspectives of a range of key stakeholders involved in research including public representatives, health and care professionals, charity representatives, academics and those involved in reviewing, funding and delivering research, we were able to identify different barriers and solutions that were not apparent in the peer reviewed publications.

Implications for research and practice

Our recommendations are new and simple to understand and provide practical guidance on ways to improve the inclusion of older people in health and care research. We next need to consider how best to implement change, in particular amongst those who are not specialists in conducting research with older people e.g. through the co-development of resources and training, and evaluate its impact on improving inclusion of older people in health and care research.

We have identified a range of factors that impact on the inclusion of older people in research and made recommendations of how these can be addressed in the future to ensure research findings are relevant to them.

We would like to thank Lesley Gibson, Cath Harris, Jim Harris, Rebecca Maier, Pauline McGlone, Gary Nestor, David Scott, Elizabeth Webb and the NIHR CRN Ageing National Specialty Group for their contributions to this work. This work was supported by the NIHR CRN and the Southwest Peninsula Applied Research Collaboration. The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

The data underlying this article are available in the article and in its online supplementary material.

National Institute for Health and Care Research , Improving Inclusion of under-Served Groups in Clinical Research: Guidance from INCLUDE Project . UK: NIHR, 2020 : https://www.nihr.ac.uk/documents/improving-inclusion-of-under-served-groups-in-clinical-research-guidance-from-include-project/25435 .

Sheehan KJ , Fitzgerald L , Hatherley S et al.  Inequity in rehabilitation interventions after hip fracture: a systematic review . Age Ageing 2019 ; 48 : 489 – 97 .

Google Scholar

Licchetta M and Stelmach M , Fiscal Sustainability and Public Spending on Health . UK: Office for Budget Responsibility, 2016 . https://obr.uk/docs/dlm_uploads/Health-FSAP.pdf .

Carvalho do Nascimento PR , Ferreira ML , Poitras S , Bilodeau M . Exclusion of older adults from ongoing clinical trials on low back pain: a review of the WHO trial registry database . J Am Geriatr Soc 2019 ; 67 : 603 – 8 .

Cruz-Jentoft AJ , Carpena-Ruiz M , Montero-Errasquín B , Sánchez-Castellano C , Sánchez-García E . Exclusion of older adults from ongoing clinical trials about type 2 diabetes mellitus . J Am Geriatr Soc 2013 ; 61 : 734 – 8 .

Hamaker ME , Stauder R , van Munster BC . Exclusion of older patients from ongoing clinical trials for hematological malignancies: an evaluation of the National Institutes of Health clinical trial registry . Oncologist 2014 ; 19 : 1069 – 75 .

Lockett J , Sauma S , Radziszewska B , Bernard MA . Adequacy of inclusion of older adults in NIH-funded phase III clinical trials . J Am Geriatr Soc 2019 ; 67 : 218 – 22 .

McMurdo ME , Roberts H , Parker S et al.  Improving recruitment of older people to research through good practice . Age Ageing 2011 ; 40 : 659 – 65 .

Richardson SJ , Carroll CB , Close J et al.  Research with older people in a world with COVID-19: identification of current and future priorities, challenges and opportunities . Age Ageing 2020 ; 49 : 901 – 6 .

Shepherd V , Hood K , Wood F . Unpacking the ‘black box of horrendousness’: a qualitative exploration of the barriers and facilitators to conducting trials involving adults lacking capacity to consent . Trials 2022 ; 23 : 471 .

Treweek S , Banister K , Bower P et al.  Developing the INCLUDE ethnicity framework-a tool to help trialists design trials that better reflect the communities they serve . Trials 2021 ; 22 : 337 . https://doi.org/10.1186/s13063-021-05276-8 .

Trial Forge . The INCLUDE Ethnicity Framework , 2022 . https://www.trialforge.org/trial-forge-centre/include/ .

Witham MD , Anderson E , Carroll C et al.  Developing a roadmap to improve trial delivery for under-served groups: results from a UK multi-stakeholder process . Trials 2020 ; 21 : 694 . https://doi.org/10.1186/s13063-020-04613-7 .

Green LW , Richard L , Potvin L . Ecological foundations of health promotion . Am J Health Promot 1996 ; 10 : 270 – 81 .

Moreno-John G , Gachie A , Fleming CM et al.  Ethnic minority older adults participating in clinical research . J Aging Health 2004 ; 16 : 93S – 123 .

Wells AA , Zebrack B . Psychosocial barriers contributing to the under-representation of racial/ethnic minorities in cancer clinical trials . Soc Work Health Care 2008 ; 46 : 1 – 14 .

Salihu HM , Wilson RE , King LM , Marty PJ , Whiteman VE . Socio-ecological model as a framework for overcoming barriers and challenges in randomized control trials in minority and underserved communities . Int J MCH AIDS 2015 ; 3 : 85 – 95 .

Taylor JS , DeMers SM , Vig EK , Borson S . The disappearing subject: exclusion of people with cognitive impairment and dementia from geriatrics research . J Am Geriatr Soc 2012 ; 60 : 413 – 9 .

Shepherd V , Wood F , Griffith R , Sheehan M , Hood K . Protection by exclusion? The (lack of) inclusion of adults who lack capacity to consent to research in clinical trials in the UK . Trials 2019 ; 20 : 474 . https://doi.org/10.1186/s13063-019-3603-1 .

Department of Health . The Mental Capacity Act . London : HMSO , 2005 .

Google Preview

Shepherd , V. and J. Davies , Conducting a Randomized Controlled Trial in Care Homes: The Challenges of Recruiting Residents Who Lack Capacity to Consent . 2020 : London : SAGE Publications Ltd . https://doi.org/10.4135/9781529726626 .

Shepherd V . Capacity and Consent to Research , 2022 . https://www.capacityconsentresearch.com/ .

National Institute on Aging . Recruiting older adults into research (ROAR) . Toolkit 2015 ; [cited 2022] . https://www.nia.nih.gov/health/recruiting-older-adults-research-roar-toolkit .

Davies K , Collerton JC , Jagger C et al.  Engaging the oldest old in research: lessons from the Newcastle 85+ study . BMC Geriatr 2010 ; 10 : 64 . https://doi.org/10.1186/1471-2318-10-64 .

Liljas AEM , Walters K , Jovicic A et al.  Strategies to improve engagement of 'hard to reach' older people in research on health promotion: a systematic review . BMC Public Health 2017 ; 17 : 349 . https://doi.org/10.1186/s12889-017-4241-8 .

Ridda I , MacIntyre CR , Lindley RI , Tan TC . Difficulties in recruiting older people in clinical trials: an examination of barriers and solutions . Vaccine 2010 ; 28 : 901 – 6 .

Witham MD , McMurdo MET . How to get older people included in clinical studies . Drugs Aging 2007 ; 24 : 187 – 96 .

Bonk J . A road map for the recruitment and retention of older adult participants for longitudinal studies . J Am Geriatr Soc 2010 ; 58 : S303 – 7 .

McDougall GJ Jr , Simpson G , Friend ML . Strategies for research recruitment and retention of older adults of racial and ethnic minorities . J Gerontol Nurs 2015 ; 41 : 14 – 23 .

Dawson S , Banister K , Biggs K et al.  Trial Forge guidance 3: randomised trials and how to recruit and retain individuals from ethnic minority groups—practical guidance to support better practice . Trials 2022 ; 23 : 672 . https://doi.org/10.1186/s13063-022-06553-w .

Provencher V , Mortenson WB , Tanguay-Garneau L , Bélanger K , Dagenais M . Challenges and strategies pertaining to recruitment and retention of frail elderly in research studies: a systematic review . Arch Gerontol Geriatr 2014 ; 59 : 18 – 24 .

Bowling CB , Whitson HE , Johnson TM 2nd . The 5Ts: preliminary development of a framework to support inclusion of older adults in research . J Am Geriatr Soc 2019 ; 67 : 342 – 6 .

Mody L , Miller DK , McGloin JM et al.  Recruitment and retention of older adults in aging research . J Am Geriatr Soc 2008 ; 56 : 2340 – 8 .

Fudge N , Wolfe CDA , McKevitt C . Involving older people in health research . Age Ageing 2007 ; 36 : 492 – 500 .

National Institute for Health and Care Research and NIHR Strategic Framework for Multiple Long-Term Conditions (Multimorbidity) MLTC-M Research . UK: NIHR, 2020 : https://www.nihr.ac.uk/documents/research-on-multiple-long-term-conditions-multimorbidity-mltc-m/24639 .

National Institute for Health and Care Research , Equality, Diversity and Inclusion Strategy 2022–2027 . UK: NIHR, 2022 : https://www.nihr.ac.uk/documents/equality-diversity-and-inclusion-strategy-2022-2027/31295 .

Nurock S . Patients may be less risk averse than committees . BMJ 2005 ; 330 : 471 – 2 .

Policy@Manchester , On Digital Inequalities: Analysis and Ideas on Addressing Digital Inequalities . 2021 , Manchester: University of Manchester : https://policyatmanchester.shorthandstories.com/on-digital-inequalities/ .

Government equalities office and equality and Humand rights commission , Equality Act 2010 : https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-021-05880-8 .

Digital NHS . Internet First Policy , 2020 . https://digital.nhs.uk/services/internet-first .

England NHS . Digital First Primary Care , 2019 . https://digital.nhs.uk/about-nhs-digital/corporate-information-anddocuments/digital-inclusion .

Digital NHS . Digital Inclusion for Health and Social Care . In: 2022 ; 01/4/22] .

Bartlett C , Doyal L , Ebrahim S et al.  The causes and effects of socio-demographic exclusions from clinical trials . Health Technol Assess 2005 ; 9 : iii-iv – x-x 1-152 . https://doi.org/10.3310/hta9380 .

Townsley CA , Selby R , Siu LL . Systematic review of barriers to the recruitment of older patients with cancer onto clinical trials . J Clin Oncol 2005 ; 23 : 3112 – 24 .

Healthtalk . Patient and Public Involvement in Research . 2017 ; [11/3/22]. https://healthtalk.org/patient-and-public-involvement-research/overview .

Dannefer D , Sell RR . Age structure, the life course and "aged heterogeneity": prospects for research and theory . Compr Gerontol B 1988 ; 2 : 1 – 10 .

Prescott M , Lilley-Kelly A , Cundill B et al.  Home-based extended rehabilitation for older people (HERO): study protocol for an individually randomised controlled multi-centre trial to determine the clinical and cost-effectiveness of a home-based exercise intervention for older people with frailty as extended rehabilitation following acute illness or injury, including embedded process evaluation . Trials 2021 ; 22 : 783 . https://doi.org/10.1186/s13063-021-05778-5 .

Mulrow CD , Gerety MB , Kanten D et al.  A randomized trial of physical rehabilitation for very frail nursing home residents . JAMA 1994 ; 271 : 519 – 24 .

Troya MI , Chew-Graham CA , Babatunde O , Bartlam B , Higginbottom A , Dikomitis L . Patient and public involvement and engagement in a doctoral research project exploring self-harm in older adults . Health Expect 2019 ; 22 : 617 – 31 .

McHenry JC , Insel KC , Einstein GO , Vidrine AN , Koerner KM , Morrow DG . Recruitment of older adults: success may be in the details . Gerontologist 2012 ; 55 : 845 – 53 .

• older adult

Supplementary data

Email alerts, citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1468-2834
• Copyright © 2024 British Geriatrics Society
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• NICE Guidance
• Health and social care delivery
• Adult's social care

Integrated health and social care for people experiencing homelessness

NICE guideline [NG214] Published: 16 March 2022

• Tools and resources
• Information for the public
• Recommendations

Recommendations for research

• Rationale and impact
• Finding more information and committee details

1 Psychologically informed environments

2 health and social care to support housing, 3 longer health and social care contacts.

The guideline committee has made the following recommendations for research.

What is the effectiveness and acceptability of clinical psychology-led psychologically informed environments and psychological approaches for people experiencing homelessness?

For a short explanation of why the committee made this recommendation for research, see the rationale section on general principles .

Full details of the evidence and the committee's discussion are in evidence reviews A and B: effectiveness of approaches to improve access to and engagement with health and social care and joined up approaches .

What structural and systems factors help or hinder commissioning and delivery of wraparound health and social care that is integrated with housing, for people experiencing homelessness?

For a short explanation of why the committee made this recommendation for research, see the rationale section on housing with health and social care support .

Full details of the evidence and the committee's discussion are in evidence review C: views and experiences of health and social care for people experiencing homelessness .

What is the effectiveness and cost effectiveness of longer health and social care contacts compared with usual care for people experiencing homelessness?

For a short explanation of why the committee made this recommendation for research, see the rationale section on planning and commissioning .

• Skip to main content
• Accessibility help

Information

We use cookies to collect anonymous data to help us improve your site browsing experience.

Click 'Accept all cookies' to agree to all cookies that collect anonymous data. To only allow the cookies that make the site work, click 'Use essential cookies only.' Visit 'Set cookie preferences' to control specific cookies.

Your cookie preferences have been saved. You can change your cookie settings at any time.

Health and social care - Planning with People: community engagement and participation guidance

Planning with People guidance sets out the responsibilities NHS boards, local authorities and Integration Joint Boards have to community engagement when health and social care services are being planned, or when changes to services are being considered and supports them to involve people meaningfully.

Part 1 – Planning with People

‘Planning with People – Community engagement and participation guidance’ (Planning with People) represents a new approach to engagement. It promotes consistency, culture change, and true collaboration, while encouraging creativity and innovation, based on best practice. It places people and communities at the centre of care service design and change, to deliver the best results. The guidance is co-owned by the Scottish Government and the Convention of Scottish Local Authorities ( COSLA ).

When ‘Planning with People’ was published in March 2020, it marked the first time in ten years since guidance on community engagement for healthcare was last issued by the Scottish Government. Much has changed since February 2010 – not least the integration of health and social care services and the coronavirus pandemic. The virus and the public health measures necessary to suppress it had, and continue to have, a substantial, wide ranging impact on our lives, and our public services [1] .

This edition of ‘Planning with People’ (February 2023) follows a review of the guidance to make sure it meets the needs of those for whom it is designed, and is aligned to the recommendations of The Independent Review of Adult Social Care in Scotland . ‘Planning with People’ has been updated following consultation and feedback from the wider Scottish public, individuals, organisations representing the equality sector, and health and social care ( HSC ) engagement professionals. A summary of the above consultations, including Healthcare Improvement Scotland – Community Engagement Citizens’ Panel Survey Nine, can be found in Annex A.

‘Planning with People’ also contains additional supporting information for public bodies, including the Quality Framework for Community Engagement, which was developed by Healthcare Improvement Scotland – Community Engagement and the Care Inspectorate. The guidance has been produced by people from right across the health and social care spectrum, and it will continue to develop as experience of collaborative community engagement grows.

NHS Boards and Integration Joint Boards have a statutory duty to involve people and communities in the planning and development of care services, and in decisions that will significantly affect how services are run.

‘Planning with People’ replaces previous guidance on engagement, and replaces the Chief Executive Letter 4 (2010) for NHS Boards.

The established major service change decision making process for NHS Boards remains unchanged.

‘Planning with People’ applies to all care services; for children, young people and adults. It should be followed not only by health and social care providers, but also by local, regional, and national planners, special health boards and all independent contractors and suppliers, such as care homes, pharmacies and general practices.

‘Planning with People’ sets out how members of the public can expect to be engaged by NHS Boards, Integration Joint Boards and Local Authorities. By recognising all the good work that is taking place, the guidance is designed to complement and strengthen organisations' existing engagement strategies. It also encourages close working between bodies to minimise duplication and share learning.

Reflecting the spirit of partnership, and to be inclusive of community members who might wish to refer to it, the language used in this guidance is deliberately accessible and ‘jargon-light’. Scotland's Health and Social Care Standards use 'care' to encompass both health and social care, so this terminology is used throughout.

The guidance was developed before, during, and after the coronavirus pandemic, which transformed methods of engagement. Digital approaches, including the use of social media, are acknowledged by ‘Planning with People’.

Healthcare Improvement Scotland – Community Engagement ( HIS - CE ) has a range of case studies to help illustrate best practice and capture impacts on communities and engaging organisations, Healthcare Improvement Scotland - Community Engagement, sharing-practice, case-studies

1.1 Defining community engagement

Effective services must be designed with and for people and communities – not delivered, top down for administrative convenience. [2] In order to be effective, community engagement must be relevant, meaningful and have a clearly defined focus. NHS Boards, Integration Joint Boards and Local Authorities should engage with the communities they serve, following the principles set out in the National Standards for Community Engagement .

This defines ‘community engagement’ as:

'A purposeful process that develops a working relationship between communities, community organisations and public and private bodies to help them to identify and act on community needs and ambitions. It involves respectful dialogue between everyone involved, aimed at improving understanding between them, and taking joint action to achieve positive change.'

1.2 Purpose of the guidance

Effective community engagement and the active participation of people is essential to ensure that Scotland's care services are fit for purpose and lead to better outcomes for people.

The Scottish Government and COSLA have developed ‘Planning with People’ to support greater collaboration between those making decisions about care services in Scotland, those delivering services, and people in communities who are affected. This guidance supports public service planners, commissioners and providers to consider how to continually improve the ways in which people and communities can become involved in developing services that meet their needs.

Supporting Leadership

The duty to involve people and communities in planning how their public services are provided is enshrined in law in Scotland.

Key statutory responsibilities, such as Joint Strategic Needs Assessment and Strategic Commissioning Planning must be fulfilled both in the letter and in the spirit of the legislation. Planning with People supports care organisations to meet their legal responsibilities.

‘Planning with People’ must be understood and adopted by all stakeholders, and there are key roles for NHS Chief Executives, Chief Officers in Integration Joint Boards, and Local Authority Chief Executives to ensure that engagement is undertaken effectively.

To achieve meaningful and effective engagement, leaders must demonstrate a commitment to it and take action to embed it within their organisations.

Organisational barriers that could hinder or impact negatively on engagement must be identified and addressed by effective leadership.

Engagement that takes place routinely helps to develop trust between communities and public bodies, fosters mutual understanding, and makes it easier to identify sustainable service improvements.

1.3 Policy and legislative context

‘Planning with People’ has been developed in response to the Ministerial Strategic Group for Health and Community Care Review of Progress with Integration of Health and Social Care , which urges an increase in the pace and effectiveness of integration across Scotland. That includes a proposal to develop revised guidance on local community engagement and participation based on existing good practice, to apply across health and social care bodies.

All relevant public bodies are expected to demonstrate how they are engaging with communities, and to evidence the impact of engagement.

This guidance takes account of relevant recent policy drivers and legislation. It promotes a shared understanding among Scotland's care planners and commissioners, to support consistently high-quality engagement with communities.

Statutory duties of community engagement

NHS Boards and Integrated Joint Boards have a statutory duty to involve people in the planning and development of services, and in the decision making process. This guidance supports care organisations to meet their legal responsibilities. It also supports delivery of obligations in respect of clinical and care governance. This includes NHS Boards duty of quality [3] and requirements of NHS Scotland in respect of clinical governance. [4] [5] [6]

NHS Boards are bound by duties of public involvement set out in the NHS (Scotland) Act 1978, Section 2B

Integration Joint Boards engagement and participation duties are specified by the Public Bodies (Joint Working) (Scotland) Act 2014 . Integration Joint Boards are expected to apply this guidance and work with colleagues in NHS Boards and Local Authorities to share learning and develop best practice.

The duty to involve people in the design and delivery of care services was strengthened with the introduction of the Community Empowerment (Scotland) Act 2015.

Participation is also a key element of a Human Rights based approach, which requires that people are supported to be active citizens and that they are involved in decisions that affect their lives.

Assurance, support and oversight

Healthcare Improvement Scotland – Community Engagement ( HIS - CE ), Healthcare Improvement Scotland - Community Engagement and the Care Inspectorate, Care Inspectorate have statutory responsibilities to assure and support improvement in the quality of care services. Where appropriate, they collaborate in the delivery of these duties.

Healthcare Improvement Scotland – Community Engagement ensures people and communities are engaged in shaping health and care services. It has a legal duty to support, ensure and monitor the discharge of health bodies’ duties in respect of public involvement, including quality assurance of changes being made by Integration Joint Boards and in primary and community health services. HIS - CE is overseen by the Scottish Health Council, a governance committee of the HIS Board. [7]

Healthcare Improvement Scotland – Community Engagement ( HIS - CE ) considers service change to be a service development or change in the way in which patients and service users access services. This may include the enhancement of a service through increased access, new resources and technologies, or new build facilities. It may also include the reduction, relocation or withdrawal of a service or the centralisation of specialist services. Some changes are made on a long-term or permanent basis while others are provided on a temporary basis.

The Care Inspectorate is responsible for inspecting and improving social care and social work services and regulates all registered services for adults and children.

Healthcare Improvement Scotland – Community Engagement ( HIS - CE ) and the Care Inspectorate, together with stakeholders, have developed the Quality Framework for Community Engagement (the Quality Framework). This supports NHS Boards, Local Authorities and Integration Joint Boards to carry out effective community engagement and demonstrate how these organisations are meeting their statutory responsibilities to engage. In addition, the Quality Framework will provide opportunities to develop practice and share learning.

Further details on the Quality Framework can be found in ‘Part 5 – Governance and decision-making’.

Joint Strategic Inspections

In partnership with other scrutiny bodies, Healthcare Improvement Scotland and the Care Inspectorate also carry out joint strategic inspections for care services of NHS Boards, Integration Joint Boards and Local Authorities. These inspections examine how integrated services are planned, commissioned and delivered to meet people's needs, and meaningful engagement is taken into account.

Healthcare Improvement Scotland and the Care Inspectorate both work to the Health and Social Care Standards in their scrutiny and improvement activities. The rights of people to be involved in decision-making regarding the provision of care underpin the joint standards, which also require people to be supported to participate fully.

Capital Investment Projects

Where capital is required, engagement with people and communities, as set out in this guidance, should be taken forward, in alignment with the Scottish Capital Investment Manual process.

Email: [email protected]

There is a problem

Thanks for your feedback

Your feedback helps us to improve this website. Do not give any personal information because we cannot reply to you directly.

PERSPECTIVE article

This article is part of the research topic.

15 Years of Frontiers in Human Neuroscience: Social Cognition and Discourse Processing

Transdiagnostic Considerations Are Critical to Understanding Childhood Neurodevelopmental Disorders Provisionally Accepted

• 1 University of Vermont, United States

The final, formatted version of the article will be published soon.

Growing dissatisfaction with the current categorical diagnostic systems has led to a movement toward transdiagnostic dimensional approaches to assessment of childhood mental health disorders. We argue that a transdiagnostic approach is especially important and appropriate when screening for neurodevelopmental disorders during early childhood. In the early childhood years, symptoms often appear in the form of developmental delays that could portend a variety of different disorders. Early intervention at this point is critical, even though a final endpoint disorder is not yet apparent. Intervening early has the potential to grow the area of weakness, possibly correcting or at least ameliorating these delays. Early intervention requires a multidisciplinary approach integrating efforts across settings and providers that monitor the development of young children. We argue here that young children's language ability is central to the development of social cognition, and a prerequisite for adequate social functioning. Social deficits are defining features of a subset of neurodevelopmental disorders such as autism spectrum disorder and social (pragmatic) communication disorder. Critically, impairment in social functioning is common in additional neurodevelopmental disorders such as attentiondeficit/hyperactivity disorder (ADHD), learning disorders, and even motor disorders. For this reason, we argue that, at the earliest sign of a possible neurodevelopmental disorder, children should be screened for language deficits prior to initiating a focused assessment for a specific type of neurodevelopmental disorder such as ADHD. Any detected language deficits should be considered in the design and implementation of the assessment, as well as the ultimate intervention plan.

Keywords: Transdiagnostic, Neurodevelopmental disorders, Early Childhood, assessment, screening, developmental delay, Language, social functioning

Received: 13 Feb 2024; Accepted: 19 Apr 2024.

Copyright: © 2024 Hoza and Shoulberg. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Mx. Betsy Hoza, University of Vermont, Burlington, United States

People also looked at

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Trop Med Infect Dis

The Importance of Understanding Social and Cultural Norms in Delivering Quality Health Care—A Personal Experience Commentary

The objectives of this paper are to provide a review of the author’s personal experiences working in culturally diverse environments and to emphasize the importance of recognizing the social determinants of health. While some determinants of health are modifiable others are not, in addition it is emphasized that cultural safety in delivering health care is crucial if services provided are to be appropriate and acceptable to health care seekers. Cultural sensitivity is needed if one is to make a change in health outcomes in culturally diverse environments. The development and delivery of culturally safe services is more acceptable to community members and is important if a difference is to be made in health inequities. Training in delivering culturally safe services should include both theoretical and practical components. Practical training should be conducted under supervision in remote settings so that trainees appreciate what their clients experience on a daily basis. Culturally “unsafe” clinical service has serious adverse effects. This commentary discusses the above factors and provides example cases from the author’s own career of where such factors have affected the health of individuals or groups.

Key Messages:

• There are a number of social determinants of health, some of these are modifiable while others such genetic make-up, gender and age are not.
• All health workers should receive education on the specific needs of their patients based on their culture, and, should understand that in a multi-cultural society, approaches to health delivery vary when dealing with patients differing cultures.
• In order for health care services to be effective they need to be both appropriate and acceptable and should be delivered by skilled and competent health care workers.
• Cultural awareness and cultural competency are required for cultural safety.
• Training for delivering culturally safe health care should include both theoretical and practical components.
• Not providing culturally safe services may result in serious adverse outcomes.

1. Introduction

In the early days of our own training, more than 50 years ago, we were made to learn of these factors. A number of texts by the very distinguished and respected Professor Michael Gelfand [ 1 , 2 , 3 ] were available. We were taught that “unless you approach your patients with understanding you will fail to win them over and as a result you will often be unable to cure them.” This is quoted in a review of the publication in the Journal of the American Medical Association in 1964 [ 4 ]. As medical students, we took part in the “Medical Anthropology” described in the book, and, as part of this we visited remote communities in Zimbabwe (then Rhodesia) and learnt how traditional practitioners delivered health care to clients that consulted with them for their needs. More recent publications are also available, and a paper by Professor John Goldsmid is referenced here [ 5 ]. Professor Goldsmid emphasizes that human behaviour and cultural practices can have a profound effect on the range and prevalence of diseases suffered by communities and that human behaviour is the forgotten factor in disease prevalence and transmission.

My training in medicine was in Zimbabwe. After graduation in 1969, I worked as an intern, senior house officer, and a general practitioner before specializing in internal medicine and sexual health. I worked as a medical officer in the Harare City Health Department where I introduced the concept of the syndromic management of sexually transmitted infections. After specializing I joined the Department of Medicine of the University of Zimbabwe Medical School where I was involved in teaching undergraduates and postgraduates and conducting research and other academic activities. During this time I also worked as a consultant for the World Health Organization and was able to visit numerous countries where I was involved in training doctors and health staff and develop and set up programs for the management and control of sexually transmitted infections (STIs) and Human Immunodeficiency Virus (HIV) infection. I became Professor of Medicine at the University of Zimbabwe College of Medicine and later became Dean of the College of Medicine. Following this I took up an appointment as Public Health Medical Officer with the Department of Health of the Northern Territory in Australia and was based in Alice Springs, developing and implementing STI control activities. This provided me the opportunity to work Indigenous Aboriginal Communities in the Northern Territory of Australia and to learn the customs and needs of the local communities. In 2009, I accepted the post of Medical Director of an Aboriginal Community Controlled Health Organization in the Northern Territory, providing comprehensive primary health care and population health.

In order to improve the health status of an individual, the health status of the population in which the individual lives and works needs to be improved. Population health is defined as health outcomes of a group of individuals. Population health focuses on specific populations looking at a broad range of factors that affect health. These factors are known as social determinants of health. The World Health Organization lists the following factors that influence the health and well-being of individuals [ 6 ]:

• Income and social status
• Level of education
• Physical environment—including safe housing, clean air, safe water, healthy workplaces and healthy communities
• Social support networks—including support from family, friends and community, and, effect of individual behaviour
• Culture, customs and traditions
• Genetics—having a predisposition for some diseases and abnormalities
• Access to health services
• Gender of the individual—some illnesses are more frequent in males than females and vice versa

It is important that all trainees in the health service delivery area are taught that the health of their clients is affected by these factors and that without addressing these aspects the client’s health may not improve or their problem will re-occur.

The importance of alternative medical systems cannot be over-emphasized. Depending on the region, country or continent, the public will seek care in alternative health systems. Hence the need to understand both what exists, and, what the health seeking behaviour of individuals is. In some countries in Southern Africa, the traditional healers play an important role in managing clients with chronic conditions and of conditions which suddenly develop without an obvious cause. An example of this is when the epidemic of HIV infection and AIDS started especially at the time (early in epidemic) when the cause of AIDS had not been established and no curative treatment was available. Illness may be “blamed” on misfortune or witchcraft and bad spirits, and hence clients would seek help in the alternative sector [ 7 ]. In many societies, the concept of disease origin is not completely understood by some of the populace, an example of this being the aetiology of disease [ 8 ]. The concept of the role of “invisible” pathogens and organisms, treatable or not, is not fully understood and requires a great deal of general education.

Example Case 1

Early during the course of the outbreak of HIV infection and AIDS in Zimbabwe, before the etiologic agent, Human T Cell Lymphotropic Virus Type 3 (HTLV3 later the nomenclature changed and the etiologic agent was called the human immunodeficiency virus (HIV) was identified, we were faced with patients presenting with symptoms and signs of AIDS-associated conditions such as generalized lymphadenopathy, wasting, opportunistic infections and generalized aggressive Kaposi Sarcoma. In informing patients that there was no known cause for the condition, we inadvertently encouraged patients to seek care in the alternate sector. Later, when the etiologic agent was identified and patients were told that there was no known cure for the infection made patients more steadfast in seeking the help of traditional practitioners known as Ngangas. The services of the traditional practitioners were acceptable to care seekers who felt assured that they could be cured as they had always consulted with them from their early childhood days. Unfortunately, the Ngangas did not have a cure for this devastating infection which took the lives of thousands of infected persons. In the early days of the HIV epidemic, we had extensive discussions with traditional healers in Zimbabwe and even encouraged them to take part in some well-designed research activities.

The determinants of health are numerous. Some of these are non-modifiable, while others may be altered to the beneficial effects of individuals and communities. Non-modifiable determinants include age, gender, genetic make-up and family history. These are ingrained factors and are not modifiable. Modifiable determinants of health include: education, employment, financial income and social status, physical environment both at work and at leisure, housing and social environment, healthy childhood development, culture and personal and community practices, and availability of health services and social support [ 9 , 10 , 11 ].

2. Population Health and Public Health

Population health focusses on specific communities and populations to determine factors that influence health. These factors include the social determinants of health listed above. While population health focuses on specific populations or communities to improve the social and economic well-being as a whole, public health is the organized effort to keep persons and communities healthy and disease-free and to prevent injury, illness and premature death [ 12 ]. Constituents of public health include health protection, health surveillance, disease prevention, injury prevention, and health promotion. Population health and public health are closely related and work together hand-in-hand.

In providing health care it is important to understand that health services should at all times be accessible as well as acceptable.

It should however be noted that health care systems alone cannot improve population health without reducing the population’s health and social inequities. A number of studies have shown that population health can improve if social needs are addressed and social conditions are improved [ 13 ].

The health of the individual and therefore of the community health is influenced by a number of complex factors that include: provision of acceptable, accessible and appropriate health care, individual and community health behaviours, physical and social environment, socioeconomic status, and public policy. These are all inter-related:

• Risky behaviours such as unhealthy diet and physical inactivity are linked with chronic diseases, but these risk factors depend on the community environment as people can only choose healthy options if these are available, and if the choices they make are safe.
• Socioeconomic status has a great influence on health as having the resources allows people to afford medical care, nutritious foods, and decent housing.

Public policy influences all of these aspects as it addresses education, employment and inequities, both nationally and locally, in both the private and public sectors.

In developing and providing health care services it is important to keep in mind a number factors including whether the service being provided is appropriate for the community receiving the service, whether the service is accessible, and finally whether the service is acceptable. These aspects need careful consideration and will determine whether the services will be used by the community. Considerations include opening hours for the service, gender distribution of staff providing services, distance of service from where people live, availability of transport, and costs for seeking the service.

3. Cultural Awareness

The health care provider’s cultural awareness is their understanding of the differences between themselves and people from other backgrounds, especially relating to differences in values and attitudes [ 14 ]. It is important to understand that a lack of awareness can lead to bad or poor decision-making and poor outcomes for persons that we are supposedly providing help. Cultural awareness helps to reduce the chances of making bad decisions and increases the chance of us making appropriate and acceptable decisions.

Training in Cultural Awareness

It is important that health care providers have an understanding of what may or may not be culturally acceptable to the clients that they are dealing with. A number of different approaches are available in delivering cultural awareness training. Training may be theoretical or practical, however, ideally both these components are necessary in order to ensure that the trainee is well-equipped in delivering high quality, acceptable care. Theoretical training may be used to sensitize trainees in being aware that their clients’ needs and beliefs may be different from what they themselves understand. It trains care providers in having an open mind and to ask their patients the right questions in the most acceptable way. Practical training is necessary to ensure that trainees consult with their patients and deliver care in an appropriate and acceptable way.

A number of training programs are available including face-to-face training, reading, and on-line courses. Some educational institutes have developed questionnaires that can be used when consulting with patients. Health care providers can ask a set of questions in order to determine the patients’ social status and figure out what the role of social determinants may play in their health. The American Academy of Family Practice has developed a screening tool to determine whether the patient has a need for social services such as housing, food security, transportation, and to determine the patient’s personal safety issues [ 15 ]. This short questionnaire may be used to determine whether the patient has modifiable social determinants of health. Training should aim to make health care providers competent in delivering culturally appropriate health care [ 16 ].

However, short training courses may not be sufficient for the delivery of culturally appropriate and culturally safe health care as shown in the Closing the Gap Report of The Australian Institute of Health and Welfare (AIHW) of the Australian Government published in 2015. This report states that cultural competency improves accessibility and effectiveness of healthcare for Indigenous people, however, in Australia, there is no coherent approach to embedding cultural competence in health services and that there is little evidence of how best to improve culturally competent healthcare delivery to Indigenous Australians [ 17 ].

Working in remote communities in the Northern Territory of Australia has been a great learning experience as well as an eye-opener. Firstly because of the remoteness of communities it is always difficult to recruit health staff. And secondly, health staff recruited tend to remain a short while. This requires that training needs to be repeated often. Staff obtained through recruitment agencies may come from all over Australia and even overseas. Recruitment agencies endeavor to train new recruits in cultural competency but of necessity the method of training is the short course type or even simply on-line. This may provide the trainee with an overview but does not provide them with a hands-on type experience. Hence, health organizations recruiting new staff need to develop their own cultural awareness and competency training. This is fine for long-term recruits but as often happens such recruits come in on short contracts. There is, therefore, a need for cultural awareness and competency training to take place during the undergraduate training of students in all disciplines of health.

Example Case 2

The lack of cultural awareness by health care providers can lead to non-acceptance of services provided. This is particularly important in providing sexual health care services. In this situation the gender of health care providers should match the gender of care seeker. When providing services for women it is important for the patient to be seen by a female care provider, and similarly when dealing with male clients the provider should be male. In my own experience both in Africa and in Australia, I found that it is better not to raise certain topics when managing patients of the opposite gender. In fact, I realized that some words are considered taboo when uttered by a male during a discussion with a female patient. Being aware of such sensitivity is important if trust is to be gained from your patients. In situations where it is necessary for the patient to be provided the information, I found that by bringing in another female staff member (with the permission of the patient) to talk to the patient in my absence was preferable and more acceptable to the patient. When I was in formal non-clinical meetings with senior executive staff, I was advised by female executive staff members that they would prefer to have discussions on sexual health topics with a female clinician.

Being culturally aware is to have the knowledge and being culturally competent is to practice in a culturally safe manner.

In order for health care services to be effective, health care that is provided needs to be acceptable to the people served. Health care services will be considered acceptable to people if they are delivered in a culturally appropriate manner. The provision of culturally appropriate care requires an understanding of the social life and customs of the population served. In Zimbabwe and many other developing countries in Africa the medical teaching curricula include theoretical and practical training in this field. Medical students are exposed to the structure and function of traditional healing process and to social and cultural norms by having to live and learn in remote communities during their undergraduate training. During such remote placements a group of students would live in the community for periods of 3–4 weeks with their medical school supervisors and become involved in dealing with medical problems that the community has to face. Depending on the stage of the students’ medical training, they are required to provide solutions to real-life or hypothetical problems that a community may face. This form of community orientated problem-based teaching takes place annually addressing different scenarios. In this way students learn to work with leaders and community decision makers including elders and traditional practitioners in dealing with major and minor issues that the community faces. The medical curriculum of the University of Zimbabwe College of Medicine is a comprehensive integrated curriculum which allows students to learn through problem solving exercises. Each year of their undergraduate training, students spend to three weeks in remote communities with supervisors and are exposed to local problems. They will observe and, where possible, assist the local community in dealing with existing problems and help identify causes of such problems as well as how to manage them locally. Medical students are placed in different communities in each of their placements.

With this approach students learn of the social, economic and physical environment that their patients come from and also get an understanding of the person’s individual characteristics and behaviours. Importantly, in addition the student is able to identify the social support networks available to patients locally. It is well understood that greater support from families, friends, and communities is linked to better health. Studying during remote placements allows students to learn the culture, customs, traditions, and the beliefs of the family and community, all of which affect health.

4. Cultural Safety

A number of different terms have been used to describe the provision of acceptable and appropriate care for persons belonging to cultures that differ from that of the care provider. While these are processes that lead to the provision of acceptable care, cultural safety is the outcome of these processes. With patient safety in mind, the concept of cultural safety is defined by the Australian Health Practitioner Regulation Authority (AHPRA) as the individual and institutional knowledge, skills, attitudes and competencies needed to deliver optimal health care for Aboriginal and Torres Strait Islander Peoples [ 18 ]. The Cultural Safety covers a large number of terms currently used interchangeably, including: cultural awareness, cultural competence, cultural capability and proficiency, cultural respect, cultural security, cultural appropriateness, cultural understanding, and cultural responsiveness. These are all important in leading to the main outcome of cultural safety. In its statement AHPRA states “that patient safety for Aboriginal and Torres Strait Islander Peoples is the norm and that patient safety includes the inextricably linked elements of clinical and cultural safety, and this link must be defined by Aboriginal and Torres Strait Islander Peoples”.

The importance of cultural safety cannot be over-emphasized especially since Australia is composed of a multi-cultural society in which everyone has the right to be treated with respect and dignity and this is crucial when delivering health care. Culturally safe health care delivery is crucial in attempting to close the gap in health outcomes in all communities.

Not providing a culturally safe service may lead to ill health such as low utilization of available services, non-compliance with referrals or prescribed interventions, reluctance in interacting with service providers, anger, and, dissatisfaction with tools and interventions used in the dominant culture [ 19 ].

In remote Aboriginal communities it is often stated that persons with chronic diseases are often not compliant with medicines prescribed for their illnesses. However, the reason for non-compliance may well be attributed to the lack of cultural awareness and competence of health practitioners, and this matter should be addressed.

5. Conclusions

Individual health is determined by a number of factors. While some of these factors are engrained in the individual and are not modifiable, the majority of determinants are modifiable. As health care providers it is our duty to identify the modifiable risk factors and initiate steps to modify these. Health is affected by the place the patient lives in, where they work, as well as where they are schooled. It is known that by eating well and staying active, not smoking, and seeking care when needed all influence health outcomes. Access to social and economic opportunities and availability of resources as well as availability of clean water and air also affect health and wellbeing [ 20 ]. Hence, addressing the social determinants health is important in achieving the goal of closing the gap in health inequities.

Training in cultural awareness and competency may be theoretical or practical. Theoretical training raises cultural awareness and provides guidance on its various different components and how to approach the topic. However, practical training provides the trainee with the experience necessary to provide culturally appropriate care and sensitizes the trainee on the multifactorial nature of the social determinants that the patient is surrounded by. Training in the classroom situation is not sufficient to reach the goals of provision of culturally appropriate care. By learning within the community, trainees are able to appreciate the fuller extent of the root causes of their patients’ ill health, and by understanding these, the care provider is able to “manage” their patient in a complete and comprehensible way. Cultural competency training should be a mandatory requirement for all trainees in every branch of the health field.

This research received no external funding.

Conflicts of Interest

The author declares no conflict of interest.