research paper on the importance of health

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Lippincott Open Access

Practice Full Report

Promoting health and well-being in healthy people 2030, associated data.

Supplemental Digital Content is Available in the Text.

Healthy People 2030 describes a vision and offers benchmarks that can be used to track progress toward the goal of all people in the United States achieving their full potential for health and well-being across the life span. This vision can be realized through evidence-based interventions and policies that address the economic, physical, and social environments in which people live, learn, work, and play. Securing health and well-being for all will benefit society as a whole. Gaining such benefits requires eliminating health disparities, achieving health equity, attaining health literacy, and strengthening the physical, social, and economic environments. Implementation of Healthy People 2030 will by strengthened by engaging users from many sectors and ensuring the effective use and alignment of resources. Promoting the nation's health and well-being is a shared responsibility—at the national, state, territorial, tribal, and community levels. It requires involving the public, private, and not-for-profit sectors.

Healthy People provides science-based national objectives with 10-year targets for improving the health of the nation. Healthy People 2030—the fifth edition of the Healthy People initiative—describes a vision and offers benchmarks that can be used to track progress toward the goal of helping all people in the United States achieve their full potential for health and well-being across the life span. Healthy People 2030 expresses an expanded focus on health and well-being and an understanding that health and well-being for all people is a shared responsibility. This vision can be achieved through evidence-based interventions and policies that address the economic, physical, and social environments in which people are born, live, learn, work, play, worship, and age. High-quality data that are accurate, timely, and accessible are required to record and report on progress 1 over the course of the decade and to direct interventions to populations that are most likely to benefit from them.

Healthy People sets the federal agenda for the nation's health, guides its direction and allocation of resources, informs federal data collection and programmatic activities, and provides a model for promoting health and well-being at the state and local levels. The initiative's emphasis on promoting health and well-being signals to the nation that it is time to work across sectors to achieve health equity. This decade Healthy People 2030 is a resource for all sectors.

As part of the development of Healthy People 2030, the US Department of Health and Human Services (HHS) sought guidance from the Secretary's Advisory Committee on National Health Promotion and Disease Prevention Objectives for 2030 (Secretary's Advisory Committee), a federal advisory committee composed of nonfederal, independent subject matter experts. The Secretary's Advisory Committee presented recommendations to the HHS Secretary for developing and implementing the objectives for 2030. The Secretary's Advisory Committee convened regularly between December 2016 and September 2019, with meetings open to the public.

Health promotion has been a cornerstone of the Healthy People initiative since its inception in 1979. The Secretary's Advisory Committee recommended that the focus of Healthy People 2030 expand beyond health promotion to the broader purpose of promoting “health and well-being.” The process that has been called health promotion no longer focuses on health alone, but now leads to health and well-being for individuals in addition to society as a whole. This offers a chance to balance the needs of individuals and society. Society is defined as “a voluntary association of individuals for common ends.” 2 Health and well-being are elements among the common ends that motivate us, as individuals, to act for the good of all. In return for participating in society, individuals expect fair and just opportunities to be as healthy and well as possible. This article provides insights into defining health and well-being, promoting health and well-being, fostering user collaboration to improve health and well-being, and measuring health and well-being, in addition to implications for policy and practice.

The Secretary's Advisory Committee produced 2 detailed briefs that offered guidance for promoting health and well-being. Secretary's Advisory Committee members, joined by additional subject matter experts, developed these 2 briefs. The original documents are available on the HealthyPeople.gov Web site. 3 , 4

Defining Health and Well-being

Healthy People 2030 refers to health and well-being in every aspect of the framework, including the vision, mission, foundational principles, plan of action, and overarching goals. 5 The expanded role for health and well-being in Healthy People 2030 was supported by the Secretary's Advisory Committee's recommendations and its definition of health and well-being as how people think, feel, and function—at a personal and social level—and how they evaluate their lives as a whole. 6 How people think, feel, and function affects their beliefs about whether their lives have meaning and purpose 7 , 8 (Table ​ (Table1). 1 ). This definition recognizes the multilevel nature of health and well-being. It acknowledges that social structures, such as families, neighborhoods, communities, organizations, institutions, policies, economies, societies, cultures, and physical environments, strongly influence health and well-being. Such influence is reciprocal between individual, social, and societal health and well-being. *

The terms “health” and “well-being” describe separate but related states; health influences well-being and, conversely, well-being affects health. 9 Health incorporates both physical and mental conditions; it implies fitness under changing circumstances, such as degradation of the physical, social, or economic environments, and must be safeguarded against threats from illness, injury, or death. Safety, as a result, is an important determinant of health. Well-being is both a determinant and an outcome of health. 10 It encompasses objective and subjective elements and reflects many aspects of life and states of being. These include physical and mental, as well as emotional, social, financial, occupational, intellectual, and spiritual, elements. 11 The terms apply to individuals as well as to groups of people (eg, families, communities) and environments (eg, physical, social, economic).

The World Health Organization defines health promotion as:

The process of enabling people to increase control over, and to improve, their health. 12 Health promotion ... covers a wide range of social and environmental interventions that are designed to benefit and protect individual people's health and quality of life by addressing and preventing the root causes of ill health, not just focusing on treatment and cure. 12

The World Health Organization identifies 3 key elements for health promotion: good governance for health; health literacy; and healthy cities. Adding the concept of well-being to this definition emphasizes that promotion of health and well-being takes place across different environments and users.

Promoting Health and Well-being

The concept of promoting health and well-being at both personal and systems levels has evolved over history, starting with ancient and classical civilizations. 13 Policy strategies for promoting health have been proposed since the 1970s. 14 More than 3 decades ago, the Ottawa Charter for Health Promotion described health as a “resource for everyday life, not the objective of living.” It noted that prerequisites for health include “peace, shelter, education, food, income, a stable ecosystem, sustainable resources, social justice, and equity.” 15 This guidance remains relevant today. Promoting well-being requires engaging an expanded and diverse array of users, disciplines, and sectors that extend beyond public health, such as mental health, housing, childcare/education, business, and aging.

Interventions to promote health and well-being occur at the individual, site-specific community, and societal levels. They address economic, social, and physical environmental and political factors (“determinants of health”) that influence health and well-being. Promoting health and well-being is critical because determinants of health—the physical, social, and economic circumstances in which people are born, live, learn, work, play, worship, and age—have disparate effects on vulnerable populations. These factors interact to affect people disproportionately based on race and class. All sectors are needed to remedy such disparities and achieve health equity.

At the individual level, interventions to promote health and well-being might focus on health behaviors, employment, housing, food security, or childcare. These interventions also would apply to the community level since they target settings where people spend their time, including home, school, work, or places where they socialize such as community centers and parks. These interventions can address designs of the built environment for ease of access and to ensure safety. The Robert Wood Johnson Foundation's Culture of Health initiative is one such national model. The Foundation defines a culture of health as one in which “good health and well-being flourish across geographic, demographic, and social sectors; fostering healthy equitable communities guides public and private decision making; and everyone has the opportunity to make choices that lead to healthy lifestyles.” 16

The concept of promoting health and well-being has evolved over the decades (Figure). Health and well-being operate on more than 1 level. Broader conditions shape individual experiences of health and well-being, and organized efforts can influence those conditions. Social structures, such as families, neighborhoods and communities, and policies, economies, and cultures also play important roles. 17 – 21

How the Concept of Health Promotion Has Evolved Over Decades of Healthy People

Engaging users from many sectors and ensuring the effective use and alignment of resources will strengthen implementation of Healthy People 2030. To promote health and well-being for all people and foster equity and social justice, socioecological factors and determinants of health must be addressed at all levels. A dynamic mix of resources will be needed for long-term improvements to livability (eg, stable housing, healthy food, clean air, education, living wage jobs) and for urgent needs (eg, acute care for illness or injury, food assistance, shelter, addiction treatment, disaster relief). Such resources will need to address a more diverse range of factors than in the past.

All too often, communities and institutions function in a reactive and responsive mode, deferring or delaying long-term investments. This way of functioning generates persistent needs for urgent services, along with pressure to maintain them. Collaborative decision-making across sectors can optimize the positive impact of resources and reduce the number of crises that happen in the first place. Identifying evidence-based programs to promote health and well-being among users can serve common interests, help users expand their thinking about solutions, and set priorities for limited time, money, and other scarce resources.

Multisectoral Collaborations to Improve Health and Well-being

Achieving population-level improvements in the coming decade will require users working at all levels to function across sectors and establish or participate in multisectoral collaborations. Such efforts can improve outcomes—not only in the health sector but also in nonpublic health or health care sectors, such as education, economics, the environment, and social cohesion. Collaboration among various users groups can benefit all partners by creating win-win solutions that recognize the interrelatedness of population health status with factors that lie outside the health care and public health systems.

Achieving optimal health and well-being requires efforts that include partners from different sectors, who operate at multiple levels (eg, state, local, community) and address the circumstances of people's lives. † Such efforts could span the behavioral, psychosocial, socioeconomic, cultural, and political circumstances of the population. No single actor has sole ownership of, accountability for, or capacity to sustain the health and well-being of an entire population. 22 – 24 The 10 “causes of the causes” of poor health comprise psychological influences (eg, social gradient, stress, and social exclusion), as well as elements of community infrastructure, such as food and transportation. 25 Thus, success depends on strengthening the capacity of communities to cocreate their own futures. 26

The COVID-19 pandemic is a case study of the reciprocal, complex relationships between the health of individuals and the health of society as a whole, as well as the resulting unintended consequences. An individual's decision not to wear a mask at a grocery store or other indoor gathering place can result in the virus' spread to other people who are present. Defining some workers as essential and required to work, such as those who work in grocery stores, transportation, health care, and in other occupations that require interaction with the public, increases the risk of infection for many low-wage earners. When essential workers are compensated with low wages, lack of financial viability creates challenges to their overall health and well-being. When health insurance is tied to employment and unemployment is soaring, unemployed people often delay seeking care. When older adults stay in isolation to avoid the possibility of infection, they can experience loneliness, depression, and mental health issues. When schools are closed and children stay at home, those who lack Internet connectivity are at risk of falling behind in their schoolwork. Those who receive free school lunches may go hungry.

To help local health departments identify strategies for promoting population health and well-being and addressing determinants of health, the National Association of County and City Health Officials (NACCHO) identified 9 domains of determinants, 27 as well as data sources for each (Table ​ (Table2). 2 ). Healthy People users at the state and tribal levels may find NACCHO's domains and data sources useful for identifying and acting upon opportunities to improve and monitor measures of health and well-being. These include indicators that are important to the success of other sectors, such as high school graduation, crime reduction, and economic prosperity.

Measuring Health and Well-being

Monitoring and documenting changes to the population's health and well-being will require the use of new data sources and types of measures. The way people evaluate their own lives as a whole is one indicator of health and well-being. Yet, systems that are outside of an individual's control shape the exposures, choices, and services that people experience. An important distinction exists between individuals' subjective ratings of their own health and well-being and the objective conditions that surround and support people as they strive to improve their health and well-being.

Measures of progress that go beyond those specific to public health and health care settings will require tapping into existing data sources across other domains and sectors. For example, data used by agricultural extension offices, planning departments at all levels, schools, businesses, parks and recreation agencies, transportation systems, the Bureau of the Census, aging services, and the financial sector, among others, can inform health and well-being. Data partnerships between public health, health care settings, and other sectors can often benefit collaborators by providing a much richer source of information for each partner as well as for the entire partnership. 28

Healthy People 2020 used functional measures, including Healthy Life Expectancy, ‡ Summary Mortality and Population Health, § and Disparities, as global health measures for assessing progress. Earlier iterations of Healthy People used life expectancy and other measures. ∥ Holistic evaluations of health and well-being status of individuals, communities, and systems require broad measures, such as life satisfaction or social cohesion. 29 – 33 Assessing progress toward improved health and well-being must consider health disparities, health literacy, multisectoral policies, and determinants of health and well-being.

Realizing the potential of Healthy People 2030 will require accurate data from credible sources at all levels, with a renewed emphasis on local action. There are barriers to generating high-quality data (eg, funding, staffing, technology). Healthy People supports local action by providing guidance for consistent data collection methods and measures, as well as examples of best practices and innovations. A data partnership infrastructure and network focused on Healthy People objectives could address and respond to new developments in data sources and data analytics. For example, a data partnership could expand the availability of locally relevant data, stimulate access to new data sources to measure determinants of health and health equity, and enable linkage of geographic and demographic data in presentation formats for Healthy People users.

Partners would be able to share data, methods, and analyses and access guidance on data developments relevant to all 3 Healthy People objective types—core, developmental, and research. A data partnership infrastructure and network that links national, tribal, state, territorial, and local data through partnerships and collaborations could enhance the nation's capacity to identify and record the achievement of Healthy People objectives and overarching goals.

Healthy People 2030 continues the Healthy People initiative's tradition of serving as a catalyst for action by expanding the focus of health promotion to promoting health and well-being (see Supplemental Digital Content file, available at http://links.lww.com/JPHMP/A716 ). This emphasizes the need to shift from a disease-specific orientation to more upstream policy efforts. Healthy People 2030 offers data, objectives, and tools for creating well-being and a healthier nation. Realizing the potential of Healthy People 2030 will require the active involvement of a variety of public and private institutions and organizations, including national, tribal, state, territorial, and local health departments. Health departments at all levels can contribute to this work by engaging multiple sectors in the implementation and monitoring of objectives.

Discussions within the public health community, and between public health and other sectors, around defining health and well-being offer opportunities to engage partners that historically have not been involved in Healthy People. Engaging new partners in the Healthy People initiative will require those who traditionally have led the initiative to understand what those partners need to succeed, communicate how new partners' goals complement those of Healthy People, and convey how engaging with Healthy People can benefit the new partners. For example, partnering to improve high school graduation rates benefits the education and public health sectors, as well as the financial sector and potentially the criminal justice system. Accomplishing that goal might involve engaging with the telecommunications sector to support students' access to affordable Internet service. By engaging in such partnerships, everyone would become more familiar with the goals of other sectors and discover more win-win opportunities.

In their health improvement plans, public health departments at all levels should think broadly about which partners from other sectors could help them advance health and well-being goals, while considering what public health can offer those sectors in achieving their own goals. For example, in Maryland, each county has been charged with having a local health improvement coalition that brings together key users to achieve locally identified needs for health and well-being and to eliminate health disparities. Organizations and individuals often need to see value for investing their time and resources before they agree to participate. Involving partners early allows them to be part of identifying issues and finding solutions.

Open access data portals at the state level are proliferating and can inform decision makers as well as the public. These data portals and related data dashboards provide community leaders and residents with current geographically tracked data and tools that support assessments and linkages to evidence-based interventions. These data initiatives offer yet another opportunity for partners to convene and develop collaborative programs for their respective populations.

One of Healthy People 2030's foundational principles is that “the health and well-being of all people and communities are essential to a thriving, equitable society.” Achieving health and well-being for all will benefit society as a whole. Achieving such benefits requires eliminating health disparities, achieving health equity, attaining health literacy, and strengthening the physical, social, and economic environments. Promoting the nation's health and well-being is a shared responsibility—at the national, state, territorial, tribal, and community levels. By enlisting the involvement of the public, private, and not-for-profit sectors in efforts to promote the health and well-being of our populations, we will improve the health of the nation and the achievement of Healthy People 2030's targets.

Implications for Policy & Practice

• Across the field of public health, the focus on health promotion should be expanded to include health and well-being.
• No one sector has the ability, responsibility, or needed expertise to promote health and well-being for all. Multisectoral approaches are needed to address the social, economic, and physical determinants of health and well-being.
• It will be critical to identify common data sources and indicators that can be used to measure and evaluate trends in health and well-being.

Supplementary Material

* Other definitions exist of the terms “health” and “well-being,” respectively. This is the definition proposed for Healthy People 2030, and it considers “health and well-being” as a single term.

† In the coming decade, Healthy People 2030 will highlight innovative and successful state- and local-level efforts through HealthyPeople.gov, webinars, and other channels.

‡ Healthy Life Expectancy (HLE) includes the following: HLE free from activity limitations at birth/age 65 years; HLE free from disability at birth/age 65 years; HLE in good or better health at birth/age 65 years.

§ Summary Mortality and Population Health includes the following: life expectancy at birth/age 65 years; any activity limitation at birth/age 65 years; any disability at birth/age 65 years; percentage in fair or poor health at birth/age 65 years.

∥ Healthy People 2010 used Life Expectancy, Healthy Life Expectancy, and Disparities. Healthy People 2000 used Years of Healthy Life; Disparities; and Clinical Preventive Services.

This article is based on 2 briefs that were prepared by the Secretary's Advisory Committee on National Health Promotion and Disease Prevention Objectives for 2030 and are available online at HealthyPeople.gov . The authors acknowledge and thank the following contributors to these original briefs: Tom Kottke, MD, MSPH; Bobby Milstein, PhD, MPH; Rebecca Rossom, MD, MSCR; Matt Stiefel, MPA, MS; and Elaine Auld, MPH, MCHES.

The authors declare no conflicts of interest.

Supplemental digital content is available for this article. Direct URL citation appears in the printed text and is provided in the HTML and PDF versions of this article on the journal's Web site ( http://www.JPHMP.com ).

Public health and research: an overview

Journal of Health Research

ISSN : 2586-940X

Article publication date: 15 October 2020

Issue publication date: 15 June 2021

This paper was to describe the overview of public health research.

Design/methodology/approach

It is a commentary piece of work from own long experience in working with the World Health Organization.

This study has innovative ideas in approaching priority areas in public health research.

Originality/value

This study opens up new thought in public health research.

• Public health

Public health research

Plianbangchang, S. (2021), "Public health and research: an overview", Journal of Health Research , Vol. 35 No. 4, pp. 374-378. https://doi.org/10.1108/JHR-03-2020-0074

Emerald Publishing Limited

Copyright © 2020, Samlee Plianbangchang

Published in Journal of Health Research . Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode

An overview of public health

Public health is the science and art of preventing disease, prolonging life and promoting human health through organized community efforts as well as the informed choice of society, public, private and voluntary organizations and communities at large. Analyzing the health of a population and the threats to that health forms the basis for public health.

Additionally, “health”, as defined in the WHO Constitution in 1948 [ 1 ], considers physical, mental and social well-being and is not merely the absence of disease or infirmity. Furthermore, public health is interdisciplinary, thus, a “public health team” may include several related disciplines in health and other social fields. Access to public health initiatives for comprehensive and integrated health care and services is always a difficult challenge in any population setting.

The objective of this commentary is to provide an overview of public health research and highlight innovative thinking in the field.

Public health systems

Public health systems are commonly defined as “all public, private, and voluntary entities that contribute to the delivery of essential public health services to people within a jurisdiction” [ 2 ]. This concept is to ensure, among other things, that all contributions to the improved health and well-being of the community or state are appropriately recognized and counted in the assessment of the provision of public health services to the community. The public health system includes public health agencies at state and local levels; all healthcare providers; public safety agencies; human services and charity organizations; education and youth development agencies; recreation and art-related organizations; economic and philanthropic agencies and environmental organizations.

The 10 essential public health services/functions

Monitoring community health situations to identify and solve health problems and prevent any health risks in the community;

Investigating and diagnosing specific health threats and health hazards in the community with the view to their early prevention;

Through modern technology in communication, informing, educating and empowering people in the community about health issues and challenges and their interventions;

Identifying/investigating and solving any problems of public health importance;

Through the full participation of people, developing policies and plans that support individual and combined health efforts in the community;

Ensuring effective enforcement of laws and regulations that protect environmental health and assure the safety of the population;

Linking people to needed personal health services and ensuring the provision of quality health care when otherwise unavailable;

Assuring the availability of effective public health workforce and competent healthcare personnel in both public and private facilities;

Objectively evaluating efficiency, effectiveness, accessibility and quality of personal as well as population-based health care facilities and services and

Undertaking study/research for new insights and innovative solutions to effectively counter prevailing and emerging health problems.

Public health rsearch aims to elucidate the influence of factors that determine the health of a population, i.e. genetic, environmental, social-cultural, economic, political, etc. The objective of public health research is to use the knowledge gained to propose policies and interventions, based on scientific evidence, and to help improve the health and well-being of the population and ultimately reduce/eradicate health inequalities.

Public health research is characterized by its multidisciplinary approach. It draws on several disciplines in its development and management, especially epidemiology/human ecology; biostatistics; physical and social sciences as applied to health; biology; genetics and toxicology. It usually entails the analysis of data on population samples on varying scales, depending on the scope and purpose of the research [ 4 ].

In practical terms, public health researchers study the statuses of population health and well-being, disability and loss of independence. They analyze the determining risk factors of these statuses or conditions, whether biological, behavioral, psycho-social or environmental. In addition, the researchers develop and assess the interventions aiming to effectively promote population health, prevent disease and compensate for disabilities and loss of independence. They also develop and assess innovation to improve efficiency and effectiveness of the organization of healthcare facilities and personnel, in public health, medical and other social service areas.

Research in health may be in the fields of health research, medical research, public health research, health services research, health systems research, environmental health research and others. These are interlinked and need to be pursued in a parallel manner in public health development.

Funding agencies

Mission and role of funding agencies

(2)Quality of research proposal

Relevance to current health-related issues and problems; responding to priority public health needs and the challenges of community and country are crucial. Applications should also be in the interest of the international/global health community.

Formulation with rational and logical thinking; Research proposals should be well-conceived and developed according to sound research methodology/protocol on epidemiological principles.

Expected result of the proposal; The result must be of high quality and should imply strong feasibility in its subsequent application/implementation in both technical, managerial and financial terms.

Public health system development research

DEIDS (development and evaluation of integrated health delivery system) (Thailand Lampang Health Development Project), 1973-1978 [ 6 ].

(2)District Health Services Development Project based on the principle of the primary health care approach, Mongar Health Development Project in Bhutan, 1984–1990 [ 7 ].

(3)Comprehensive and Integrated Health Research Development Project on hill- tribe population, 2014–present (still ongoing)

This is a long-term large research and development project with many sub-projects focusing on various specific issues of the target population. It is a multi-agency and interdisciplinary endeavor developed and implemented through the coordination of the Maefah Luang University Center of Excellence on health of the hill-tribe population. It includes the development of integrated and comprehensive social and health care services with an emphasis on equity, social justice and self-reliance in the community. There are multiple sources of funding, however mainly from the National Research Council of Thailand.

Some suggested areas for contemporary public health research

Universal health coverage for all

Financial sustainability in the long term

Equal accessibility by all people, regardless of their social and economic status

The right approach to development, proactive or passive strategy

Affordability in financial and managerial terms of the country concerned.

(2)Social impact of drug abuse /addiction

The social and economic impact of (chaotic situations created by) drug abuse and addiction

The general health of a population that is gradually undermined by drug abuse and drug addiction, leading, among other things to worsened national social and economic growth.

(3)Health and social care of elderly population

Intervention to reduce the degree of morbidity/disability and dependence

Efforts to increase social and economic productivity/independence

Program for self-help and self-reliance in an aged population

Preference between aging and aged programs, the difference between the two.

(4)Emerging infectious disease (EID)

The epidemiological, environmental and ecological approach in an integrated manner

The importance of social and behavioral change

Emerging mutation, assortment and genetic change in infectious agents

Impact of global climate change on EID, etc.

Even though it is mainly conceptual, the article may be able to help open new visions in public health research for better and sustainable health in the entire population in various localities.

1 World Health Organization [WHO] . WHO basic document . 47th ed. Geneva : WHO ; 2009 [cited 2019 Nov 28]. Available from: https://apps.who.int/iris/handle/10665/44192 .

2 World Health Organization [WHO] . What is health policy and systems research (HPSR)? [cited 2019 Nov 28]. Available from: https://www.who.int/alliance-hpsr/about/hpsr/en/ .

3 Centers for Disease Control and Prevention [CDC] , Office for state, tribal, local and territorial support . The 10 Essential Public Health Services: An Overview [cited 2019 Nov 28]. Available from: https://www.cdc.gov/publichealthgateway/publichealthservices/pdf/essential-phs.pdf .

4 Centers for Disease Control and Prevention [CDC] . Public health 101 series [cited 2019 Nov 28]. Available from: https://www.cdc.gov/publichealth101/ .

5 Centers for Disease Control and Prevention [CDC] . 24/7 Saving Lives, Protecting People [cited 2019 Nov 28]. Available from: https://www.cdcfoundation.org/cdc-247-saving-lives-protecting-people# .

6 Memoir on Occasion of the Death Anniversary of Dr. Somboon Vacharothai . Bangkok ; 2014 : 39 - 43 . (On Thailand DEIDS Project) .

7 World Health Organization [WHO] , Regional office for south-east asia [SEARO]. Sasakawa health prize: stories from south-east asia . New Delhi : WHO/SEARO ; 2012 : 125 - 146 [cited 2019 Nov 28]. Available from: https://apps.who.int/iris/bitstream/handle/10665/205878/B4905.pdf?sequence=1&isAllowed=y .

Corresponding author

Related articles, we’re listening — tell us what you think, something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

Loading metrics

Open Access

Peer-reviewed

Research Article

Assessing the impact of healthcare research: A systematic review of methodological frameworks

Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Validation, Writing – original draft, Writing – review & editing

Affiliation Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom

Roles Conceptualization, Formal analysis, Funding acquisition, Methodology, Project administration, Supervision, Validation, Writing – review & editing

* E-mail: [email protected]

Roles Data curation, Formal analysis, Methodology, Validation, Writing – review & editing

Roles Formal analysis, Methodology, Supervision, Validation, Writing – review & editing

• Samantha Cruz Rivera, 
• Derek G. Kyte, 
• Olalekan Lee Aiyegbusi, 
• Thomas J. Keeley, 
• Melanie J. Calvert

• Published: August 9, 2017
• https://doi.org/10.1371/journal.pmed.1002370
• Reader Comments

Increasingly, researchers need to demonstrate the impact of their research to their sponsors, funders, and fellow academics. However, the most appropriate way of measuring the impact of healthcare research is subject to debate. We aimed to identify the existing methodological frameworks used to measure healthcare research impact and to summarise the common themes and metrics in an impact matrix.

Methods and findings

Two independent investigators systematically searched the Medical Literature Analysis and Retrieval System Online (MEDLINE), the Excerpta Medica Database (EMBASE), the Cumulative Index to Nursing and Allied Health Literature (CINAHL+), the Health Management Information Consortium, and the Journal of Research Evaluation from inception until May 2017 for publications that presented a methodological framework for research impact. We then summarised the common concepts and themes across methodological frameworks and identified the metrics used to evaluate differing forms of impact. Twenty-four unique methodological frameworks were identified, addressing 5 broad categories of impact: (1) ‘primary research-related impact’, (2) ‘influence on policy making’, (3) ‘health and health systems impact’, (4) ‘health-related and societal impact’, and (5) ‘broader economic impact’. These categories were subdivided into 16 common impact subgroups. Authors of the included publications proposed 80 different metrics aimed at measuring impact in these areas. The main limitation of the study was the potential exclusion of relevant articles, as a consequence of the poor indexing of the databases searched.

Conclusions

The measurement of research impact is an essential exercise to help direct the allocation of limited research resources, to maximise research benefit, and to help minimise research waste. This review provides a collective summary of existing methodological frameworks for research impact, which funders may use to inform the measurement of research impact and researchers may use to inform study design decisions aimed at maximising the short-, medium-, and long-term impact of their research.

Author summary

Why was this study done.

• There is a growing interest in demonstrating the impact of research in order to minimise research waste, allocate resources efficiently, and maximise the benefit of research. However, there is no consensus on which is the most appropriate tool to measure the impact of research.
• To our knowledge, this review is the first to synthesise existing methodological frameworks for healthcare research impact, and the associated impact metrics by which various authors have proposed impact should be measured, into a unified matrix.

What did the researchers do and find?

• We conducted a systematic review identifying 24 existing methodological research impact frameworks.
• We scrutinised the sample, identifying and summarising 5 proposed impact categories, 16 impact subcategories, and over 80 metrics into an impact matrix and methodological framework.

What do these findings mean?

• This simplified consolidated methodological framework will help researchers to understand how a research study may give rise to differing forms of impact, as well as in what ways and at which time points these potential impacts might be measured.
• Incorporating these insights into the design of a study could enhance impact, optimizing the use of research resources.

Citation: Cruz Rivera S, Kyte DG, Aiyegbusi OL, Keeley TJ, Calvert MJ (2017) Assessing the impact of healthcare research: A systematic review of methodological frameworks. PLoS Med 14(8): e1002370. https://doi.org/10.1371/journal.pmed.1002370

Academic Editor: Mike Clarke, Queens University Belfast, UNITED KINGDOM

Received: February 28, 2017; Accepted: July 7, 2017; Published: August 9, 2017

Copyright: © 2017 Cruz Rivera et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and supporting files.

Funding: Funding was received from Consejo Nacional de Ciencia y Tecnología (CONACYT). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript ( http://www.conacyt.mx/ ).

Competing interests: I have read the journal's policy and the authors of this manuscript have the following competing interests: MJC has received consultancy fees from Astellas and Ferring pharma and travel fees from the European Society of Cardiology outside the submitted work. TJK is in full-time paid employment for PAREXEL International.

Abbreviations: AIHS, Alberta Innovates—Health Solutions; CAHS, Canadian Academy of Health Sciences; CIHR, Canadian Institutes of Health Research; CINAHL+, Cumulative Index to Nursing and Allied Health Literature; EMBASE, Excerpta Medica Database; ERA, Excellence in Research for Australia; HEFCE, Higher Education Funding Council for England; HMIC, Health Management Information Consortium; HTA, Health Technology Assessment; IOM, Impact Oriented Monitoring; MDG, Millennium Development Goal; NHS, National Health Service; MEDLINE, Medical Literature Analysis and Retrieval System Online; PHC RIS, Primary Health Care Research & Information Service; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; PROM, patient-reported outcome measures; QALY, quality-adjusted life year; R&D, research and development; RAE, Research Assessment Exercise; REF, Research Excellence Framework; RIF, Research Impact Framework; RQF, Research Quality Framework; SDG, Sustainable Development Goal; SIAMPI, Social Impact Assessment Methods for research and funding instruments through the study of Productive Interactions between science and society

Introduction

In 2010, approximately US$240 billion was invested in healthcare research worldwide [ 1 ]. Such research is utilised by policy makers, healthcare providers, and clinicians to make important evidence-based decisions aimed at maximising patient benefit, whilst ensuring that limited healthcare resources are used as efficiently as possible to facilitate effective and sustainable service delivery. It is therefore essential that this research is of high quality and that it is impactful—i.e., it delivers demonstrable benefits to society and the wider economy whilst minimising research waste [ 1 , 2 ]. Research impact can be defined as ‘any identifiable ‘benefit to, or positive influence on the economy, society, public policy or services, health, the environment, quality of life or academia’ (p. 26) [ 3 ].

There are many purported benefits associated with the measurement of research impact, including the ability to (1) assess the quality of the research and its subsequent benefits to society; (2) inform and influence optimal policy and funding allocation; (3) demonstrate accountability, the value of research in terms of efficiency and effectiveness to the government, stakeholders, and society; and (4) maximise impact through better understanding the concept and pathways to impact [ 4 – 7 ].

Measuring and monitoring the impact of healthcare research has become increasingly common in the United Kingdom [ 5 ], Australia [ 5 ], and Canada [ 8 ], as governments, organisations, and higher education institutions seek a framework to allocate funds to projects that are more likely to bring the most benefit to society and the economy [ 5 ]. For example, in the UK, the 2014 Research Excellence Framework (REF) has recently been used to assess the quality and impact of research in higher education institutions, through the assessment of impact cases studies and selected qualitative impact metrics [ 9 ]. This is the first initiative to allocate research funding based on the economic, societal, and cultural impact of research, although it should be noted that research impact only drives a proportion of this allocation (approximately 20%) [ 9 ].

In the UK REF, the measurement of research impact is seen as increasingly important. However, the impact element of the REF has been criticised in some quarters [ 10 , 11 ]. Critics deride the fact that REF impact is determined in a relatively simplistic way, utilising researcher-generated case studies, which commonly attempt to link a particular research outcome to an associated policy or health improvement despite the fact that the wider literature highlights great diversity in the way research impact may be demonstrated [ 12 , 13 ]. This led to the current debate about the optimal method of measuring impact in the future REF [ 10 , 14 ]. The Stern review suggested that research impact should not only focus on socioeconomic impact but should also include impact on government policy, public engagement, academic impacts outside the field, and teaching to showcase interdisciplinary collaborative impact [ 10 , 11 ]. The Higher Education Funding Council for England (HEFCE) has recently set out the proposals for the REF 2021 exercise, confirming that the measurement of such impact will continue to form an important part of the process [ 15 ].

With increasing pressure for healthcare research to lead to demonstrable health, economic, and societal impact, there is a need for researchers to understand existing methodological impact frameworks and the means by which impact may be quantified (i.e., impact metrics; see Box 1 , 'Definitions’) to better inform research activities and funding decisions. From a researcher’s perspective, understanding the optimal pathways to impact can help inform study design aimed at maximising the impact of the project. At the same time, funders need to understand which aspects of impact they should focus on when allocating awards so they can make the most of their investment and bring the greatest benefit to patients and society [ 2 , 4 , 5 , 16 , 17 ].

Box 1. Definitions

• Research impact: ‘any identifiable benefit to, or positive influence on, the economy, society, public policy or services, health, the environment, quality of life, or academia’ (p. 26) [ 3 ].
• Methodological framework: ‘a body of methods, rules and postulates employed by a particular procedure or set of procedures (i.e., framework characteristics and development)’ [ 18 ].
• Pathway: ‘a way of achieving a specified result; a course of action’ [ 19 ].
• Quantitative metrics: ‘a system or standard of [quantitative] measurement’ [ 20 ].
• Narrative metrics: ‘a spoken or written account of connected events; a story’ [ 21 ].

Whilst previous researchers have summarised existing methodological frameworks and impact case studies [ 4 , 22 – 27 ], they have not summarised the metrics for use by researchers, funders, and policy makers. The aim of this review was therefore to (1) identify the methodological frameworks used to measure healthcare research impact using systematic methods, (2) summarise common impact themes and metrics in an impact matrix, and (3) provide a simplified consolidated resource for use by funders, researchers, and policy makers.

Search strategy and selection criteria

Initially, a search strategy was developed to identify the available literature regarding the different methods to measure research impact. The following keywords: ‘Impact’, ‘Framework’, and ‘Research’, and their synonyms, were used during the search of the Medical Literature Analysis and Retrieval System Online (MEDLINE; Ovid) database, the Excerpta Medica Database (EMBASE), the Health Management Information Consortium (HMIC) database, and the Cumulative Index to Nursing and Allied Health Literature (CINAHL+) database (inception to May 2017; see S1 Appendix for the full search strategy). Additionally, the nonindexed Journal of Research Evaluation was hand searched during the same timeframe using the keyword ‘Impact’. Other relevant articles were identified through 3 Internet search engines (Google, Google Scholar, and Google Images) using the keywords ‘Impact’, ‘Framework’, and ‘Research’, with the first 50 results screened. Google Images was searched because different methodological frameworks are summarised in a single image and can easily be identified through this search engine. Finally, additional publications were sought through communication with experts.

Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (see S1 PRISMA Checklist ), 2 independent investigators systematically screened for publications describing, evaluating, or utilising a methodological research impact framework within the context of healthcare research [ 28 ]. Papers were eligible if they included full or partial methodological frameworks or pathways to research impact; both primary research and systematic reviews fitting these criteria were included. We included any methodological framework identified (original or modified versions) at the point of first occurrence. In addition, methodological frameworks were included if they were applicable to the healthcare discipline with no need of modification within their structure. We defined ‘methodological framework’ as ‘a body of methods, rules and postulates employed by a particular procedure or set of procedures (i.e., framework characteristics and development)’ [ 18 ], whereas we defined ‘pathway’ as ‘a way of achieving a specified result; a course of action’ [ 19 ]. Studies were excluded if they presented an existing (unmodified) methodological framework previously available elsewhere, did not explicitly describe a methodological framework but rather focused on a single metric (e.g., bibliometric analysis), focused on the impact or effectiveness of interventions rather than that of the research, or presented case study data only. There were no language restrictions.

Data screening

Records were downloaded into Endnote (version X7.3.1), and duplicates were removed. Two independent investigators (SCR and OLA) conducted all screening following a pilot aimed at refining the process. The records were screened by title and abstract before full-text articles of potentially eligible publications were retrieved for evaluation. A full-text screening identified the publications included for data extraction. Discrepancies were resolved through discussion, with the involvement of a third reviewer (MJC, DGK, and TJK) when necessary.

Data extraction and analysis

Data extraction occurred after the final selection of included articles. SCR and OLA independently extracted details of impact methodological frameworks, the country of origin, and the year of publication, as well as the source, the framework description, and the methodology used to develop the framework. Information regarding the methodology used to develop each methodological framework was also extracted from framework webpages where available. Investigators also extracted details regarding each framework’s impact categories and subgroups, along with their proposed time to impact (‘short-term’, ‘mid-term’, or ‘long-term’) and the details of any metrics that had been proposed to measure impact, which are depicted in an impact matrix. The structure of the matrix was informed by the work of M. Buxton and S. Hanney [ 2 ], P. Buykx et al. [ 5 ], S. Kuruvila et al. [ 29 ], and A. Weiss [ 30 ], with the intention of mapping metrics presented in previous methodological frameworks in a concise way. A consensus meeting with MJC, DGK, and TJK was held to solve disagreements and finalise the data extraction process.

Included studies

Our original search strategy identified 359 citations from MEDLINE (Ovid), EMBASE, CINAHL+, HMIC, and the Journal of Research Evaluation, and 101 citations were returned using other sources (Google, Google Images, Google Scholar, and expert communication) (see Fig 1 ) [ 28 ]. In total, we retrieved 54 full-text articles for review. At this stage, 39 articles were excluded, as they did not propose new or modified methodological frameworks. An additional 15 articles were included following the backward and forward citation method. A total of 31 relevant articles were included in the final analysis, of which 24 were articles presenting unique frameworks and the remaining 7 were systematic reviews [ 4 , 22 – 27 ]. The search strategy was rerun on 15 May 2017. A further 19 publications were screened, and 2 were taken forward to full-text screening but were ineligible for inclusion.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pmed.1002370.g001

Methodological framework characteristics

The characteristics of the 24 included methodological frameworks are summarised in Table 1 , 'Methodological framework characteristics’. Fourteen publications proposed academic-orientated frameworks, which focused on measuring academic, societal, economic, and cultural impact using narrative and quantitative metrics [ 2 , 3 , 5 , 8 , 29 , 31 – 39 ]. Five publications focused on assessing the impact of research by focusing on the interaction process between stakeholders and researchers (‘productive interactions’), which is a requirement to achieve research impact. This approach tries to address the issue of attributing research impact to metrics [ 7 , 40 – 43 ]. Two frameworks focused on the importance of partnerships between researchers and policy makers, as a core element to accomplish research impact [ 44 , 45 ]. An additional 2 frameworks focused on evaluating the pathways to impact, i.e., linking processes between research and impact [ 30 , 46 ]. One framework assessed the ability of health technology to influence efficiency of healthcare systems [ 47 ]. Eight frameworks were developed in the UK [ 2 , 3 , 29 , 37 , 39 , 42 , 43 , 45 ], 6 in Canada [ 8 , 33 , 34 , 44 , 46 , 47 ], 4 in Australia [ 5 , 31 , 35 , 38 ], 3 in the Netherlands [ 7 , 40 , 41 ], and 2 in the United States [ 30 , 36 ], with 1 model developed with input from various countries [ 32 ].

https://doi.org/10.1371/journal.pmed.1002370.t001

Methodological framework development

The included methodological frameworks varied in their development process, but there were some common approaches employed. Most included a literature review [ 2 , 5 , 7 , 8 , 31 , 33 , 36 , 37 , 40 – 46 ], although none of them used a recognised systematic method. Most also consulted with various stakeholders [ 3 , 8 , 29 , 31 , 33 , 35 – 38 , 43 , 44 , 46 , 47 ] but used differing methods to incorporate their views, including quantitative surveys [ 32 , 35 , 43 , 46 ], face-to-face interviews [ 7 , 29 , 33 , 35 , 37 , 42 , 43 ], telephone interviews [ 31 , 46 ], consultation [ 3 , 7 , 36 ], and focus groups [ 39 , 43 ]. A range of stakeholder groups were approached across the sample, including principal investigators [ 7 , 29 , 43 ], research end users [ 7 , 42 , 43 ], academics [ 3 , 8 , 39 , 40 , 43 , 46 ], award holders [ 43 ], experts [ 33 , 38 , 39 ], sponsors [ 33 , 39 ], project coordinators [ 32 , 42 ], and chief investigators [ 31 , 35 ]. However, some authors failed to identify the stakeholders involved in the development of their frameworks [ 2 , 5 , 34 , 41 , 45 ], making it difficult to assess their appropriateness. In addition, only 4 of the included papers reported using formal analytic methods to interpret stakeholder responses. These included the Canadian Academy of Health Sciences framework, which used conceptual cluster analysis [ 33 ]. The Research Contribution [ 42 ], Research Impact [ 29 ], and Primary Health Care & Information Service [ 31 ] used a thematic analysis approach. Finally, some authors went on to pilot their framework, which shaped refinements on the methodological frameworks until approval. Methods used to pilot the frameworks included a case study approach [ 2 , 3 , 30 , 32 , 33 , 36 , 40 , 42 , 44 , 45 ], contrasting results against available literature [ 29 ], the use of stakeholders’ feedback [ 7 ], and assessment tools [ 35 , 46 ].

Major impact categories

1. primary research-related impact..

A number of methodological frameworks advocated the evaluation of ‘research-related impact’. This encompassed content related to the generation of new knowledge, knowledge dissemination, capacity building, training, leadership, and the development of research networks. These outcomes were considered the direct or primary impacts of a research project, as these are often the first evidenced returns [ 30 , 62 ].

A number of subgroups were identified within this category, with frameworks supporting the collection of impact data across the following constructs: ‘research and innovation outcomes’; ‘dissemination and knowledge transfer’; ‘capacity building, training, and leadership’; and ‘academic collaborations, research networks, and data sharing’.

1 . 1 . Research and innovation outcomes . Twenty of the 24 frameworks advocated the evaluation of ‘research and innovation outcomes’ [ 2 , 3 , 5 , 7 , 8 , 29 – 39 , 41 , 43 , 44 , 46 ]. This subgroup included the following metrics: number of publications; number of peer-reviewed articles (including journal impact factor); citation rates; requests for reprints, number of reviews, and meta-analysis; and new or changes in existing products (interventions or technology), patents, and research. Additionally, some frameworks also sought to gather information regarding ‘methods/methodological contributions’. These advocated the collection of systematic reviews and appraisals in order to identify gaps in knowledge and determine whether the knowledge generated had been assessed before being put into practice [ 29 ].

1 . 2 . Dissemination and knowledge transfer . Nineteen of the 24 frameworks advocated the assessment of ‘dissemination and knowledge transfer’ [ 2 , 3 , 5 , 7 , 29 – 32 , 34 – 43 , 46 ]. This comprised collection of the following information: number of conferences, seminars, workshops, and presentations; teaching output (i.e., number of lectures given to disseminate the research findings); number of reads for published articles; article download rate and number of journal webpage visits; and citations rates in nonjournal media such as newspapers and mass and social media (i.e., Twitter and blogs). Furthermore, this impact subgroup considered the measurement of research uptake and translatability and the adoption of research findings in technological and clinical applications and by different fields. These can be measured through patents, clinical trials, and partnerships between industry and business, government and nongovernmental organisations, and university research units and researchers [ 29 ].

1 . 3 . Capacity building , training , and leadership . Fourteen of 24 frameworks suggested the evaluation of ‘capacity building, training, and leadership’ [ 2 , 3 , 5 , 8 , 29 , 31 – 35 , 39 – 41 , 43 ]. This involved collecting information regarding the number of doctoral and postdoctoral studentships (including those generated as a result of the research findings and those appointed to conduct the research), as well as the number of researchers and research-related staff involved in the research projects. In addition, authors advocated the collection of ‘leadership’ metrics, including the number of research projects managed and coordinated and the membership of boards and funding bodies, journal editorial boards, and advisory committees [ 29 ]. Additional metrics in this category included public recognition (number of fellowships and awards for significant research achievements), academic career advancement, and subsequent grants received. Lastly, the impact metric ‘research system management’ comprised the collection of information that can lead to preserving the health of the population, such as modifying research priorities, resource allocation strategies, and linking health research to other disciplines to maximise benefits [ 29 ].

1 . 4 . Academic collaborations , research networks , and data sharing . Lastly, 10 of the 24 frameworks advocated the collection of impact data regarding ‘academic collaborations (internal and external collaborations to complete a research project), research networks, and data sharing’ [ 2 , 3 , 5 , 7 , 29 , 34 , 37 , 39 , 41 , 43 ].

2. Influence on policy making.

Methodological frameworks addressing this major impact category focused on measurable improvements within a given knowledge base and on interactions between academics and policy makers, which may influence policy-making development and implementation. The returns generated in this impact category are generally considered as intermediate or midterm (1 to 3 years). These represent an important interim stage in the process towards the final expected impacts, such as quantifiable health improvements and economic benefits, without which policy change may not occur [ 30 , 62 ]. The following impact subgroups were identified within this category: ‘type and nature of policy impact’, ‘level of policy making’, and ‘policy networks’.

2 . 1 . Type and nature of policy impact . The most common impact subgroup, mentioned in 18 of the 24 frameworks, was ‘type and nature of policy impact’ [ 2 , 7 , 29 – 38 , 41 – 43 , 45 – 47 ]. Methodological frameworks addressing this subgroup stressed the importance of collecting information regarding the influence of research on policy (i.e., changes in practice or terminology). For instance, a project looking at trafficked adolescents and women (2003) influenced the WHO guidelines (2003) on ethics regarding this particular group [ 17 , 21 , 63 ].

2 . 2 . Level of policy impact . Thirteen of 24 frameworks addressed aspects surrounding the need to record the ‘level of policy impact’ (international, national, or local) and the organisations within a level that were influenced (local policy makers, clinical commissioning groups, and health and wellbeing trusts) [ 2 , 5 , 8 , 29 , 31 , 34 , 38 , 41 , 43 – 47 ]. Authors considered it important to measure the ‘level of policy impact’ to provide evidence of collaboration, coordination, and efficiency within health organisations and between researchers and health organisations [ 29 , 31 ].

2 . 3 . Policy networks . Five methodological frameworks highlighted the need to collect information regarding collaborative research with industry and staff movement between academia and industry [ 5 , 7 , 29 , 41 , 43 ]. A policy network emphasises the relationship between policy communities, researchers, and policy makers. This relationship can influence and lead to incremental changes in policy processes [ 62 ].

3. Health and health systems impact.

A number of methodological frameworks advocated the measurement of impacts on health and healthcare systems across the following impact subgroups: ‘quality of care and service delivering’, ‘evidence-based practice’, ‘improved information and health information management’, ‘cost containment and effectiveness’, ‘resource allocation’, and ‘health workforce’.

3 . 1 . Quality of care and service delivery . Twelve of the 24 frameworks highlighted the importance of evaluating ‘quality of care and service delivery’ [ 2 , 5 , 8 , 29 – 31 , 33 – 36 , 41 , 47 ]. There were a number of suggested metrics that could be potentially used for this purpose, including health outcomes such as quality-adjusted life years (QALYs), patient-reported outcome measures (PROMs), patient satisfaction and experience surveys, and qualitative data on waiting times and service accessibility.

3 . 2 . Evidence-based practice . ‘Evidence-based practice’, mentioned in 5 of the 24 frameworks, refers to making changes in clinical diagnosis, clinical practice, treatment decisions, or decision making based on research evidence [ 5 , 8 , 29 , 31 , 33 ]. The suggested metrics to demonstrate evidence-based practice were adoption of health technologies and research outcomes to improve the healthcare systems and inform policies and guidelines [ 29 ].

3 . 3 . Improved information and health information management . This impact subcategory, mentioned in 5 of the 24 frameworks, refers to the influence of research on the provision of health services and management of the health system to prevent additional costs [ 5 , 29 , 33 , 34 , 38 ]. Methodological frameworks advocated the collection of health system financial, nonfinancial (i.e., transport and sociopolitical implications), and insurance information in order to determine constraints within a health system.

3 . 4 . Cost containment and cost-effectiveness . Six of the 24 frameworks advocated the subcategory ‘cost containment and cost-effectiveness’ [ 2 , 5 , 8 , 17 , 33 , 36 ]. ‘Cost containment’ comprised the collection of information regarding how research has influenced the provision and management of health services and its implication in healthcare resource allocation and use [ 29 ]. ‘Cost-effectiveness’ refers to information concerning economic evaluations to assess improvements in effectiveness and health outcomes—for instance, the cost-effectiveness (cost and health outcome benefits) assessment of introducing a new health technology to replace an older one [ 29 , 31 , 64 ].

3 . 5 . Resource allocation . ‘Resource allocation’, mentioned in 6frameworks, can be measured through 2 impact metrics: new funding attributed to the intervention in question and equity while allocating resources, such as improved allocation of resources at an area level; better targeting, accessibility, and utilisation; and coverage of health services [ 2 , 5 , 29 , 31 , 45 , 47 ]. The allocation of resources and targeting can be measured through health services research reports, with the utilisation of health services measured by the probability of providing an intervention when needed, the probability of requiring it again in the future, and the probability of receiving an intervention based on previous experience [ 29 , 31 ].

3 . 6 . Health workforce . Lastly, ‘health workforce’, present in 3 methodological frameworks, refers to the reduction in the days of work lost because of a particular illness [ 2 , 5 , 31 ].

4. Health-related and societal impact.

Three subgroups were included in this category: ‘health literacy’; ‘health knowledge, attitudes, and behaviours’; and ‘improved social equity, inclusion, or cohesion’.

4 . 1 . Health knowledge , attitudes , and behaviours . Eight of the 24 frameworks suggested the assessment of ‘health knowledge, attitudes, behaviours, and outcomes’, which could be measured through the evaluation of levels of public engagement with science and research (e.g., National Health Service (NHS) Choices end-user visit rate) or by using focus groups to analyse changes in knowledge, attitudes, and behaviour among society [ 2 , 5 , 29 , 33 – 35 , 38 , 43 ].

4 . 2 . Improved equity , inclusion , or cohesion and human rights . Other methodological frameworks, 4 of the 24, suggested capturing improvements in equity, inclusion, or cohesion and human rights. Authors suggested these could be using a resource like the United Nations Millennium Development Goals (MDGs) (superseded by Sustainable Development Goals [SDGs] in 2015) and human rights [ 29 , 33 , 34 , 38 ]. For instance, a cluster-randomised controlled trial in Nepal, which had female participants, has demonstrated the reduction of neonatal mortality through the introduction of maternity health care, distribution of delivery kits, and home visits. This illustrates how research can target vulnerable and disadvantaged groups. Additionally, this research has been introduced by the World Health Organisation to achieve the MDG ‘improve maternal health’ [ 16 , 29 , 65 ].

4 . 3 . Health literacy . Some methodological frameworks, 3 of the 24, focused on tracking changes in the ability of patients to make informed healthcare decisions, reduce health risks, and improve quality of life, which were demonstrably linked to a particular programme of research [ 5 , 29 , 43 ]. For example, a systematic review showed that when HIV health literacy/knowledge is spread among people living with the condition, antiretroviral adherence and quality of life improve [ 66 ].

5. Broader economic impacts.

Some methodological frameworks, 9 of 24, included aspects related to the broader economic impacts of health research—for example, the economic benefits emerging from the commercialisation of research outputs [ 2 , 5 , 29 , 31 , 33 , 35 , 36 , 38 , 67 ]. Suggested metrics included the amount of funding for research and development (R&D) that was competitively awarded by the NHS, medical charities, and overseas companies. Additional metrics were income from intellectual property, spillover effects (any secondary benefit gained as a repercussion of investing directly in a primary activity, i.e., the social and economic returns of investing on R&D) [ 33 ], patents granted, licences awarded and brought to the market, the development and sales of spinout companies, research contracts, and income from industry.

The benefits contained within the categories ‘health and health systems impact’, ‘health-related and societal impact’, and ‘broader economic impacts’ are considered the expected and final returns of the resources allocated in healthcare research [ 30 , 62 ]. These benefits commonly arise in the long term, beyond 5 years according to some authors, but there was a recognition that this could differ depending on the project and its associated research area [ 4 ].

Data synthesis

Five major impact categories were identified across the 24 included methodological frameworks: (1) ‘primary research-related impact’, (2) ‘influence on policy making’, (3) ‘health and health systems impact’, (4) ‘health-related and societal impact’, and (5) ‘broader economic impact’. These major impact categories were further subdivided into 16 impact subgroups. The included publications proposed 80 different metrics to measure research impact. This impact typology synthesis is depicted in ‘the impact matrix’ ( Fig 2 and Fig 3 ).

CIHR, Canadian Institutes of Health Research; HTA, Health Technology Assessment; PHC RIS, Primary Health Care Research & Information Service; RAE, Research Assessment Exercise; RQF, Research Quality Framework.

https://doi.org/10.1371/journal.pmed.1002370.g002

AIHS, Alberta Innovates—Health Solutions; CAHS, Canadian Institutes of Health Research; IOM, Impact Oriented Monitoring; REF, Research Excellence Framework; SIAMPI, Social Impact Assessment Methods for research and funding instruments through the study of Productive Interactions between science and society.

https://doi.org/10.1371/journal.pmed.1002370.g003

Commonality and differences across frameworks

The ‘Research Impact Framework’ and the ‘Health Services Research Impact Framework’ were the models that encompassed the largest number of the metrics extracted. The most dominant methodological framework was the Payback Framework; 7 other methodological framework models used the Payback Framework as a starting point for development [ 8 , 29 , 31 – 35 ]. Additional methodological frameworks that were commonly incorporated into other tools included the CIHR framework, the CAHS model, the AIHS framework, and the Exchange model [ 8 , 33 , 34 , 44 ]. The capture of ‘research-related impact’ was the most widely advocated concept across methodological frameworks, illustrating the importance with which primary short-term impact outcomes were viewed by the included papers. Thus, measurement of impact via number of publications, citations, and peer-reviewed articles was the most common. ‘Influence on policy making’ was the predominant midterm impact category, specifically the subgroup ‘type and nature of policy impact’, in which frameworks advocated the measurement of (i) changes to legislation, regulations, and government policy; (ii) influence and involvement in decision-making processes; and (iii) changes to clinical or healthcare training, practice, or guidelines. Within more long-term impact measurement, the evaluations of changes in the ‘quality of care and service delivery’ were commonly advocated.

In light of the commonalities and differences among the methodological frameworks, the ‘pathways to research impact’ diagram ( Fig 4 ) was developed to provide researchers, funders, and policy makers a more comprehensive and exhaustive way to measure healthcare research impact. The diagram has the advantage of assorting all the impact metrics proposed by previous frameworks and grouping them into different impact subgroups and categories. Prospectively, this global picture will help researchers, funders, and policy makers plan strategies to achieve multiple pathways to impact before carrying the research out. The analysis of the data extraction and construction of the impact matrix led to the development of the ‘pathways to research impact’ diagram ( Fig 4 ). The diagram aims to provide an exhaustive and comprehensive way of tracing research impact by combining all the impact metrics presented by the different 24 frameworks, grouping those metrics into different impact subgroups, and grouping these into broader impact categories.

NHS, National Health Service; PROM, patient-reported outcome measure; QALY, quality-adjusted life year; R&D, research and development.

https://doi.org/10.1371/journal.pmed.1002370.g004

This review has summarised existing methodological impact frameworks together for the first time using systematic methods ( Fig 4 ). It allows researchers and funders to consider pathways to impact at the design stage of a study and to understand the elements and metrics that need to be considered to facilitate prospective assessment of impact. Users do not necessarily need to cover all the aspects of the methodological framework, as every research project can impact on different categories and subgroups. This review provides information that can assist researchers to better demonstrate impact, potentially increasing the likelihood of conducting impactful research and reducing research waste. Existing reviews have not presented a methodological framework that includes different pathways to impact, health impact categories, subgroups, and metrics in a single methodological framework.

Academic-orientated frameworks included in this review advocated the measurement of impact predominantly using so-called ‘quantitative’ metrics—for example, the number of peer-reviewed articles, journal impact factor, and citation rates. This may be because they are well-established measures, relatively easy to capture and objective, and are supported by research funding systems. However, these metrics primarily measure the dissemination of research finding rather than its impact [ 30 , 68 ]. Whilst it is true that wider dissemination, especially when delivered via world-leading international journals, may well lead eventually to changes in healthcare, this is by no means certain. For instance, case studies evaluated by Flinders University of Australia demonstrated that some research projects with non-peer-reviewed publications led to significant changes in health policy, whilst the studies with peer-reviewed publications did not result in any type of impact [ 68 ]. As a result, contemporary literature has tended to advocate the collection of information regarding a variety of different potential forms of impact alongside publication/citations metrics [ 2 , 3 , 5 , 7 , 8 , 29 – 47 ], as outlined in this review.

The 2014 REF exercise adjusted UK university research funding allocation based on evidence of the wider impact of research (through case narrative studies and quantitative metrics), rather than simply according to the quality of research [ 12 ]. The intention was to ensure funds were directed to high-quality research that could demonstrate actual realised benefit. The inclusion of a mixed-method approach to the measurement of impact in the REF (narrative and quantitative metrics) reflects a widespread belief—expressed by the majority of authors of the included methodological frameworks in the review—that individual quantitative impact metrics (e.g., number of citations and publications) do not necessary capture the complexity of the relationships involved in a research project and may exclude measurement of specific aspects of the research pathway [ 10 , 12 ].

Many of the frameworks included in this review advocated the collection of a range of academic, societal, economic, and cultural impact metrics; this is consistent with recent recommendations from the Stern review [ 10 ]. However, a number of these metrics encounter research ‘lag’: i.e., the time between the point at which the research is conducted and when the actual benefits arise [ 69 ]. For instance, some cardiovascular research has taken up to 25 years to generate impact [ 70 ]. Likewise, the impact may not arise exclusively from a single piece of research. Different processes (such as networking interactions and knowledge and research translation) and multiple individuals and organisations are often involved [ 4 , 71 ]. Therefore, attributing the contribution made by each of the different actors involved in the process can be a challenge [ 4 ]. An additional problem associated to attribution is the lack of evidence to link research and impact. The outcomes of research may emerge slowly and be absorbed gradually. Consequently, it is difficult to determine the influence of research in the development of a new policy, practice, or guidelines [ 4 , 23 ].

A further problem is that impact evaluation is conducted ‘ex post’, after the research has concluded. Collecting information retrospectively can be an issue, as the data required might not be available. ‘ex ante’ assessment is vital for funding allocation, as it is necessary to determine the potential forthcoming impact before research is carried out [ 69 ]. Additionally, ex ante evaluation of potential benefit can overcome the issues regarding identifying and capturing evidence, which can be used in the future [ 4 ]. In order to conduct ex ante evaluation of potential benefit, some authors suggest the early involvement of policy makers in a research project coupled with a well-designed strategy of dissemination [ 40 , 69 ].

Providing an alternate view, the authors of methodological frameworks such as the SIAMPI, Contribution Mapping, Research Contribution, and the Exchange model suggest that the problems of attribution are a consequence of assigning the impact of research to a particular impact metric [ 7 , 40 , 42 , 44 ]. To address these issues, these authors propose focusing on the contribution of research through assessing the processes and interactions between stakeholders and researchers, which arguably take into consideration all the processes and actors involved in a research project [ 7 , 40 , 42 , 43 ]. Additionally, contributions highlight the importance of the interactions between stakeholders and researchers from an early stage in the research process, leading to a successful ex ante and ex post evaluation by setting expected impacts and determining how the research outcomes have been utilised, respectively [ 7 , 40 , 42 , 43 ]. However, contribution metrics are generally harder to measure in comparison to academic-orientated indicators [ 72 ].

Currently, there is a debate surrounding the optimal methodological impact framework, and no tool has proven superior to another. The most appropriate methodological framework for a given study will likely depend on stakeholder needs, as each employs different methodologies to assess research impact [ 4 , 37 , 41 ]. This review allows researchers to select individual existing methodological framework components to create a bespoke tool with which to facilitate optimal study design and maximise the potential for impact depending on the characteristic of their study ( Fig 2 and Fig 3 ). For instance, if researchers are interested in assessing how influential their research is on policy making, perhaps considering a suite of the appropriate metrics drawn from multiple methodological frameworks may provide a more comprehensive method than adopting a single methodological framework. In addition, research teams may wish to use a multidimensional approach to methodological framework development, adopting existing narratives and quantitative metrics, as well as elements from contribution frameworks. This approach would arguably present a more comprehensive method of impact assessment; however, further research is warranted to determine its effectiveness [ 4 , 69 , 72 , 73 ].

Finally, it became clear during this review that the included methodological frameworks had been constructed using varied methodological processes. At present, there are no guidelines or consensus around the optimal pathway that should be followed to develop a robust methodological framework. The authors believe this is an area that should be addressed by the research community, to ensure future frameworks are developed using best-practice methodology.

For instance, the Payback Framework drew upon a literature review and was refined through a case study approach. Arguably, this approach could be considered inferior to other methods that involved extensive stakeholder involvement, such as the CIHR framework [ 8 ]. Nonetheless, 7 methodological frameworks were developed based upon the Payback Framework [ 8 , 29 , 31 – 35 ].

Limitations

The present review is the first to summarise systematically existing impact methodological frameworks and metrics. The main limitation is that 50% of the included publications were found through methods other than bibliographic databases searching, indicating poor indexing. Therefore, some relevant articles may not have been included in this review if they failed to indicate the inclusion of a methodological impact framework in their title/abstract. We did, however, make every effort to try to find these potentially hard-to-reach publications, e.g., through forwards/backwards citation searching, hand searching reference lists, and expert communication. Additionally, this review only extracted information regarding the methodology followed to develop each framework from the main publication source or framework webpage. Therefore, further evaluations may not have been included, as they are beyond the scope of the current paper. A further limitation was that although our search strategy did not include language restrictions, we did not specifically search non-English language databases. Thus, we may have failed to identify potentially relevant methodological frameworks that were developed in a non-English language setting.

In conclusion, the measurement of research impact is an essential exercise to help direct the allocation of limited research resources, to maximise benefit, and to help minimise research waste. This review provides a collective summary of existing methodological impact frameworks and metrics, which funders may use to inform the measurement of research impact and researchers may use to inform study design decisions aimed at maximising the short-, medium-, and long-term impact of their research.

Supporting information

S1 appendix. search strategy..

https://doi.org/10.1371/journal.pmed.1002370.s001

S1 PRISMA Checklist. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist.

https://doi.org/10.1371/journal.pmed.1002370.s002

Acknowledgments

We would also like to thank Mrs Susan Bayliss, Information Specialist, University of Birmingham, and Mrs Karen Biddle, Research Secretary, University of Birmingham.

• View Article
• PubMed/NCBI
• Google Scholar
• 3. HEFCE. REF 2014: Assessment framework and guidance on submissions 2011 [cited 2016 15 Feb]. Available from: http://www.ref.ac.uk/media/ref/content/pub/assessmentframeworkandguidanceonsubmissions/GOS%20including%20addendum.pdf .
• 8. Canadian Institutes of Health Research. Developing a CIHR framework to measure the impact of health research 2005 [cited 2016 26 Feb]. Available from: http://publications.gc.ca/collections/Collection/MR21-65-2005E.pdf .
• 9. HEFCE. HEFCE allocates £3.97 billion to universities and colleges in England for 2015–1 2015. Available from: http://www.hefce.ac.uk/news/newsarchive/2015/Name,103785,en.html .
• 10. Stern N. Building on Success and Learning from Experience—An Independent Review of the Research Excellence Framework 2016 [cited 2016 05 Aug]. Available from: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/541338/ind-16-9-ref-stern-review.pdf .
• 11. Matthews D. REF sceptic to lead review into research assessment: Times Higher Education; 2015 [cited 2016 21 Apr]. Available from: https://www.timeshighereducation.com/news/ref-sceptic-lead-review-research-assessment .
• 12. HEFCE. The Metric Tide—Report of the Independent Review of the Role of Metrics in Research Assessment and Management 2015 [cited 2016 11 Aug]. Available from: http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independentresearch/2015/The,Metric,Tide/2015_metric_tide.pdf .
• 14. LSE Public Policy Group. Maximizing the impacts of your research: A handbook for social scientists. http://www.lse.ac.uk/government/research/resgroups/LSEPublicPolicy/Docs/LSE_Impact_Handbook_April_2011.pdf . London: LSE; 2011.
• 15. HEFCE. Consultation on the second Research Excellence Framework. 2016.
• 18. Merriam-Webster Dictionary 2017. Available from: https://www.merriam-webster.com/dictionary/methodology .
• 19. Oxford Dictionaries—pathway 2016 [cited 2016 19 June]. Available from: http://www.oxforddictionaries.com/definition/english/pathway .
• 20. Oxford Dictionaries—metric 2016 [cited 2016 15 Sep]. Available from: https://en.oxforddictionaries.com/definition/metric .
• 21. WHO. WHO Ethical and Safety Guidelines for Interviewing Trafficked Women 2003 [cited 2016 29 July]. Available from: http://www.who.int/mip/2003/other_documents/en/Ethical_Safety-GWH.pdf .
• 31. Kalucy L, et al. Primary Health Care Research Impact Project: Final Report Stage 1 Adelaide: Primary Health Care Research & Information Service; 2007 [cited 2016 26 Feb]. Available from: http://www.phcris.org.au/phplib/filedownload.php?file=/elib/lib/downloaded_files/publications/pdfs/phcris_pub_3338.pdf .
• 33. Canadian Academy of Health Sciences. Making an impact—A preferred framework and indicators to measure returns on investment in health research 2009 [cited 2016 26 Feb]. Available from: http://www.cahs-acss.ca/wp-content/uploads/2011/09/ROI_FullReport.pdf .
• 39. HEFCE. RAE 2008—Guidance in submissions 2005 [cited 2016 15 Feb]. Available from: http://www.rae.ac.uk/pubs/2005/03/rae0305.pdf .
• 41. Royal Netherlands Academy of Arts and Sciences. The societal impact of applied health research—Towards a quality assessment system 2002 [cited 2016 29 Feb]. Available from: https://www.knaw.nl/en/news/publications/the-societal-impact-of-applied-health-research/@@download/pdf_file/20021098.pdf .
• 48. Weiss CH. Using social research in public policy making: Lexington Books; 1977.
• 50. Kogan M, Henkel M. Government and research: the Rothschild experiment in a government department: Heinemann Educational Books; 1983.
• 51. Thomas P. The Aims and Outcomes of Social Policy Research. Croom Helm; 1985.
• 52. Bulmer M. Social Science Research and Government: Comparative Essays on Britain and the United States: Cambridge University Press; 2010.
• 53. Booth T. Developing Policy Research. Aldershot, Gower1988.
• 55. Kalucy L, et al Exploring the impact of primary health care research Stage 2 Primary Health Care Research Impact Project Adelaide: Primary Health Care Research & Information Service (PHCRIS); 2009 [cited 2016 26 Feb]. Available from: http://www.phcris.org.au/phplib/filedownload.php?file=/elib/lib/downloaded_files/publications/pdfs/phcris_pub_8108.pdf .
• 56. CHSRF. Canadian Health Services Research Foundation 2000. Health Services Research and Evidence-based Decision Making [cited 2016 February]. Available from: http://www.cfhi-fcass.ca/migrated/pdf/mythbusters/EBDM_e.pdf .
• 58. W.K. Kellogg Foundation. Logic Model Development Guide 2004 [cited 2016 19 July]. Available from: http://www.smartgivers.org/uploads/logicmodelguidepdf.pdf .
• 59. United Way of America. Measuring Program Outcomes: A Practical Approach 1996 [cited 2016 19 July]. Available from: https://www.bttop.org/sites/default/files/public/W.K.%20Kellogg%20LogicModel.pdf .
• 60. Nutley S, Percy-Smith J and Solesbury W. Models of research impact: a cross sector review of literature and practice. London: Learning and Skills Research Centre 2003.
• 61. Spaapen J, van Drooge L. SIAMPI final report [cited 2017 Jan]. Available from: http://www.siampi.eu/Content/SIAMPI_Final%20report.pdf .
• 63. LSHTM. The Health Risks and Consequences of Trafficking in Women and Adolescents—Findings from a European Study 2003 [cited 2016 29 July]. Available from: http://www.oas.org/atip/global%20reports/zimmerman%20tip%20health.pdf .
• 70. Russell G. Response to second HEFCE consultation on the Research Excellence Framework 2009 [cited 2016 04 Apr]. Available from: http://russellgroup.ac.uk/media/5262/ref-consultation-response-final-dec09.pdf .

• Open access
• Published: 17 April 2017

Understanding relevance of health research: considerations in the context of research impact assessment

• Mark J. Dobrow 1 ,
• Fiona A. Miller 1 ,
• Cy Frank 2 &
• Adalsteinn D. Brown 1  

Health Research Policy and Systems volume  15 , Article number:  31 ( 2017 ) Cite this article

14k Accesses

24 Citations

9 Altmetric

Metrics details

With massive investment in health-related research, above and beyond investments in the management and delivery of healthcare and public health services, there has been increasing focus on the impact of health research to explore and explain the consequences of these investments and inform strategic planning. Relevance is reflected by increased attention to the usability and impact of health research, with research funders increasingly engaging in relevance assessment as an input to decision processes. Yet, it is unclear whether relevance is a synonym for or predictor of impact, a necessary condition or stage in achieving it, or a distinct aim of the research enterprise. The main aim of this paper is to improve our understanding of research relevance, with specific objectives to (1) unpack research relevance from both theoretical and practical perspectives, and (2) outline key considerations for its assessment.

Our approach involved the scholarly strategy of review and reflection. We prepared a draft paper based on an exploratory review of literature from various fields, and gained from detailed and insightful analysis and critique at a roundtable discussion with a group of key health research stakeholders. We also solicited review and feedback from a small sample of expert reviewers.

Conclusions

Research relevance seems increasingly important in justifying research investments and guiding strategic research planning. However, consideration of relevance has been largely tacit in the health research community, often depending on unexplained interpretations of value, fit and potential for impact. While research relevance seems a necessary condition for impact – a process or component of efforts to make rigorous research usable – ultimately, relevance stands apart from research impact. Careful and explicit consideration of research relevance is vital to gauge the overall value and impact of a wide range of individual and collective research efforts and investments. To improve understanding, this paper outlines four key considerations, including how research relevance assessments (1) orientate to, capture and compare research versus non-research sources, (2) consider both instrumental versus non-instrumental uses of research, (3) accommodate dynamic temporal-shifting perspectives on research, and (4) align with an intersubjective understanding of relevance.

Peer Review reports

Various levels of government in Canada collectively invest multiple billions of dollars in health-related research per annum, above and beyond investments in the management and delivery of healthcare and public health services. In recognition of this sizeable collective commitment, much work has focused on the impact of health research to explore and explain the consequences of these investments and inform strategic planning. Relevance is tacit in the increased attention to the usability and impact of health research. Additionally, research funders increasingly engage in relevance assessment as an input to decision processes; yet, it is unclear whether relevance is a synonym for or predictor of impact, a necessary condition or stage in achieving it, or a distinct aim of the research enterprise. Therefore, the main aim of this paper is to improve our understanding of research relevance as it relates to research quality and research impact, with specific objectives to (1) unpack research relevance from both theoretical and practical perspectives, and (2) outline key considerations for the assessment of research relevance.

Globally, there has been increasing critical assessment of the value of health research investments [ 1 – 3 ], with growing interest in research impact assessment (RIA) in the health sector [ 4 – 6 ]. RIA focuses on understanding how research activity can directly and indirectly advance knowledge, influence decision-making, and effect health and socio-economic outcomes, with a small but growing body of work seeking to develop better measures to evaluate (and ideally attribute) the returns on health research investments [ 6 ]. The Canadian Academy of Health Sciences (CAHS) released a comprehensive report on the subject in 2009 that presented a call for action, with a number of recommendations including establishing collaborative efforts among Canadian research funders to advance frameworks and sets of indicators and metrics for health research impact [ 4 ]. The CAHS impact framework [ 4 ], which drew on the Buxton and Hanney [ 7 ] ‘payback model’, among others, has provided a thoughtful starting point for considering the impact of health research in Canada. Subsequent work by Alberta Innovates – Health Solutions (AIHS) on a Research to Impact Framework (described in Graham et al. [ 8 ]) provides further insights on operationalising RIA frameworks for health research in Canada.

These initiatives are part of a broadly discussed shift in approaches to knowledge production, from an emphasis on investigator-initiated, curiosity driven work judged and guided by scientists, to expanded approaches to knowledge production, drawing on a wider set of actors and approaches, and emphasising relevance and usability. This shift from science produced by and for scientists to knowledge production that is “ socially distributed, application-oriented, trans-disciplinary, and subject to multiple accountabilities ” [ 9 ] has been characterised as a shift from ‘mode 1’ to ‘mode 2’ knowledge regimes. In the language of mode 2, interest in research ‘impact’ expresses a concern for application or consequence, and – in the economic language of return on investment – a concern that the yield is at least equal to the investment in the research itself. Extending this reasoning, interest in research ‘relevance’ may reflect a concern for accountability – linking research to the actor(s) for whom the research is performed and who will, ideally, put it to use.

In Canada, interest in research impact and relevance appears to have been felt most forcefully in the context of health services and policy research, which has long been encouraged to orient to the needs of policymakers, health system planners and related decision makers. More recently, there has been increased attention to ensuring that all forms of health research are ‘patient oriented’ – that is, that the research is prioritised, conducted and applied in ways that are accountable to this important end user. This call has been picked up on several fronts, including by the Canadian Institutes of Health Research (CIHR), which released its Strategy for Patient-Oriented Research (SPOR) in 2011. The SPOR vision “…is to demonstrably improve health outcomes and enhance patients’ health care experience through integration of evidence at all levels in the health care system ” [ 10 ]. In some respects, it represents a fundamental re-orientation for the primary funder of health research in Canada.

Though relevance is tacit in attention to research impact and the wider concern with mode 2 knowledge production, explicit attention to the meaning or measurement of research relevance is limited. The CAHS and AIHS frameworks, for example, acknowledge ‘relevance’ of health research but do not clearly define the term nor describe approaches for assessing it [ 4 , 8 ]. Rather, these frameworks emphasise the role of broad stakeholder engagement approaches and feedback mechanisms as methods for addressing relevance. For example, the AIHS framework notes the challenge of, and need to, move “ …beyond the collection of traditional scientific indicators […] to include measures of greater interest to the broader stakeholder community… ” [ 8 ] without stating explicitly how “ greater interest ” or related concepts such as relevance should be judged. As currently constructed, these RIA frameworks provide important advances in how we think about the impact of health research, but they were not intended to provide guidance specifically to the assessment of the relevance of health research.

Despite this lack of specific guidance on research relevance from a scholarly or measurement perspective, attention to it as a practical component of health research funding and organisation is evolving. There is, for example, growing use of ‘relevance assessment’ by research funders. The Canadian Health Services Research Foundation, in particular, was an innovator in incorporating relevance review into its applied research funding programmes, including promoting partnerships and knowledge translation (KT) with health system stakeholders [ 11 ]. Current applications for funding from the Institute of Gender and Health at CIHR go through ‘relevance review’ ( http://www.cihr-irsc.gc.ca/e/45212.html ). Similarly, applications for Ontario’s Health System Research Fund are judged based on ‘internal review of relevance and impact’ ( http://www.health.gov.on.ca/en/pro/ministry/research/cihr.aspx ). However, given the lack of conceptual clarity on research relevance, and in particular, how relevance assessment aligns with and differs from impact assessment, there is a critical gap in our understanding that has implications for both its contemporary and ongoing application and our ability to make sound research investment decisions.

This work was commissioned by the Ontario SPOR SUPPORT (Support for People and Patient-Oriented Research and Trials) Unit (OSSU) – one of several units established at provincial and regional levels across Canada to work with CIHR in pursuing the SPOR. Like other research organisations, OSSU saw the need to consider the relevance of the research it supported, and it established both scientific and relevance advisory committees as part of its original governance structure [ 12 ], tasking the latter to “ …develop a measure, or small set of strategic measures, that serves to inspire the Ontario research, implementation, provider and patient communities to come together to make a difference for patients ” [ 12 ]. In the spirit of research and scholarship, OSSU then asked what exactly this commitment to research ‘relevance’ entailed.

Our approach to answering this question involved the scholarly strategy of review and reflection. As with the early investigations into research impact assessment, we were surprised to find so little reflexive attention to the topic within the health research community [ 13 ]. We prepared a draft paper based on an exploratory review of literature from various fields, and gained from detailed and insightful analysis and critique at a roundtable discussion with a small group of key health research stakeholders. We also solicited review and feedback from a small sample of expert reviewers.

The structure of our paper is as follows. First, to ‘unpack’ the concept of relevance, we review theoretical literature and then consider practical work both from within and outside the health sector, to ask what has been argued and concluded about the nature of relevance and its appropriate assessment. Next, we outline a series of forward-looking considerations for assessing research relevance and conclude with reflections on how research relevance assessment fits with evolving interest in RIA.

Unpacking relevance

Theoretical perspectives.

Before considering the relevance of health research, we need to step back and consider what we mean by the term ‘relevance’. A range of descriptors is often used to define relevance, including ‘pertinent to…’, ‘bearing upon…’, ‘connected with…’, or ‘appropriate to…’, ‘…the matter at hand’, as well as ‘germane’, ‘apropos’, ‘material’, ‘applicable’ and ‘satisfactory’. A large body of dedicated theoretical work on relevance, drawn from many fields and perspectives, such as computer science, information science, statistics/probability theory, artificial intelligence, cognitive science, epistemology, linguistics and jurisprudence [ 14 ], reflects its importance but also the challenge for establishing a common understanding of the term [ 14 , 15 ]. For example, Gärdenfors [ 16 ], in his discussion on the logic of relevance, noted that “ …relevance ought to be a central concept in the philosophy of science… ” given the position that “ …it is only relevant information that is of any importance… ” (p. 351). However, from a ‘research’ relevance perspective, the theoretical work on relevance has been linked to ‘information’, ‘evidence’, ‘reasoning’, ‘argument’ and ‘decision’ [ 15 – 18 ], each presenting variable framing that impedes practical definition or consistent comprehension of the term. Floridi [ 14 ] recently suggested that existing theories are “ …utterly useless when it comes to establish the actual relevance of some specific piece of information ” (p. 69), and goes on to advance a ‘subjectivist’ interpretation, with relevance judged by the questioner. While a subjectivist approach to relevance is intuitively appealing, its contribution to the assessment of research relevance presents particular challenges that we will discuss later in the paper.

Another approach to unpacking relevance is to consider the theoretical model behind the broad-based research strategies that have governed research investments and policies in high-income countries since the end of the Second World War. For the better part of the 20th century, a linear model was the dominant conceptual framework, whereby basic research was viewed as a necessary input for applied research, which then led to development and production [ 19 , 20 ]. In the late 1990s, an alternate thesis was introduced when Stokes proposed a new model for broad-based research strategy – known as Pasteur’s Quadrant – that highlighted the conceptual relationship between the ‘quest to understand’ and ‘practical needs’ [ 21 ]. While some research is clearly focused on advances in basic research (e.g. Niels Bohr’s foundational research on atomic structure and quantum theory), and some research is clearly focused on applied problems (e.g. Thomas Edison’s practical inventions), Stokes emphasised the potential for use-inspired basic research (e.g. Louis Pasteur’s foundational research on microbiology that addressed contemporaneous population health challenges). Pasteur’s Quadrant invokes consideration of ‘relevance’ with some commentators framing the two-by-two relationship as the relevance for advancement of basic knowledge and the relevance for immediate application [ 22 ]. Stokes’ model adds conceptual insight on the role of relevance when considering the value of research to society, however, it was not intended to specifically conceptualise the term and does not distinguish it from other related concepts such as research impact or value. Therefore, to provide further insights, we next consider relevance in practical settings.

Health sector perspectives

In the health sector, the idea that research should be ‘relevant’ is commonplace. Commitments to ‘knowledge translation’ and the ‘knowledge to action cycle’ [ 23 ] emphasise issues of relevance and provide considerable insight into approaches to ensuring research usability and use. At the same time, the health research community has given disproportionate attention to issues of research quality, with an emphasis on internal validity that may downplay external validity and suggest some tension between rigour and relevance. Thus, though the concept of relevance is of central importance to the health research enterprise, the failure to unpack it or explore it both theoretically and practically leaves room for misunderstanding and misapplication.

In the health sector, research relevance often arises as a practical question of the ‘fit’ between a body of knowledge or research approach and a specific field or issue (e.g. public health, primary healthcare, healthcare access, genomics, alternative healthcare, healthcare reform in rural areas). The results of two recent International Society for Pharmacoeconomics and Outcomes Research task forces take this approach. The task forces developed questionnaires to assess the relevance and credibility of research other than randomised controlled trials (e.g. observational research, meta-network analysis) to inform healthcare decision-making [ 24 , 25 ]. Both make similar observations about relevance, reinforcing the subjectivist approach noted earlier, and can be summarised by the following statement by Berger et al. [ 24 ]:

“ Relevance addresses whether the results of the study/apply [sic] to the setting of interest to the decision maker. It addresses issues of external validity similar to the population, interventions, comparators, outcomes, and setting framework from evidence based medicine. There is no correct answer for relevance. Relevance is determined by each decision maker, and the relevance assessment determined by one decision maker will not necessarily apply to other decision makers. Individual studies may be designed with the perspective of particular decision makers in mind (e.g. payer or provider) ” (p.148, emphasis added).

Research relevance in health is also noted in discussion and debate regarding the value of qualitative research relative to the more established forms of quantitative health research. For example, Mays and Pope [ 26 ] suggest that qualitative research can be assessed “… by two broad criteria: validity and relevance ”. Their further discussion provides some insight into the several ways that research might be relevant, suggesting that:

“[r] esearch can be relevant when it either adds to knowledge or increases the confidence with which existing knowledge is regarded. Another important dimension of relevance is the extent to which findings can be generalised beyond the setting in which they were generated ” [ 27 ].

The work of Mays and Pope positions research relevance amidst the longstanding tension between internal and external validity. This tension reflects opposing foci on internal validity as the quality/rigour of research methodology and external validity as the applicability/transferability of research to other settings or contexts. While external validity is not the only measure of relevance – as research may remain relevant to some contexts even when not generalisable to others – it is an important component, and one that has not always attracted sufficient attention. For example, the Canadian health research community has focused considerable practical attention on internal validity as a critical component of evidence for clinical and health policy decisions. Evidence-based medicine, the Cochrane Collaboration, the Canadian and United States task forces on preventive healthcare/services and a long list of aligned groups have developed and established many tools to assess the quality of research evidence (e.g. GRADE [ 28 ]), with a predominant focus on issues of internal validity, and an emphasis on evidence hierarchies that is sometimes seen to be incompatible with ‘real world’ relevance. The relative lack of similar approaches or tools that focus on external validity in health research is notable, though movements to marshal evidence in support of sound public policy, such as the Campbell Collaboration, have attended to issues of external validity in other areas of health and social policy [ 29 ]. Further, there are emerging approaches and tools for documenting the external validity of health research and facilitating its use [ 30 ]. For example, WHO has supported the development of workbooks to contextualise health systems guidance for different contexts [ 31 ] and the field of local applicability and transferability of research has emerged to facilitate the adaptation of interventions from one setting to another, including the development of some well-documented tools like RE-AIM [ 32 ].

Alongside these emerging approaches and tools sits the established field of KT. KT has a strong history in Canada with a distinctive feature being a reliance on stakeholder engagement to support a commitment to improve research relevance. For example, the AIHS framework relies heavily on KT and stakeholder engagement approaches as part of its RIA, describing the mobilisation of knowledge through “ …a process of interactions, feedback, and engagement using a variety of mechanisms (e.g. collaborations, partnerships, networks, knowledge brokering) with relevant target audiences (i.e. actors and performers) across the health sector ” ([ 8 ] p. 362). Experience in stakeholder engagement, particularly with clinical, management and policy decision-makers, has become fairly extensive and there is now increased attention on engaging patients as core stakeholders in health research. If relevance is truly subjective, then KT efforts (including engagement, dissemination, promotion, communication) would appear to represent reasonable approaches for articulating, conveying and improving research relevance. However, if there are underlying elements of relevance that are more universal, then there is a risk that KT efforts – and subjectivist approaches to ensuring relevance – are akin to commercial marketing or communication strategies where the aim is to ‘sell’ more product and/or generate more influence that may not align with a more objective lens.

In sum, the health research community in Canada has a longstanding history of critically appraising research quality based on study design and research methodology, with greater emphasis on internal rather than external validity. As the same time, there is established expertise in KT, emphasising engagement with research users and adaptation to settings or contexts of use – approaches that may imply a subjectivist interpretation of relevance. Thus, while relevance is an important concept for the health research enterprise, its use is largely tacit and taken for granted.

Non-health sector perspectives

To unpack relevance further we consider some non-health sector perspectives that give attention to the term, often with formal definitions or taxonomies established. Examples include the legal, financial accounting, education and web search (information retrieval) sectors, each of which are briefly described below.

From a legal perspective, relevance has a specific meaning that relates to the admissibility of evidence in terms of its probative value (i.e. the extent to which evidence contributes to proving an important matter of fact) [ 33 ]. For example, a common objection to legal testimony or evidence is that it is ‘irrelevant’ [ 34 ]. Legal processes for considering the admissibility or legal-relevance of evidence are firmly established, requiring explicit declaration of evidentiary sources and direct consideration of that evidence as it relates both to a specific case and related historical precedents, something that is undeveloped in the health sector [ 35 ]. It is the formality, explicitness and retrospective nature of this process, which is directly associated with a specific case (or decision), that is characteristic of the consideration of relevance in the legal context.

Financial accounting provides another perspective on relevance. In this field, relevance is viewed as a fundamental component of generally accepted accounting principles. Relevance and materiality are emphasised such that accountants and auditors focus on financial information that meets the decision-making needs of users and is expected to affect their decisions. In financial accounting, ‘value relevance’ provides a more focused perspective on relevance, defined as “ …the ability of information disclosed by financial statements to capture and summarise firm value. Value relevance can be measured through the statistical relations between information presented by financial statements and stock market values or returns ” [ 36 ]. Similar to the legal perspective, the financial accounting perspective on relevance is set with a formal context, where the focal point (i.e. financial performance) is clear and principles (i.e. generally accepted accounting principles) and processes (i.e. financial reporting and auditing) are clearly established and monitored.

Education provides a slightly more expansive approach to operationalising relevance, given the more general aim of the enterprise. In the United States, the Glossary of Education Reform [ 37 ] notes that “ …the term relevance typically refers to learning experiences that are either directly applicable to the personal aspirations, interests, or cultural experiences of students (personal relevance) or that are connected in some way to real-world issues, problems, and contexts (life relevance) ”. They further state that “ personal relevance occurs when learning is connected to an individual student’s interests, aspirations, and life experiences ”, while “ life relevance occurs when learning is connected in some way to real-world issues, problems, and contexts outside of school ”. A similar framing of relevance in this context suggests that it “…extends the learning beyond the classroom by teaching students to apply what they are learning to real world situations ” [ 38 ]. While the education sector also makes numerous references to a ‘rigour and relevance’ dyad [ 39 ] in contrast to the dominance of the internal validity focus in healthcare, it is the prominent dual focus on ‘personal’ relevance (with its subjectivist orientation) and ‘life’ or ‘real world’ relevance (with its more universal orientation) that seems to most clearly define the education sector’s perspective on relevance.

One of the most intensive and competitive sectors focusing on relevance is the web search (or information retrieval) field. This includes dominant search engines such as Google and Bing, as well as a wide range of commercial and social media sites such as Amazon, eBay, Facebook and LinkedIn, that compete either directly or indirectly on their ability to identify relevant information in response to user queries. Therefore, the ability of these organisations to advance the theory and practice related to relevance is fundamental to their success. For example, Google was built upon the effectiveness of its search algorithm, which is in a constant state of evolution. Both explicit and implicit approaches to assess relevance are used to contribute to search algorithm refinements [ 40 ]. The explicit approach focuses on ‘relevance ratings’, whereby evaluators (e.g. human raters) are contracted to assess the degree of ‘helpfulness’ of search results paired to specific search queries [ 41 ]. The implicit approach to assess relevance monitors and aggregates search behaviour of millions of users who are likely unaware that their behaviour is being assessed. Google has more recently advanced ‘personalised relevance’, which uses past individual search behaviour to personalise/tailor future search results for the same individual. Pariser has critiqued this concept as “ the filter bubble ” [ 42 ], warning that Google’s intent to optimise search algorithms for personal relevance creates a “ …personal ecosystem of information… ” that limits the diversity of search results and promotes insularity. This personal relevance is situated within the pervasiveness of social media, which facilitates the advancement of ‘social relevance’. Personal and social relevance highlight two important orientations towards relevance – one built on increasingly detailed understanding of individual preferences and the other reflecting the growing power and increasing accessibility of crowd-sourced perspectives. Overall, web search has made important contributions to how we understand and operationalise relevance, including the use of increasingly sophisticated explicit and implicit feedback mechanisms and the ability to draw upon and analyse big data sets. Web search has also exposed the contrasting orientations of personal and social relevance that underscore the challenges of combining or integrating different relevance assessments.

These non-health sector perspectives on relevance highlight several considerations. First, they reinforce general findings that point to perspective, decision context, timeliness and precision of focus or ‘fit’ as key elements of relevance. Additionally, they highlight a few distinctive considerations. The formalistic contexts of financial accounting and law emphasise issues such as precedent and legitimacy, implying that relevance in a research sense might require the demonstration of some legitimate or credible association between research and its use or user, among other considerations. Further, the complex consumerist world of social media highlights some of the challenges of a purely subjectivist definition of relevance. Whereas the International Society for Pharmacoeconomics and Outcomes Research guidance takes a subjectivist stance in suggesting that, “[t] here is no correct answer for relevance ” [ 24 ], the “ filter bubble ” criticised by Pariser [ 42 ] suggests otherwise. Relevance solely to the personally-perceived interests of a research user is unlikely to adequately serve the collective commitments to health and health equity that are especially germane to the health research enterprise.

Forward-looking considerations for assessing the relevance of health research

To this point, we have endeavoured to unpack relevance from theoretical and practical perspectives. In light of these insights and in the context of persistent interest in research impact assessment and evolving interest in research relevance, we now turn to some specific forward-looking considerations for research relevance assessment (RRA).

Relevance of research versus everything else

The first consideration for RRA is the acknowledgement that research is only one of many sources of insight to inform the needs or actions of research users. A research user is influenced by a wide range of political, legal, media, economic and other contextual information, interactions and experiences, as well as prevailing organisational governance, leadership, culture and values that all serve to complement (and often dominate) any insights that might be derived from research [ 43 ]. This reality implies that ‘relevance’ has a different meaning for researchers and research users. Researchers are typically interested in the relevance of a specific research product or activity for identifiable actions of (potentially) multiple research users; relevance is here judged relative to both the perceived needs of research users, and the extent and content of other related research. In contrast, research users are typically focused on identifying multiple relevant inputs to guide a specific action, only some of which may be research; relevance is here judged relative to both the research user’s needs and the form and content of the other inputs.

Given these distinct orientations to research relevance, RRA needs to be explicit about its comparative lens. Clear distinctions should be made between relevance based on the merits of the research product or activity (researcher lens) and relevance based on the relative value of research compared to other research and non-research sources (research user lens). RRA provides an opportunity to build more robust ways to characterise and assess the contribution of research to research users, including a more systematic and transparent articulation of anticipated research uses (akin to the Research Councils UK’s ‘Pathways to Impact’ [ 44 ] or descriptions of planned study design and methodological approach published in study protocols/registrations for randomised controlled trials or systematic reviews).

Beyond instrumental uses of research

The considerations noted above rely heavily on instrumental uses of research. Theoretically derived definitions of relevance, such as Floridi’s [ 14 ], tend to focus on the response to a specified question. This suggests a direct and tangible connection between research and its ‘use’. However, as Weiss [ 43 ] and others have observed, most types of research use are not instrumental, where use is documented and explicitly addresses a specific query or challenge for a research user. Rather, research use tends to be more conceptual, where use is indirect and evolves over time, or symbolic, where use may be politically or tactically motivated [ 43 ]. Research may also create externalities or unintended effects. For example, general research activity might support an engaged learning environment, interactive research relationships, and additional research-related discourse that provides benefits that are not attributable to any specific research product or activity. This has important ramifications for how research is funded and the role that relevance can play in that assessment. Ultimately, RRA needs to go beyond a singular understanding of research use as instrumental use, to develop better methods for capturing and assessing the relevance of the many non-instrumental uses of research.

The temporal factor

Another closely related consideration for RRA is the temporal context. Almost all research is conducted in a temporally defined period. Yet, while the quality of research is typically characterised by its methodology, which is a static feature typically not subject to temporal variation (e.g. the assessed quality of a randomised controlled trial should be consistent over time), relevance of research can be considered at any time (e.g. prior to the initiation of a research study or at different points in time post-completion) and is therefore subject to dynamic perceptions as they pertain to evolving action or decision contexts. Cohen [ 15 ] suggests that “ …relevance, like reasoning, has a prospective dimension as well as a retrospective one. It helps prediction as well as explanation ” (p. 182). The important insight is that, in contrast to research quality, the relevance of a specific research product can change over time, making assessment of research relevance more challenging.

This requires RRA to acknowledge the temporal factor and its associated implications for research relevance. At minimum, RRA should specify the temporal context as either pre-research (e.g. proposal/funding stage) or post-research (e.g. after research results have been produced). RRA at the pre-research stage focuses on proposed inputs and hypothetical outputs and outcomes, and may be more likely to overestimate instrumental research use and underestimate non-instrumental use. RRA at the post-research stage focuses mainly on the importance and value of actual outputs and tangible results, and may capture more non-instrumental research use. The pre-research stage is clearly aligned with research funding/investment processes, while the post-research stage can contribute to retrospective return-on-investment calculations and more general research impact assessment. However, employing this simple temporal categorisation should not lead us to lose sight of the dynamic, iterative nature of research relevance and the opportunity to assess it at interim and ongoing stages that captures re-interpretations or re-applications of research findings over time.

Moving from a subjective to an intersubjective understanding of relevance

An underlying theme in our review of relevance is subjectivity. Consider the broad scientific paradigms of positivism and interpretivism that are typically respectively aligned with research quality and research relevance. Research quality can be viewed as relating to characteristics or features that are assessed objectively, while research relevance may be seen as subjectively adjudicated. The subjective focus emphasises the variability of different perspectives and contexts and the suggestion that anyone can have a different take on the relevance of a specific research product or activity. For RRA, this reinforces a user-centred orientation to relevance assessment that privileges the judgment of the interrogator and raises the key question regarding who is positioned as the main arbiter of research relevance.

However, while relevance may never be characterised as universal, it could be argued that it is not purely subjective either. Rather, relevance may be more consistent with an intersubjective understanding that emphasises the extent of agreement or shared understanding among individual subjective perspectives representing a way to bridge the personal and the universal. The intersubjective view, while not presenting an objective approach to measuring relevance, does provide a road towards a meaningful and structured assessment of research relevance. It also emphasises the importance of representation in forging the intersubjective judgments that guide the research enterprise.

This paper has unpacked research relevance from different perspectives and outlined key considerations for its assessment. Alongside research impact assessment, research relevance seems increasingly important in justifying research investments and guiding strategic research planning. Indeed, judgments of ‘relevance’ are becoming a key component of the health research enterprise. However, consideration of relevance has been largely tacit in the health research community, often depending on unexplained interpretations of value, fit and potential for impact. Reviewing the various uses of relevance in health research, the concept is sometimes used as a synonym for research impact or positioned as a reliable predictor of later consequence. In many ways, research relevance seems a necessary condition for impact – a process or component of efforts to make rigorous research usable. However, relevance is not a necessary or sufficient condition to achieve impact. We expect that research that is relevant, and thus accountable to specific and legitimate users, will be impactful, but this may not necessarily be the case where other factors intervene. Additionally, we may expect that research that is impactful will be appropriately accountable – but again, this is not necessarily the case. Ultimately, relevance stands apart from research impact. Like rigour, relevance is a complementary but distinctive dimension of what it is that ensures ‘the good’ in health research.

While ‘relevance’ is ever-present, understanding of the concept in terms of health research is emergent and not well codified. To improve our understanding, this paper outlines four key considerations, including how research relevance assessments (1) orientate to, capture and compare research versus non-research sources, (2) consider both instrumental versus non-instrumental uses of research, (3) accommodate dynamic temporal-shifting perspectives on research, and (4) align with an intersubjective understanding of relevance. We believe careful and explicit consideration of research relevance, guided by transparent principles and processes is vital to gauge the overall value and impact of a wide range of individual and collective research efforts and investments. We hope this paper generates more discussion and debate to facilitate progress.

Abbreviations

Alberta Innovates – Health Solutions

Canadian Academy of Health Sciences

Canadian Health Services Research Foundation

Canadian Institutes of Health Research

knowledge translation

Ontario SPOR SUPPORT Unit

research impact assessment

research relevance assessment

Strategy for Patient-Oriented Research

Kleinert S, Horton R. How should medical science change? Lancet. 2014;383(9913):197–8.

Article   PubMed   Google Scholar  

Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, Howells DW, Ioannidis JPA, Oliver S. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156–65.

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.

Panel on the Return on Investments in Health Research. Making an impact: a preferred framework and indicators to measure returns on investment in health research. Ottawa: Canadian Academy of Health Sciences; 2009.

Google Scholar  

Banzi R, Moja L, Pistotti V, Facchini A, Liberati A. Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews. Health Res Policy Syst. 2011;9:26.

Article   PubMed   PubMed Central   Google Scholar  

Milat AJ, Bauman AE, Redman S. A narrative review of research impact assessment models and methods. Health Res Policy Syst. 2015;13:18.

Buxton M, Hanney S. How can payback from health services research be assessed? J Health Serv Res Policy. 1996;1:35–43.

CAS   PubMed   Google Scholar  

Graham KER, Chorzempa HL, Valentine PA, J M. Evaluating health research impact: Development and implementation of the Alberta Innovates – Health Solutions impact framework. Res Eval. 2012;21:354–67

Nowotny H, Scott P, Gibbons M. Introduction: ‘Mode 2’ Revisited: The New Production of Knowledge. Minerva. 2003;41(3):179–94.

Article   Google Scholar  

Canadian Institutes of Health Research. Strategy for Patient-Oriented Research. 2013. http://www.cihr-irsc.gc.ca/e/41204.html . Accessed 27 Jul 2014.

Lomas J. Preface: The first ones over the barricade. In: Potvin L, Armstrong P, editors. Shaping Academia for the Public Good: Critical Reflections on the CHSRF/CIHR Chair Program. Toronto: University of Toronto Press; 2013.

Ontario Ministry of Health and Long-Term Care – Community and Health Promotion Branch. Ontario Support for People and Patient-Oriented Research and Trials (SUPPORT) Unit: Business Plan. Toronto, ON: MOHLTC; 2013.

Buxton M. The payback of ‘Payback’: challenges in assessing research impact. Res Eval. 2011;20(3):259–60.

Floridi L. Understanding epistemic relevance. Erkenntnis. 2008;69(1):69–92.

Cohen LJ. Some steps towards a general theory of relevance. Synthese. 1994;101(2):171–85.

Gärdenfors P. On the logic of relevance. Synthese. 1978;37(3):351–67.

Schlesinger GN. Relevance Theoria. 1986;57(1):57–67.

Keynes JM. A treatise on probability. London: MacMillan and Co. Limited; 1921.

Bush V. Science - the endless frontier. A report to the President on a program of postwar scientific research. Washington: National Science Foundation; 1945.

Dudley JM. Defending basic research. Nat Photonics. 2013;7:338–9.

Article   CAS   Google Scholar  

Stokes DE. Pasteur's quadrant - basic science and technological innovation. Washington: Brookings Institution Press; 1997.

Tushman M, O'Reilly C. Research and relevance: implications of Pasteur's Quadrant for doctoral programs and faculty development. Acad Manag J. 2007;50(4):769–74.

Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13–24.

Berger ML, Martin BC, Husereau D, Worley K, Allen JD, Yang W, Quon NC, Mullins CD, Kahler KH, Crown W. A questionnaire to assess the relevance and credibility of observational studies to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health. 2014;17(2):143–56.

Jansen JP, Trikalinos T, Cappelleri JC, Daw J, Andes S, Eldessouki R, Salanti G. Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health. 2014;17(2):157–73.

Mays N, Pope C. Qualitative research in health care: assessing quality in qualitative research. BMJ. 2000;320(7226):50–2.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Pope C, Ziebland S, Mays N. Qualitative research in health care: analysing qualitative data. BMJ. 2000;2000(320):114–6.

Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ, for the GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924–6.

Campbell Collaboration. Campbell Collaboration Systematic Reviews: Policies and Guidelines (Version 1.2). 2016. https://www.campbellcollaboration.org/library/campbell-collaboration-systematic-reviews-policies-and-guidelines.html . Accessed 27 Mar 2017.

Burchett H, Umoquit M, Dobrow M. How do we know when research from one setting can be useful in another? A review of external validity, applicability and transferability. J Health Serv Res Policy. 2011;16(4):238–44.

World Health Organization. WHO recommendations: Optimizing health worker roles to improve access to key maternal and newborn health interventions through task shifting. Annex 8 Contextualizing the guidelines – workbook. 2012. http://www.who.int/reproductivehealth/publications/maternal_perinatal_health/Annex_8_Contextualizing_Workbook.pdf?ua=1 . Accessed 27 Sep 2016.

Re-aim.org. What is RE-AIM. 2017. http://re-aim.org/about/what-is-re-aim . Accessed 27 Mar 2017.

Center for the Study of Language and Information, Stanford University. Stanford Encyclopedia of Philosopy: The legal concept of evidence. 2015. http://plato.stanford.edu/entries/evidence-legal/ . Accessed 27 Sep 2016.

Cornell University Law School. Rule 402. General admissibility of relevant evidence. 2016. https://www.law.cornell.edu/rules/fre/rule_402 . Accessed 27 Sep 2016.

Giacomini M. One of these things is not like the others: the idea of precedence in health technology assessment and coverage decisions. Milbank Q. 2005;83(2):193–223.

Karğın S. The Impact of IFRS on the Value Relevance of Accounting Information: Evidence from Turkish Firms. Int J Econ Finance. 2013;5(4):71–80.

Abbott S. Relevance. In: Abbott S, editor. The glossary of education reform. Portland: Great Schools Partnership; 2013.

Pearson. Rigor and relevance: an overview. 2014. http://www.californiareading.com/media/pdf/rigor_and_relevance.pdf . Accessed 12 Oct 2014.

International Center for Leadership in Education. The Rigor Relevance Framework. 2016. http://www.leadered.com/our-philosophy/rigor-relevance-framework.php . Accessed 27 Sep 2016.

Google. Google Inside Search – Algorithms. 2016. http://www.google.ca/insidesearch/howsearchworks/algorithms.html . Accessed 27 Sep 2016.

Google. Search Quality Evaluator Guidelines. 2016. Accessed 28 Mar 2016.

Pariser E. The Filter Bubble: What the Internet is Hiding from You. New York: Penguin Group Inc.; 2011.

Weiss C. The many meanings of research utilization. Public Adm Rev. 1979;39(5):426–31.

Research Councils UK. Pathways to Impact. 2014. http://www.rcuk.ac.uk/innovation/impacts . Accessed 22 Feb 2017.

Download references

Acknowledgements

We acknowledge and appreciate the contributions of participants of a roundtable discussion to gather feedback on an earlier version of this paper. Participants included Simon Denegri, National Director for Public Participation and Engagement in Research, National Institute for Health Research (NIHR) UK, and Chair of INVOLVE, UK; Lee Fairclough, Vice-President, Quality Improvement, Health Quality Ontario; Michael Hillmer, Director, Planning, Research and Analysis Branch, Ontario Ministry of Health and Long-Term Care; John McLaughlin, Chief Science Officer and Senior Scientist, Public Health Ontario; Allison Paprica, Director, Strategic Partnerships, ICES; Michael Schull, President and CEO, ICES; and Vasanthi Srinivasan, Executive Director, Ontario Strategy for Patient-Oriented Research (SPOR) SUPPORT Unit (OSSU). We also want to thank John Lavis of the McMaster Health Forum for his very helpful comments on an earlier draft. Though we owe these individuals and organisations many thanks for their insights and support, we alone are responsible for the final product.

This work was commissioned by the Ontario SPOR Support Unit (OSSU). The executive director of the OSSU was one of the participants in a roundtable discussion to gather feedback on an earlier version of this paper, but beyond that, the OSSU did not have any role in the design of the study, collection, analysis or interpretation of the data, or writing of the manuscript.

Availability of data and materials

Not applicable. No datasets were generated or analysed during the development of the article.

Authors’ contributions

ADB acquired funding for the study. MJD, FAM and ADB conceptualised the study. MJD, FAM, CF and ADB participated in the review and writing of the manuscript. MJD, FAM and ADB participated in the roundtable discussion. MJD, FAM and ADB reviewed and approved the final version of the manuscript (CF passed away prior to submission of the manuscript).

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and affiliations.

Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, 155 College Street, 4th Floor, Toronto, ON, M5T 3M6, Canada

Mark J. Dobrow, Fiona A. Miller & Adalsteinn D. Brown

Alberta Innovates - Health Solutions, Edmonton, Alberta, Canada

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mark J. Dobrow .

Additional information

This article is dedicated to the memory of Dr Cy Frank, our co-author and esteemed colleague, whose untimely death occurred midway through development of this work. Among his many interests, Dr Frank was a champion for improving understanding of research impact assessment and provided many insights on the concept of research relevance, some of which we expand upon in this article. His many contributions to the health sector will live on, but he will be greatly missed.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Dobrow, M.J., Miller, F.A., Frank, C. et al. Understanding relevance of health research: considerations in the context of research impact assessment. Health Res Policy Sys 15 , 31 (2017). https://doi.org/10.1186/s12961-017-0188-6

Download citation

Received : 27 September 2016

Accepted : 07 March 2017

Published : 17 April 2017

DOI : https://doi.org/10.1186/s12961-017-0188-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health research systems
• Research relevance
• Research impact

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Here’s how you know

• U.S. Department of Health and Human Services
• National Institutes of Health

The Importance of Research on Health and Well-Being

Director’s Page Helene M. Langevin, M.D.

February 7, 2019

As I’ve dived into my role as Director at NCCIH, one of the things that’s made me so energized about the position is the smart, pragmatic thinking embedded within the Strategic Plan NCCIH adopted in 2016. It’s a twofold cogent recognition of: 1) the very real challenges faced daily by patients and their health care providers, and 2) the opportunities the research community has to offer much-needed evidence to inform decisions about patient care. Especially important is our third strategic plan objective , which focuses on how we can explore the potential of complementary health approaches to foster health promotion and disease prevention across the lifespan.

Why is this part of NCCIH’s strategic plan so important? Since the beginning of the 20th century, modern medicine and biomedical research have overwhelmingly focused on the study and treatment of disease. In contrast, health—and especially the return to health after an illness—has received comparatively little attention.

This emphasis on treating diseases is largely a byproduct of a very good thing—the tremendous gains yielded by researchers in finding treatments for diseases and the effectiveness of pharmacologic approaches in both treating and managing diseases. These strides in advancing human health can be seen in antibiotics to treat bacterial infections or medications to manage chronic illnesses, such as diabetes, hypertension, and rheumatic diseases.

Yet these critical successes in treating and managing disease may also mean that the often-painstaking task of helping the patient recuperate during the “convalescent” period after an acute illness, or following an exacerbation of a chronic relapsing condition, has not yet been adequately studied.

Though the treatment-focused model is dominant in our research and health care ecosystem, there has been a longstanding awareness that many chronic diseases can be prevented or better managed by incorporating nonpharmacologic interventions such as nutrition, exercise, and stress management. When these methods are incorporated into care and patients are able to make lasting behavioral changes, the end result can be more durable improvements in health. Many complementary and integrative health practices follow this model, and there’s preliminary evidence indicating that some complementary approaches may be useful in encouraging improved self-care, a better personal sense of well-being, and a greater commitment to a healthy lifestyle.

In fact, one of the research strategies within NCCIH’s current strategic plan is to advance understanding of the mechanisms through which mind and body approaches affect health, resilience, and well-being. This includes a focus on methodologically rigorous evaluations that will lead to a greater understanding of whether, when, how, and for whom such practices can have substantial impact. For example, we support research designed to understand the ways in which an integrative approach to treating chronic back pain can lead to lasting healing and improved function and well-being. This research may provide critically important new therapeutic approaches for those patients who have not found relief with surgery or pain medicines.

I’m very much looking forward to the National Advisory Council for Complementary and Integrative Health (NACCIH) meeting on Friday, February 8th. It will be my first Council meeting as the director of NCCIH. The meeting will feature a symposium, “NIH Research on Well-Being,” and I invite you to listen to the conversation on this important research topic, as well as hear updates on the Center’s activities, policies, and funding priorities during the meeting’s open session. NCCIH will livestream the open session of Council on NIH Videocast from 10:00 a.m. to 3:20 p.m. ET, and it will be archived.

Past Messages From the Director

NCCIH Heads to Cleveland for ICIMH April 5, 2024

Addressing Pain Requires Understanding How We Heal March 27, 2024

Let’s Leverage All of Us To Learn More About Health and Disease in Diverse Populations February 19, 2024

More messages

Why the reliance on data? Findings and statistics from research studies can impact us emotionally, add credibility to an article, and ground us in the real world. However, the importance of research findings reaches far beyond providing knowledge to the general population. Research and evaluation studies — those studies that assess a program’s impact — are integral to promoting mental health and reducing the burden of mental illness in different populations.

Mental health research identifies biopsychosocial factors — how biological, psychological and social functioning are interacting — detecting trends and social determinants in population health. That data greatly informs the current state of mental health in the U.S. and around the world. Findings from such studies also influence fields such as public health, health care and education. For example, mental health research and evaluation can impact public health policies by assisting public health professionals in strategizing policies to improve population mental health.

Research helps us understand how to best promote mental health in different populations. From its definition to how it discussed, mental health is seen differently in every community. Thus, mental health research and evaluation not only reveals mental health trends but also informs us about how to best promote mental health in different racial and ethnic populations. What does mental health look like in this community? Is there stigma associated with mental health challenges? How do individuals in the community view those with mental illness? These are the types of questions mental health research can answer.

Data aids us in understanding whether the mental health services and resources that are available meet mental health needs. Many times the communities where needs are the greatest are the ones where there are limited services and resources available. Mental health research and evaluation informs public health professionals and other relevant stakeholders of the gaps that currently exist so they can prioritize policies and strategies for communities where gaps are the greatest.

Research establishes evidence for the effectiveness of public health policies and programs. Mental health research and evaluation help develop evidence for the effectiveness of healthcare policies and strategies as well as mental health promotion programs. This evidence is crucial for showcasing the value and return on investment for programs and policies, which can justify local, state and federal expenditures. For example, mental health research studies evaluating the impact of Mental Health First Aid (MHFA) have revealed that individuals taking the course show increases in knowledge about mental health, greater confidence to assist others in distress, and improvements in their own mental wellbeing. They have been fundamental in assisting organizations and instructors in securing grant funding to bring MHFA to their communities.

The findings from mental health research and evaluation studies provide crucial information about the specific needs within communities and the impacts of public education programs like MHFA. These studies provide guidance on how best to improve mental health in different contexts and ensure financial investments go towards programs proven to improve population mental health and reduce the burden of mental illness in the U.S.

In 2021, in a reaffirmation of its dedication and commitment to mental health and substance use research and community impact, Mental Health First Aid USA introduced MHFA Research Advisors. The group advises and assists Mental Health First Aid USA on ongoing research and future opportunities related to individual MHFA programs, including Youth MHFA, teen MHFA and MHFA at Work.

Through this advisory group and evaluation efforts at large, Mental Health First Aid USA will #BeTheDifference for mental health research and evaluation across communities in the US.

Learn more about MHFA Research Advisors and how you can share your research with us.

Get the latest MHFA blogs, news and updates delivered directly to your inbox so you never miss a post.

Share and help spread the word.

Related stories.

No related posts.

• Alzheimer's disease & dementia
• Arthritis & Rheumatism
• Attention deficit disorders
• Autism spectrum disorders
• Biomedical technology
• Diseases, Conditions, Syndromes
• Endocrinology & Metabolism
• Gastroenterology
• Gerontology & Geriatrics
• Health informatics
• Inflammatory disorders
• Medical economics
• Medical research
• Medications
• Neuroscience
• Obstetrics & gynaecology
• Oncology & Cancer
• Ophthalmology
• Overweight & Obesity
• Parkinson's & Movement disorders
• Psychology & Psychiatry
• Radiology & Imaging
• Sleep disorders
• Sports medicine & Kinesiology
• Vaccination
• Breast cancer
• Cardiovascular disease
• Chronic obstructive pulmonary disease
• Colon cancer
• Coronary artery disease
• Heart attack
• Heart disease
• High blood pressure
• Kidney disease
• Lung cancer
• Multiple sclerosis
• Myocardial infarction
• Ovarian cancer
• Post traumatic stress disorder
• Rheumatoid arthritis
• Schizophrenia
• Skin cancer
• Type 2 diabetes
• Full List »

share this!

April 9, 2024

This article has been reviewed according to Science X's editorial process and policies . Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

trusted source

written by researcher(s)

Preparing for the next health crisis: COVID-19 showed the importance of community-engaged research

by Simran Purewal and Julia Smith, The Conversation

Community-engaged research involves the active and meaningful involvement of people directly affected by a research problem. Like most activities that require personal interaction, this type of research was disrupted by the restrictions of the COVID-19 pandemic.

This research approach is intended to build trustworthy relationships and yield mutual benefits. Community-engaged projects have garnered attention over the past two decades as they focus on tackling inequities , which often arise during public health crises.

Conducting community-engaged research has several benefits, like increasing the relevancy of studies , incorporating lived experience , and supporting the sharing of findings back to affected communities. Community engagement also plays an important role in emergency responses. Community-engaged emergency responses can promote the uptake of public health interventions and bolster advocacy efforts.

Community-engaged research during the pandemic

Researchers' ability to engage with communities was impacted by the COVID-19 pandemic. Public health measures focused on preventing the spread of COVID-19 (such as limits on in-person gatherings) halted traditional forms of fieldwork. For example, researchers could not safely host in-person interviews. They were forced to quickly adapt to unfamiliar virtual teaching and learning platforms.

At the same time, civil society organizations (CSOs), non-profit agencies operating separately from government and business, were stretched thin as their demand soared. These organizations are sought after as community research partners since they are embedded in the communities they serve and provide crucial services to community members. Examples include the United Way BC and Sources Community Resource Center , which provide direct services, support and relief to communities across B.C.

In 2023, the Pacific Institute on Pathogens, Pandemics, and Society ( PIPPS ) hosted a roundtable with community-engaged researchers at Simon Fraser University to learn about their experiences engaging with communities amid the pandemic. Findings from our roundtable, supplementing this article, have also been included in our Community-Engaged Research during Health Crises: Engaging with Civil Society Organizations handbook published by PIPPS and SFU Community Engaged Research Initiative .

Barriers to conducting community-engaged research

Roundtable attendees first discussed the challenges of conducting research remotely, with one noting how their research plans were put on pause for more than three months because of pandemic-related restrictions. Attendees also highlighted how managing multiple forms of online communication disrupted their work-life balance.

Several attendees found it difficult to recruit research participants; they discussed the challenges of the digital divide , referring to the gap between communities' access to information and communication technologies.

Researchers faced challenges with ethics review boards, which did not consider the risks and unique considerations of engaging communities in a public health crisis. While ethics applications were expedited, researchers felt they lacked guidance for community-engaged research during the pandemic.

Researchers also found it hard to maintain relationships with communities amidst the pandemic. Some key ways researchers connect with communities are through sharing findings and hosting food-sharing events, such as lunch and learns. These opportunities were not available during the pandemic. In addition, many of their long-standing relationships with CSOs were strained as they experienced layoffs and increased demand. Researchers did not want to impede on CSOs' frontline pandemic-related efforts.

Opportunities emerging from the pandemic

Despite the challenges they faced, researchers identified a range of opportunities that emerged as a result of the pandemic. They noted how they could increase the scope of their projects since virtual tools, like Zoom, allowed them to reach rural and remote communities. These platforms also provided low-barrier forms of participation for participants with accommodation needs.

Researchers also discussed how the pandemic forced the "professional veneers to slip away." Over time, researchers connected on a more vulnerable level with their community partners, as they all attempted to get through the pandemic. Collectively, they showed up in their most authentic way and practiced humility in their partnerships. Moreover, research teams emphasized the importance of building community, which reinforced their commitment to mutual benefit .

Through their community-engaged work, some researchers hired people with lived experience of the research problem of interest. This opportunity emerged during the pandemic. Compensating members of the research team for their knowledge strengthened the relevancy of their findings as they directly learned how the pandemic was impacting distinct groups.

Lessons learned: Conducting community-engaged research in future crises

Participants were asked what they would do differently in future health crises. Some discussed the significance of holding informal check-ins with their teams to openly discuss professional and personal challenges. Others pointed to the need for knowledge and resource sharing with other community-engaged researchers, to break down silos.

Additionally, attendees underscored the benefits of interdisciplinary research teams , bringing together diverse skills and expertise. In health crises, they aim to work collaboratively with academics and service providers from CSOs.

Based on the key themes of the roundtable, three recommendations emerged to support community-engaged research in future public health crises:

1. Post-secondary institutions should develop guidance for community-engaged research in health emergencies

Since post-secondary institutions increasingly recognize the importance of community-university partnerships , institutions should create protocols to support community-engaged research in public health crises . Attention should be paid to crisis-related considerations, including funding sources, resource challenges and ethics.

2. Develop targeted funding opportunities for community-engaged research partnerships

During the pandemic, research unrelated to COVID-19 faced funding drawbacks and resource constraints . This may have secondary effects in the " post-pandemic era ," especially for projects addressing health inequities. To avoid these unintended consequences, partnerships should be proactively supported by post-secondary institutions and funding agencies, to provide research partners with honoraria for their time and insights shared, and to help academic researchers build networks for engagement.

3. Prioritize capacity-building in partnerships

Mutual benefit is a guiding principle of community-engaged research . In partnerships, academic researchers often benefit through career advancement and a sense of fulfillment . Yet, community partners are not always assured the same benefits. To ensure mutual benefit, capacity building , referring to the process of building skills, abilities and resources, should guide community-engaged research partnerships. Both academic researchers and community partners bring significant assets to projects; these assets and training gaps should be uncovered at the outset of projects and considered throughout the partnership.

In addition to these recommendations, a repeated theme for participants was how community-engaged research should be viewed as a fundamental component of their work, as opposed to an afterthought. Rather than treating community-engaged research as a " peripheral activity ," the principles of community engagement should be embedded in research, teaching and learning.

Explore further

Feedback to editors

Researchers demonstrate miniature brain stimulator in humans

18 hours ago

Study reveals potential to reverse lung fibrosis using the body's own healing technique

Apr 12, 2024

Researchers discover cell 'crosstalk' that triggers cancer cachexia

Study improves understanding of effects of household air pollution during pregnancy

Wearable sensors for Parkinson's can improve with machine learning, data from healthy adults

New insights on B cells: Researchers explore building better antibodies and curbing autoimmune diseases

Grieving pet owners comforted by 'supernatural' interactions

Study shows AI improves accuracy of skin cancer diagnoses

Cell's 'garbage disposal' may have another role: Helping neurons near skin sense the environment

Chlamydia vaccine shows promise in early trial

Related stories.

Study identifies successful methods to recruit South Asian women for breast cancer research

Mar 11, 2024

Closing the clinical trials disparity gap

Feb 22, 2024

New study examines ethics of community-engaged research from the perspective of community partners

Apr 21, 2022

How to strengthen community resilience in a world plagued by crises

Jan 15, 2024

Research group releases new white paper on molecular HIV epidemiology

May 26, 2023

Building trust and saving lives: A community approach to genetic education

Jan 12, 2024

Recommended for you

Study of data from thousands of women suggests ovarian cycle is regulated by circadian rhythm

Researchers shed light on the molecular causes of different functions of opioid receptors

Using AI to spot parasites in stool samples

Let us know if there is a problem with our content.

Use this form if you have come across a typo, inaccuracy or would like to send an edit request for the content on this page. For general inquiries, please use our contact form . For general feedback, use the public comments section below (please adhere to guidelines ).

Please select the most appropriate category to facilitate processing of your request

Thank you for taking time to provide your feedback to the editors.

Your feedback is important to us. However, we do not guarantee individual replies due to the high volume of messages.

E-mail the story

Your email address is used only to let the recipient know who sent the email. Neither your address nor the recipient's address will be used for any other purpose. The information you enter will appear in your e-mail message and is not retained by Medical Xpress in any form.

Newsletter sign up

Get weekly and/or daily updates delivered to your inbox. You can unsubscribe at any time and we'll never share your details to third parties.

More information Privacy policy

Donate and enjoy an ad-free experience

We keep our content available to everyone. Consider supporting Science X's mission by getting a premium account.

E-mail newsletter

• Open access
• Published: 27 July 2006

The importance of human resources management in health care: a global context

• Stefane M Kabene 1 , 3 ,
• Carole Orchard 3 ,
• John M Howard 2 ,
• Mark A Soriano 1 &
• Raymond Leduc 1  

Human Resources for Health volume  4 , Article number:  20 ( 2006 ) Cite this article

799k Accesses

168 Citations

12 Altmetric

Metrics details

This paper addresses the health care system from a global perspective and the importance of human resources management (HRM) in improving overall patient health outcomes and delivery of health care services.

We explored the published literature and collected data through secondary sources.

Various key success factors emerge that clearly affect health care practices and human resources management. This paper will reveal how human resources management is essential to any health care system and how it can improve health care models. Challenges in the health care systems in Canada, the United States of America and various developing countries are examined, with suggestions for ways to overcome these problems through the proper implementation of human resources management practices. Comparing and contrasting selected countries allowed a deeper understanding of the practical and crucial role of human resources management in health care.

Proper management of human resources is critical in providing a high quality of health care. A refocus on human resources management in health care and more research are needed to develop new policies. Effective human resources management strategies are greatly needed to achieve better outcomes from and access to health care around the world.

Peer Review reports

Defining human resources in health care

Within many health care systems worldwide, increased attention is being focused on human resources management (HRM). Specifically, human resources are one of three principle health system inputs, with the other two major inputs being physical capital and consumables [ 1 ]. Figure 1 depicts the relationship between health system inputs, budget elements and expenditure categories.

Relationship between health system inputs, budget elements and expenditure categories . Source: World Health Report 2000 Figure 4.1 pg.75. http://www.who.int.proxy.lib.uwo.ca:2048/whr/2000/en/whr00_ch4_en.pdf Figure 1 identifies three principal health system inputs: human resources, physical capital and consumables. It also shows how the financial resources to purchase these inputs are of both a capital investment and a recurrent character. As in other industries, investment decisions in health are critical because they are generally irreversible: they commit large amounts of money to places and activities that are difficult, even impossible, to cancel, close or scale down [1].

Human resources, when pertaining to health care, can be defined as the different kinds of clinical and non-clinical staff responsible for public and individual health intervention [ 1 ]. As arguably the most important of the health system inputs, the performance and the benefits the system can deliver depend largely upon the knowledge, skills and motivation of those individuals responsible for delivering health services [ 1 ].

As well as the balance between the human and physical resources, it is also essential to maintain an appropriate mix between the different types of health promoters and caregivers to ensure the system's success [ 1 ]. Due to their obvious and important differences, it is imperative that human capital is handled and managed very differently from physical capital [ 1 ]. The relationship between human resources and health care is very complex, and it merits further examination and study.

Both the number and cost of health care consumables (drugs, prostheses and disposable equipment) are rising astronomically, which in turn can drastically increase the costs of health care. In publicly-funded systems, expenditures in this area can affect the ability to hire and sustain effective practitioners. In both government-funded and employer-paid systems, HRM practices must be developed in order to find the appropriate balance of workforce supply and the ability of those practitioners to practise effectively and efficiently. A practitioner without adequate tools is as inefficient as having the tools without the practitioner.

Key questions and issues pertaining to human resources in health care

When examining health care systems in a global context, many general human resources issues and questions arise. Some of the issues of greatest relevance that will be discussed in further detail include the size, composition and distribution of the health care workforce, workforce training issues, the migration of health workers, the level of economic development in a particular country and sociodemographic, geographical and cultural factors.

The variation of size, distribution and composition within a county's health care workforce is of great concern. For example, the number of health workers available in a country is a key indicator of that country's capacity to provide delivery and interventions [ 2 ]. Factors to consider when determining the demand for health services in a particular country include cultural characteristics, sociodemographic characteristics and economic factors [ 3 ].

Workforce training is another important issue. It is essential that human resources personnel consider the composition of the health workforce in terms of both skill categories and training levels [ 2 ]. New options for the education and in-service training of health care workers are required to ensure that the workforce is aware of and prepared to meet a particular country's present and future needs [ 2 ]. A properly trained and competent workforce is essential to any successful health care system.

The migration of health care workers is an issue that arises when examining global health care systems. Research suggests that the movement of health care professionals closely follows the migration pattern of all professionals in that the internal movement of the workforce to urban areas is common to all countries [ 2 ]. Workforce mobility can create additional imbalances that require better workforce planning, attention to issues of pay and other rewards and improved overall management of the workforce [ 2 ]. In addition to salary incentives, developing countries use other strategies such as housing, infrastructure and opportunities for job rotation to recruit and retain health professionals [ 2 ], since many health workers in developing countries are underpaid, poorly motivated and very dissatisfied [ 3 ]. The migration of health workers is an important human resources issue that must be carefully measured and monitored.

Another issue that arises when examining global health care systems is a country's level of economic development. There is evidence of a significant positive correlation between the level of economic development in a country and its number of human resources for health [ 3 ]. Countries with higher gross domestic product (GDP) per capita spend more on health care than countries with lower GDP and they tend to have larger health workforces [ 3 ]. This is an important factor to consider when examining and attempting to implement solutions to problems in health care systems in developing countries.

Socio-demographic elements such as age distribution of the population also play a key role in a country's health care system. An ageing population leads to an increase in demand for health services and health personnel [ 3 ]. An ageing population within the health care system itself also has important implications: additional training of younger workers will be required to fill the positions of the large number of health care workers that will be retiring.

It is also essential that cultural and geographical factors be considered when examining global health care systems. Geographical factors such as climate or topography influence the ability to deliver health services; the cultural and political values of a particular nation can also affect the demand and supply of human resources for health [ 3 ]. The above are just some of the many issues that must be addressed when examining global health care and human resources that merit further consideration and study.

The impact of human resources on health sector reform

When examining global health care systems, it is both useful and important to explore the impact of human resources on health sector reform. While the specific health care reform process varies by country, some trends can be identified. Three of the main trends include efficiency, equity and quality objectives [ 3 ].

Various human resources initiatives have been employed in an attempt to increase efficiency. Outsourcing of services has been used to convert fixed labor expenditures into variable costs as a means of improving efficiency. Contracting-out, performance contracts and internal contracting are also examples of measures employed [ 3 ].

Many human resources initiatives for health sector reform also include attempts to increase equity or fairness. Strategies aimed at promoting equity in relation to needs require more systematic planning of health services [ 3 ]. Some of these strategies include the introduction of financial protection mechanisms, the targeting of specific needs and groups, and re-deployment services [ 3 ]. One of the goals of human resource professionals must be to use these and other measures to increase equity in their countries.

Human resources in health sector reform also seek to improve the quality of services and patients' satisfaction. Health care quality is generally defined in two ways: technical quality and sociocultural quality. Technical quality refers to the impact that the health services available can have on the health conditions of a population [ 3 ]. Sociocultural quality measures the degree of acceptability of services and the ability to satisfy patients' expectations [ 3 ].

Human resource professionals face many obstacles in their attempt to deliver high-quality health care to citizens. Some of these constraints include budgets, lack of congruence between different stakeholders' values, absenteeism rates, high rates of turnover and low morale of health personnel [ 3 ].

Better use of the spectrum of health care providers and better coordination of patient services through interdisciplinary teamwork have been recommended as part of health sector reform [ 4 ]. Since all health care is ultimately delivered by people, effective human resources management will play a vital role in the success of health sector reform.

In order to have a more global context, we examined the health care systems of Canada, the United States of America, Germany and various developing countries. The data collection was achieved through secondary sources such as the Canadian Health Coalition, the National Coalition on Health Care and the World Health Organization Regional Office for Europe. We were able to examine the main human resources issues and questions, along with the analysis of the impact of human resources on the health care system, as well as the identification of the trends in health sector reform. These trends include efficiency, equity and quality objectives.

Health care systems

The Canadian health care system is publicly funded and consists of five general groups: the provincial and territorial governments, the federal government, physicians, nurses and allied health care professionals. The roles of these groups differ in numerous aspects. See Figure 2 for an overview of the major stakeholders in the Canadian health care system.

Overview of the major stakeholders in the Canadian health care system . Figure 2 depicts the major stakeholders in the Canadian health care system and how they relate.

Provincial and territorial governments are responsible for managing and delivering health services, including some aspects of prescription care, as well as planning, financing, and evaluating hospital care provision and health care services [ 5 ]. For example, British Columbia has shown its commitment to its health care program by implementing an increase in funding of CAD 6.7 million in September 2003, in order to strengthen recruitment, retention and education of nurses province-wide [ 6 ]. In May 2003, it was also announced that 30 new seats would be funded to prepared nurse practitioners at the University of British Columbia and at the University of Victoria [ 6 ]. Recently the Ontario Ministry of Health and Long Term Care announced funding for additional nurse practitioner positions within communities. Furthermore, most provinces and territories in Canada have moved the academic entry requirement for registered nurses to the baccalaureate level, while increasing the length of programmes for Licensed Practice Nurses to meet the increasing complexity of patient-care needs. Several provinces and territories have also increased seats in medical schools aimed towards those students wishing to become family physicians [ 7 ].

The federal government has other responsibilities, including setting national health care standards and ensuring that standards are enforced by legislative acts such as the Canada Health Act (CHA) [ 5 ]. Constitutionally the provinces are responsible for the delivery of health care under the British North America (BNA) Act; the provinces and territories must abide by these standards if they wish to receive federal funding for their health care programs [ 8 ]. The federal government also provides direct care to certain groups, including veterans and First Nation's peoples, through the First Nationals and Inuit Health Branch (FNIHB). Another role of the federal government is to ensure disease protection and to promote health issues [ 5 ].

The federal government demonstrates its financial commitment to Canada's human resources in health care by pledging transfer funds to the provinces and direct funding for various areas. For example, in the 2003 Health Care Renewal Accord, the federal government provided provinces and territories with a three-year CAD 1.5 billion Diagnostic/Medical Equipment Fund. This was used to support specialized staff training and equipment that improved access to publicly funded services [ 6 ].

The third group – private physicians – is generally not employed by the government, but rather is self-employed and works in a private practice. They deliver publicly-funded care to Canadian citizens. Physicians will negotiate fee schedules for their services with their provincial governments and then submit their claims to the provincial health insurance plan in order to receive their reimbursement [ 5 ].

The roles of nurses consist of providing care to individuals, groups, families, communities and populations in a variety of settings. Their roles require strong, consistent and knowledgeable leaders, who inspire others and support professional nursing practice. Leadership is an essential element for high-quality professional practice environments in which nurses can provide high-quality nursing care [ 9 ].

In most Canadian health care organizations, nurses manage both patient care and patient care units within the organization. Nurses have long been recognized as the mediators between the patient and the health care organization [ 10 ]. In care situations, they generally perform a coordinating role for all services needed by patients. They must be able to manage and process nursing data, information and knowledge to support patient care delivery in diverse care-delivery settings [ 10 ]. Workplace factors most valued by nurses include autonomy and control over the work environment, ability to initiate and sustain a therapeutic relationship with patients and a collaborative relationship with physicians at the unit level [ 11 ].

In addition to doctors and nurses, there are many more professionals involved in the health care process. Allied health care professionals can consist of pharmacists, dietitians, social workers and case managers, just to name a few. While much of the focus is on doctors and nurses, there are numerous issues that affect other health care providers as well, including workplace issues, scopes of practice and the impact of changing ways of delivering services [ 12 ]. Furthermore, with health care becoming so technologically advanced, the health care system needs an increasing supply of highly specialized and skilled technicians [ 12 ]. Thus we can see the various roles played by these five groups and how they work together to form the Canadian health care system.

Canada differs from other nations such as the United States of America for numerous reasons, one of the most important being the CHA. As previously mentioned, the CHA sets national standards for health care in Canada. The CHA ensures that all Canadian citizens, regardless of their ability to pay, will have access to health care services in Canada. "The aim of the CHA is to ensure that all eligible residents of Canada have reasonable access to insured health services on a prepaid basis, without direct charges at the point of service" [ 6 ].

Two of the most significant stipulations of the CHA read: "reasonable access to medically necessary hospital and physician services by insured persons must be unimpeded by financial or other barriers" and "health services may not be withheld on the basis of income, age, health status, or gender" [ 5 ]. These two statements identify the notable differences between the Canadian and American health care systems. That is, coverage for the Canadian population is much more extensive.

Furthermore in Canada, there has been a push towards a more collaborative, interdisciplinary team approach to delivering health care; this raises many new issues, one of which will involve successful knowledge transfer within these teams [ 13 ]. Effective knowledge management, which includes knowledge transfer, is increasingly being recognized as a crucial aspect of an organization's basis for long-term, sustainable, competitive advantage [ 34 ]. Even though health care in Canada is largely not for profit, there will still be the need for effective knowledge management practices to be developed and instituted. The introduction of interdisciplinary health teams in Canadian hospitals is a relatively new phenomenon and their connection to the knowledge management policies and agendas of governments and hospital administrations raises important questions about how such teams will work and to what extent they can succeed in dealing with the more difficult aspects of knowledge management, such as the transfer of tacit knowledge.

The multidisciplinary approach tends to be focused around specific professional disciplines, with health care planning being mainly top-down and dominated by medical professionals. Typically there is a lead professional (usually a physician) who determines the care and, if necessary, directs the patient to other health care specialists and allied professionals (aides, support workers). There is generally little involvement by the patient in the direction and nature of the care. Interdisciplinary health care is a patient-centred approach in which all those involved, including the patient, have input into the decisions being made.

The literature on teamwork and research on the practices in hospitals relating to multidisciplinary teams suggests that interdisciplinary teams face enormous challenges [ 13 ], therefore multidisciplinary teamwork will continue to be a vital part of the health care system. However, the goal of this teamwork should not be to displace one health care provider with another, but rather to look at the unique skills each one brings to the team and to coordinate the deployment of these skills. Clients need to see the health worker most appropriate to deal with their problem [ 14 ].

Some of the issues regarding the Canadian public system of health have been identified in the Mazankowski Report, which was initiated by Alberta's Premier Ralph Klein in 2000. Many issues have arisen since this time and have been debated among Canadians. One of the most contentious, for example, is the possibility of introducing a two-tier medical system. One tier of the proposed new system would be entirely government-funded through tax dollars and would serve the same purpose as the current publicly-funded system. The second tier would be a private system and funded by consumers [ 5 ].

However, the CHA and the Canadian Nurses Association (CNA) are critical of any reforms that pose a threat to the public health care system. It should be noted that although Canada purports to have a one-tier system, the close proximity of private, fee-for-service health care in the United States really creates a pay-as-you-go second tier for wealthy Canadians. In addition, many health care services such as most prescriptions and dental work are largely funded by individuals and/or private or employer paid insurance plans.

It is important to realize the differences between the proposed two-tier system and the current health care system. Presently, the public health care system covers all medically necessary procedures and the private sector provides 30% for areas such as dental care. With the new system, both public and private care would offer all services and Canadians would have the option of choosing between the two.

The proposal of the two-tier system is important because it highlights several important issues that concern many Canadians, mainly access to the system and cost reduction. Many Canadians believe the current public system is not sustainable and that a two-tiered system would force the public system to become more efficient and effective, given the competition of the private sector. However, the two-tiered system is not within the realm of consideration, since the majority of Canadians are opposed to the idea of a privatized system [ 5 ]. No proposals have come forward that show how a privately funded system would provide an equal quality of services for the same cost as the current publicly funded system.

United States of America

The health care system in the United States is currently plagued by three major challenges. These include: rapidly escalating health care costs, a large and growing number of Americans without health coverage and an epidemic of substandard care [ 15 ].

Health insurance premiums in the United States have been rising at accelerating rates. The premiums themselves, as well as the rate of increase in premiums, have increased every year since 1998; independent studies and surveys indicate that this trend is likely to continue over the next several years [ 15 ]. As a result of these increases, it is more difficult for businesses to provide health coverage to employees, with individuals and families finding it more difficult to pay their share of the cost of employer-sponsored coverage [ 15 ]. The rising trend in the cost of employer-sponsored family health coverage is illustrated in Figure 3 .

The trend of the cost of employer-sponsored family health care coverage in the United States . Source: National Coalition on Health Care 2004 pg.9. http://www.nchc.org/materials/studies/reform.pdf . Figure 3 illustrates the increase in health insurance premiums since 2001. These increases are making it more difficult for businesses to continue to provide health coverage for their employees and retirees [15].

To help resolve this problem, health maintenance organizations (HMO) have been introduced, with the goal of focusing on keeping people well and out of hospitals in the hope of decreasing employer costs. HMOs are popular alternatives to traditional health care plans offered by insurance companies because they can cover a wide variety of services, usually at a significantly lower cost [ 16 ]. HMOs use "networks" of selected doctors, hospitals, clinics and other health care providers that together provide comprehensive health services to the HMOs members [ 16 ]. The overall trade-off with an HMO is reduced choice in exchange for increased affordability.

Another problem to address regarding the American health care system is the considerable and increasing number of Americans without health coverage. Health care coverage programs such as Medicare offer a fee-for-service plan that covers many health care services and certain drugs. It also provides access to any doctor or hospital that accepts Medicare [ 17 ]. Patients with limited income and resources may qualify for Medicaid, which provide extra help paying for prescription drug costs [ 17 ]. However, according to figures from the United States Census Bureau, the number of Americans without health coverage grew to 43.6 million in 2002; it is predicted that the number of uninsured Americans will increase to between 51.2 and 53.7 million in 2006 [ 15 ].

Those Americans without health care insurance receive less care, receive care later and are, on average, less healthy and less able to function in their daily lives than those who have health care insurance. Additionally, the risk of mortality is 25% higher for the uninsured than for the insured [ 15 ].

Despite excellent care in some areas, the American health care system is experiencing an epidemic of substandard care; the system is not consistently providing high-quality care to its patients [ 15 ]. There appears to be a large discrepancy between the care patients should be receiving and the care they are actually getting. The Institute of Medicine has estimated that between 44 000 and 98 000 Americans die each year from preventable medical errors in hospitals [ 15 ].

It is also useful to examine the demographic characteristics of those Americans more likely to receive substandard care. Research shows that those Americans with little education and low income receive a lower standard of care [ 18 ]. This finding may be explained by the fact that patients who have lower education levels tend to have more difficulty explaining their concerns to physicians, as well as eliciting a response for those concerns because health professionals often do not value their opinions [ 18 ].

Case studies

As shown by the extensive literature, statistics and public opinion, there is a growing need for health care reform in the United States of America. There is a duty and responsibility of human resources professionals to attempt to elicit change and implement policies that will improve the health care system.

It is informative to examine case studies in which human resources professionals have enacted positive change in a health care setting. One such case from 1995 is that of a mid-sized, private hospital in the New York metropolitan area. This case presents a model of how human resources can be an agent for change and can partner with management to build an adaptive culture to maintain strong organizational growth [ 19 ].

One of the initiatives made by human resources professionals in an attempt to improve the overall standard of care in the hospital was to examine and shape the organization's corporate culture. Steps were taken to define the values, behaviors and competences that characterized the current culture, and analyze these against the desired culture [ 19 ]. A climate survey was conducted in the organization; it became the goal of the human resources professionals to empower employees to be more creative and innovative [ 19 ]. To achieve this, a new model of care was designed that emphasized a decentralized nursing staff and a team-based approach to patient care. Nursing stations were redesigned to make them more accessible and approachable [ 19 ].

Human resources management also played an important role in investing in employee development. This was achieved by assisting employees to prepare and market themselves for internal positions and if desired, helping them pursue employment opportunities outside the organization [ 19 ]. This case makes obvious the important roles that human resources management can play in orchestrating organizational change.

Another case study that illustrates the importance of human resources management to the health care system is that of The University of Nebraska Medical Center in 1995. During this period, the hospital administrative staff recognized a variety of new challenges that were necessitating organizational change. Some of these challenges included intense price competition and payment reform in health care, reduced state and federal funding for education and research, and changing workforce and population demographics [ 20 ]. The organizational administrators recognized that a cultural reformation was needed to meet these new challenges. A repositioning process was enacted, resulting in a human resources strategy that supported the organization's continued success [ 20 ]. This strategy consisted of five major objectives, each with a vision statement and series of action steps.

Staffing: Here, the vision was to integrate a series of organization-wide staffing strategies that would anticipate and meet changing workforce requirements pertaining to staff, faculty and students. To achieve this vision, corporate profiles were developed for each position to articulate the core competences and skills required [ 20 ].

Performance management: The vision was to hold all faculty and staff accountable and to reward individual and team performance. With this strategy, managers would be able to provide feedback and coaching to employees in a more effective and timely manner [ 20 ].

Development and learning: The vision was to have all individuals actively engaged in the learning process and responsible for their own development. Various unit-based training functions were merged into a single unit, which defined critical technical and behavioral competencies [ 20 ].

Valuing people: The vision was to have the hospital considered as a favored employer and to be able to attract and retain the best talent. To facilitate this vision, employee services such as child care and wellness were expanded [ 20 ].

Organizational effectiveness. The vision was to create an organization that is flexible, innovative and responsive [ 20 ]. The developments of these human resources strategies were essential to the effectiveness of the organization and to demonstrate the importance of human resources in the health care industry.

Both these case studies illustrate that effective human resources management is crucial to health care in a practical setting and that additional human resources initiatives are required if solutions are to be found for the major problems in the United States health care system.

Approximately 92% of Germany's population receives health care through the country's statutory health care insurance program, Gesetzliche Krankenversicherung (GKV). GKV designed an organizational framework for health care in Germany and has identified and constructed the roles of payers, providers and hospitals. Private, for-profit companies cover slightly less than 8% of the population. This group would include, for example, civil servants and the self-employed. It is estimated that approximately 0.2% of the population does not have health care insurance [ 21 ]. This small fragment may be divided into two categories: either the very rich, who do not require it, or the very poor, who obtain their coverage through social insurance. All Germans, regardless of their coverage, use the same health care facilities. With these policies nearly all citizens are guaranteed access to high-quality medical care [ 22 ].

While the federal government plays a major part in setting the standards for national health care policies, the system is actually run by national and regional autonomous organizations. Rather than being financed solely through taxes, the system is covered mostly by health care premiums [ 22 ]. In 2003, about 11.1% of Germany's gross domestic product (GDP) went into the health care system [ 23 ] versus the United States, with 15% [ 24 ] and Canada at 9.9% [ 25 ]. However, Germany still put about one third of its social budget towards health care [ 22 ].

The supply of physicians in Germany is high, especially compared to the United States, and this is attributed largely to the education system. If one meets the academic requirements in Germany, the possibility to study medicine is legally guaranteed [ 26 ]. This has led to a surplus of physicians and unemployment for physicians has become a serious problem. In 2001, the unemployment rate for German physicians of 2.1% led many German doctors to leave for countries such as Norway, Sweden and the United Kingdom, all of which actively recruit from Germany [ 27 ].

Germany's strong and inexpensive academic system has led the country to educate far more physicians than the United States and Canada. In 2003, Germany had 3.4 practicing physicians per 1000 inhabitants [ 23 ], versus the United States, which had 2.3 practicing physicians per 1000 inhabitants in 2002 [ 24 ] and Canada, which had 2.1 practicing physicians per 1000 inhabitants in 2003 [ 25 ]. It is also remarkable that health spending per capita in Germany (USD 2996) [ 23 ] amounted to about half of health spending per capita in the United States (USD 5635) [ 24 ], and slightly less than Canada's health spending (USD 3003) [ 25 ]. This clearly demonstrates the Germans' strength regarding cost containment.

There are several issues that physicians face in the German health care system. In a 1999 poll, 49.9% of respondents said they were very or fairly satisfied with their health care system, while 47.7% replied they were very or fairly dissatisfied with it [ 28 ]. Furthermore, the degree of competition between physicians is very high in Germany and this could lead to a reduction in physician earnings. Due to this competition, many younger physicians currently face unemployment. The German law also limits the number of specialists in certain geographical areas where there are issues of overrepresentation [ 22 ]. Thus, the oversupply of physicians in Germany leads to many challenges, including human resources management in the health care system.

In Germany a distinction is made between office-based physicians and hospital-based physicians. The income of office-based physicians is based on the number and types of services they provide, while hospital-based physicians are compensated on a salary basis. This division has created a separated workforce that German legislation is now working to eliminate by encouraging the two parties to work together, with the aim of reducing overall medical costs [ 22 ].

Developing countries

Accessing good-quality health care services can be incredibly arduous for those living in developing countries, and more specifically, for those residing in rural areas. For many reasons, medical personnel and resources may not be available or accessible for such residents. As well, the issue of migrant health care workers is critical. Migrant health workers can be defined as professionals who have a desire and the ability to leave the country in which they were educated and migrate to another country. The workers are generally enticed to leave their birth country by generous incentive offers from the recruiting countries [ 29 ].

Developing countries struggle to find means to improve living conditions for their residents; countries such as Ghana, Kenya, South Africa and Zimbabwe are seeking human resources solutions to address their lack of medically trained professionals. Shortages in these countries are prevalent due to the migration of their highly educated and medically trained personnel.

Professionals tend to migrate to areas where they believe their work will be more thoroughly rewarded. The International Journal for Equity in Health (2003) suggested that those who work in the health care profession tend to migrate to areas that are more densely populated and where their services may be better compensated. Health care professionals look to areas that will provide their families with an abundance of amenities, including schools for their children, safe neighborhoods and relatives in close proximity. For medical professionals, the appeal of promotions also serves as an incentive for educating oneself further [ 30 ]. As one becomes more educated, the ability and opportunity to migrate increases and this can lead to a further exodus of needed health care professionals.

These compelling reasons tend to cause medical professionals to leave their less-affluent and less-developed areas and migrate to areas that can provide them with better opportunities. This has caused a surplus in some areas and a huge deficit in others. This epidemic can be seen in nations such as Nicaragua. Its capital city, Managua, holds only one fifth of the country's population, yet it employs almost 50% of the medically trained health care workers. The same situation can be found in other countries, such as Bangladesh, where almost one third of the available health personnel are employed "in four metropolitan districts where less than 15% of the population lives" [ 30 ]. Clearly this presents a problem for those living outside these metropolitan districts.

Other possible explanations put forth by Dussault and Franceschini, both of the Human Development Division of the World Bank Institute, include "management style, incentive and career structures, salary scales, recruitment, posting and retention practices" [ 31 ]. Salary scales can differ quite drastically between originating and destination countries, which are shown in Figures 4 and 5 . They also state that in developing countries the earning potential one would see in more affluent or populated urban areas is much higher than one would expect to earn in rural areas.

Ratio of nurse wages (PPP USD), destination country to source country . Source: Vujicic M, Zurn P, Diallo K, Orvill A, Dal Poz MR 2004. http://www.human-resources-health.com/content/2/1/3 . Figure 4 shows the difference between the wage in the source country and destination country for nurses. This difference is also known as the "wage premium" [29].

Ratio of physician wages (PPP USD), destination country to source country . Source: Vujicic M, Zurn P, Diallo K, Orvill A, Dal Poz MR 2004. http://www.human-resources-health.com/content/2/1/3 . Figure 5 shows the difference between the wage in the source country and destination country for physicians [29].

As more health professionals emigrate to urban areas, the workloads for those in the rural areas greatly increase. This leads to a domino effect, in that those in such dire situations look for areas where they may be able to find more satisfactory and less demanding working conditions [ 31 ]. Vujicic et al. (2004) summarizes numerous variables that influence the migration pattern and has created a formula to express their impact. It is possible to quantify the factors, and human resources professionals need to look at the costs and benefits of altering the factors so that the migration pattern is more favorable. This formula is expressed as the results shown in Table 1 , which shows the different reasons for one to migrate in terms of the popularity of a given reason.

There is a tendency for developed countries faced with decreasing numbers of nationally trained medical personnel to recruit already-trained individuals from other nations by enticing them with incentives. Zimbabwe has been particularly affected by this problem. In 2001, out of approximately 730 nursing graduates, more than one third (237) of them relocated to the United Kingdom [ 29 ]. This was a dramatic increase from 1997, when only 26 (approximately 6.2%) of the 422 nursing program graduates migrated to the United Kingdom [ 29 ]. This leads to the loss of skilled workers in developing countries and can be very damaging, since the education systems in developing countries are training individuals for occupations in the medical profession, yet are not able to retain them [ 29 ].

Countries that have the capacity to educate more people than necessary in order to meet their domestic demand have tried to counterbalance this problem by increasing their training quota. Vujicic et al. (2004) identify that "the Philippines has for many years trained more nurses than are required to replenish the domestic stock, in an effort to encourage migration and increase the level of remittance flowing back into the country" [ 29 ].

Developed countries attract internationally trained medical professionals for many reasons. To begin with, "political factors, concerns for security, domestic birth rates, the state of the economy and war (both at home and abroad)" [ 26 ] influence the number of people that will be allowed or recruited into a country. Also, due to the conditions of the labor market compared to the demand in developed countries, governments may make allowances to their strict policies regarding the type of and number of professionals they will allow into their country [ 29 ]. This can be seen in a Canadian example:

Canada maintains] a list of occupations within which employment vacancies [are] evident. Potential immigrants working in one of these [listed] occupations would have a much higher chance of being granted entry than if they worked in a non-listed occupation [ 29 ].

Though Canada attracts internationally trained medical professionals, those employment vacancies may not always be open. Although there may be up to 10 000 international medical graduates (IMG) in Canada, many are not legally allowed to practice. Many immigrants cannot afford the costs of retraining and may be forced to find a new job in a completely unrelated field, leaving their skills to go to waste [ 32 ]. In 2004, Ontario had between 2000 and 4000 IMGs looking for work in medical fields related to their training and background [ 33 ]. That year, IMG Ontario accepted 165 IMGs into assessment and training positions, which was a 50% increase over the last year, and a 600% increase from the 24 positions in 1999 [ 33 ].

Another appeal for developed countries with regard to foreign trained health care professionals is that they may be less of a financial burden to the host country than those trained domestically. This is because educational costs and the resources necessary for training are already taken care of by the international medical schools and governments [ 29 ]. Though these reasons may make recruiting foreign medical professionals seem appealing, there are still ongoing debates as to whether those trained outside the host country are equally qualified and culturally sensitive to the country to which they migrate. Developing countries are addressing these concerns by establishing health professional training programs similar to those in developed countries [ 29 ]. These practices can be seen in, "the majority of nursing programs in Bangladesh, the Philippines and South Africa [which] are based on curricula from United Kingdom or USA nursing schools" [ 29 ]. Because of these actions, those who are trained may be more likely to leave and use their skills where they will be recognized and more highly rewarded.

There are also ethical considerations when examining the practice of recruiting health care professionals, particularly if they are recruited from regions or countries where health care shortages already exist. The rights of individuals to move as they see fit may need to be balanced against the idea of the greater good of those left behind.

Due to the shortages, it has been found the level of health service in rural or poor areas has decreased, leading to lower quality and productivity of health services, closure of hospital wards, increased waiting times, reduced numbers of available beds for inpatients, diversion of emergency department patients and underuse of remaining personnel or substitution with persons lacking the required skills for performing critical interventions [ 30 ].

The article "Not enough here, too many there: understanding geographical imbalances in the distribution of the health workforce" (2003), states that a reduced number of health care workers in a given area has a direct effect on the life expectancy of its residents. For example, in the rural areas of Mexico, life expectancy is 55 years, compared to 71 years in the urban areas. Additionally, in "the wealthier, northern part of the country, infant mortality is 20/1000 as compared to more than 50/1000 in the poorer southern states" [ 31 ].

Globalization – a common thread

While the issues raised in this article are common to many countries, the approaches taken to address them may not be the same in each country. Factors affecting the approaches that can be taken, some of which have been raised, include demographics, resources and philosophical and political perspectives. However, an overarching issue that affects not only health care but many other areas is that of globalization itself.

Different countries have traditionally had different perspectives on health care that have influenced their approaches to health care delivery. In Canada for example, health care is considered a right; its delivery is defined by the five main principles of the Canada Health Act, which officially precludes a significant role for private delivery of essential services. In the United States, health care is treated more as another service that, while it should be accessible, is not considered a right. Therefore there is a much larger private presence in health care delivery the United States than there is in Canada. In other parts of the world, the approach to health care falls between these perspectives.

As the move towards globalization for many goods and services increases, countries will have to consider how this will affect their approaches to health care delivery. As mentioned earlier, there is already a degree of labor mobility within a country that affects the quality and availability of health care services. There is also already a degree of international mobility of health care workers, as shown by the number of workers recruited developed countries.

While the international mobility of labor is generally not as unencumbered as that for goods and capital, that may be changing as more and more regional free trade agreements are considered. Canada, the United States and Mexico have NAFTA (North American Free Trade Agreement), Europe has the EU (European Union) and talks are under way to consider expanding the NAFTA agreement to include Central and South America, to expand EU membership and to consider an Asian trading bloc including China and India.

If health care becomes a part of these new trade agreements, countries will be obliged to treat health care delivery according to the rules of the agreement. Using the NAFTA as an example, if health care is included, governments could not treat domestic providers more favorably than foreign firms wanting to deliver services. In Canada the concern is that it would mean the end of the Canada Health Act, since NAFTA would allow private, for-profit American or Mexican firms to open.

All five issues raised in this research would be affected by the increase in international trade agreements that included health care. Therefore, governments, health care providers and human resources professionals cannot ignore this important consideration and trend when examining solutions to the issues. Depending upon their relative negotiation strengths and positions, some countries may not benefit as much as others with these agreements.

For example, it is more likely that countries with well-developed private, for-profit, health care expertise, such as the United States, would expand into developing countries rather than the other way around. If there is an increased ability for labor mobility, then it is likely that health care professionals in the poorer, developing countries would move to where the opportunities are better. We already see this internally in the move from rural to urban centers; this would likely continue if the health care professionals had the opportunity to move out of country to where they could have greater financial rewards for their expertise.

When considering the countries examined in this paper, it is likely that Canada and the United States would initially be the two most likely to move towards a more integrated approach to health care delivery. There is already a trade agreement in place, many of the factors influencing health care are similar (demographics, training, level of economic development, geography, cultural factors) and they are currently each other's largest trading partners. While the current agreement, which includes Mexico, does not cover health care, there is pressure to broaden the agreement to include areas not currently covered. If this happens, human resources professionals will have to increase their understanding of what the new health care delivery realities could be. For example, if the move is more towards the Canadian example of a largely not-for-profit, mainly publicly-funded health care delivery system, then it will be more of an adjustment for the American professionals.

However, the likelihood of the Canadian approach to health care's being adopted in the United States is very slim. During the presidency of Bill Clinton, the government attempted to introduce a more universal health care delivery system, which failed completely. Even though there are over 40 million Americans with no health care coverage, the idea of a universal, publicly-funded system went nowhere. Also, within Canada there is increasing pressure to consider a more active role for private health care delivery. Therefore, it is more likely that Canadian health care and human resource professionals will have to adapt to a style more like the American, privately delivered, for-profit approach.

If this is the direction of change, human resources professionals in Canada will need to adjust how they approach the challenges and new realities. For instance, there would likely be an increased role for insurance companies and health maintenance organizations (HMO) as they move towards the managed care model of the United States. With an HMO approach, financial as well as health needs of the patients are considered when making medical decisions. An insured patient would select from the range of services and providers that his/her policy covers and approves. Human resources professionals would need to work with a new level of administration, the HMO, which currently does not exist to any significant degree in Canada.

As mentioned earlier, it is likely that developing countries would be receiving health care models and approaches from developed countries rather than the other way around. In particular, a country such as the United States that has a strong, private, for-profit approach already in place would likely be the source from which the health care models would be drawn. Therefore, health care, as well as human resources professionals in those countries, would also need to adapt to these new realities.

In Germany, where there is currently an oversupply of physicians, a move towards a more global approach to health care delivery, through increased trade agreements, could result in even more German health care professionals' leaving the country. The challenge to be addressed by human resource professionals within the German health care system in this situation would be to prevent, or slow, the loss of the best professionals to other countries. Spending public resources in educating professionals only to have significant numbers of them leave the country is not a financially desirable or sustainable situation for a country.

While examining health care systems in various countries, we have found significant differences pertaining to human resources management and health care practices. It is evident that in Canada, CHA legislation influences human resources management within the health care sector. Furthermore, the result of the debate on Canada's one-tier versus two-tier system may have drastic impacts on the management of human resources in health care. Additionally, due to a lack of Canadian trained health professionals, we have found that Canada and the United States have a tendency to recruit from developing countries such as South Africa and Ghana, in order to meet demand.

Examination of the relationship between health care in the United States and human resources management reveals three major problems: rapidly escalating health care costs, a growing number of Americans without health care coverage and an epidemic regarding the standard of care. These problems each have significant consequences for the well-being of individual Americans and will have devastating affects on the physical and psychological health and well-being of the nation as a whole.

The physical health of many Americans is compromised because these factors make it difficult for individuals to receive proper consultation and treatment from physicians. This can have detrimental effects on the mental state of the patient and can lead to large amounts of undue stress, which may further aggravate the physical situation.

Examining case studies makes it evident that human resources management can and does play an essential role in the health care system. The practices, policies and philosophies of human resources professionals are imperative in developing and improving American health care. The implication is that further research and studies must be conducted in order to determine additional resource practices that can be beneficial to all organizations and patients.

Compared to the United States, Canada and developing countries, Germany is in a special situation, given its surplus of trained physicians. Due to this surplus, the nation has found itself with a high unemployment rate in the physician population group. This is a human resources issue that can be resolved through legislation. Through imposing greater restrictive admissions criteria for medical schools in Germany, they can reduce the number of physicians trained. Accompanying the surplus problem is the legislative restriction limiting the number of specialists allowed to practice in geographical areas. These are two issues that are pushing German-trained physicians out of the country and thus not allowing the country to take full advantage of its national investment in training these professionals.

Developing countries also face the problem of investing in the training of health care professionals, thus using precious national resources, but losing many of their trained professionals to other areas of the world that are able to provide them with more opportunities and benefits. Human resources professionals face the task of attempting to find and/or retain workers in areas that are most severely affected by the loss of valuable workers.

Human resources management plays a significant role in the distribution of health care workers. With those in more developed countries offering amenities otherwise unavailable, chances are that professionals will be more enticed to relocate, thus increasing shortages in all areas of health care. Due to an increase in globalization, resources are now being shared more than ever, though not always distributed equally.

Human resources implications of the factors

While collectively the five main areas addressed in the article represent health care issues affecting and affected by human resources practices, they are not all equal in terms of their influence in each country. For instance, in Canada there are fewer health care issues surrounding the level of economic development or migration of health workers, whereas these issues are much more significant in developing countries. In the United States, the level of economic development is not a significant issue, but the accessibility of health care based upon an individual's financial situation certainly is, as evidenced by the more than 40 million Americans who have no health care coverage. Germany's issues with the size of its health care worker base have to do with too many physicians, whereas in Canada one of the issues is having too few physicians. Table 2 summarizes some of the implications for health care professionals with regard to the five main issues raised in the article. One of the main implications of this paper, as shown in Table 2 , is that HRP will have a vital role in addressing all the factors identified. Solutions to health care issues are not just medical in nature.

Policy approaches in a global approach to health care delivery

As mentioned at the start of this paper, there are three main health system inputs: human resources, physical capital and consumables. Given that with sufficient resources any country can obtain the same physical capital and consumables, it is clear that the main differentiating input is the human resources. This is the input that is the most difficult to develop, manage, motivate, maintain and retain, and this is why the role of the human resources professional is so critical.

The case studies described earlier showed how human resources initiatives aimed at improving organizational culture had a significant and positive effect on the efficiency and effectiveness of the hospitals studied. Ultimately all health care is delivered by people, so health care management can really be considered people management; this is where human resources professionals must make a positive contribution.

Human resource professionals understand the importance of developing a culture that can enable an organization to meet its challenges. They understand how communities of practice can form around common goals and interests, and the importance of aligning these to the goals and interests of the organization.

Given the significant changes that globalization of health care can introduce, it is important that human resources professionals be involved at the highest level of strategic planning, and not merely be positioned at the more functional, managerial levels. By being actively involved at the strategic levels, they can ensure that the HR issues are raised, considered and properly addressed.

Therefore, human resources professionals will also need to have an understanding not only of the HR area, but of all areas of an organization, including strategy, finance, operations, etc. This need will have an impact on the educational preparation as well as the possible need to have work experience in these other functional areas.

We have found that the relationship between human resources management and health care is extremely complex, particularly when examined from a global perspective. Our research and analysis have indicated that several key questions must be addressed and that human resources management can and must play an essential role in health care sector reform.

The various functions of human resources management in health care systems of Canada, the United States of America, Germany and various developing countries have been briefly examined. The goals and motivations of the main stakeholders in the Canadian health care system, including provincial governments, the federal government, physicians, nurses and allied health care professionals, have been reviewed. The possibility of a major change in the structure of Canadian health care was also explored, specifically with regard to the creation of a two-tier system. The American health care system is currently challenged by several issues; various American case studies were examined that displayed the role of human resources management in a practical setting. In Germany, the health care situation also has issues due to a surplus of physicians; some of the human resources implications of this issue were addressed. In developing countries, the migration of health workers to more affluent regions and/or countries is a major problem, resulting in citizens in rural areas of developing countries experiencing difficulties receiving adequate medical care.

Since all health care is ultimately delivered by and to people, a strong understanding of the human resources management issues is required to ensure the success of any health care program. Further human resources initiatives are required in many health care systems, and more extensive research must be conducted to bring about new human resources policies and practices that will benefit individuals around the world.

World Health Organization: World Health Report 2000. Health Systems: Improving Performance. Geneva. 2000, [ http://www.who.int.proxy.lib.uwo.ca:2048/whr/2000/en/whr00_ch4_en.pdf ]

Google Scholar  

World Health Organization: World Health Report 2003: Shaping the Future. Geneva. 2003, [ http://www.who.int.proxy.lib.uwo.ca:2048/whr/2003/en/Chapter7-en.pdf ]

Book   Google Scholar  

Zurn P, Dal Poz MR, Stilwell B, Adams O: Imbalance in the health workforce. Human Resources for Health. 2004, 2: 13-10.1186/1478-4491-2-13.

Article   PubMed   PubMed Central   Google Scholar  

Kirby MJL: The health of Canadians – the federal role. The Senate of the Government of Canada. 2002, Ottawa, ON: Government of Canada, 6: 78-

Makarenko J: The Mazankowski Report: A Diagnosis of Health Care in Canada. 2002, Edmonton, AB: Government of the Province of Alberta, [ http://www.mapleleafweb.com/features/medicare/mazankowski/index.html .]

Dosanjh U: Canada Health Act Report 2003–2004. 2004, Ottawa: Government of Canada, [ http://www.hcsc.gc.ca/medicare/Documents/CHAAR%202003-04.pdf ]

Ministry of Health and Long term Care: Report on the Integration of Primary and Health Care Nurse Practitioners into the Province of Ontario. 2005, Toronto, ON, [ http://www.health.gov.on.ca/english/public/pub/ministry_reports/nurseprac03/exec_summ.pdf ]

Canadian Health Coalition: The History of Medicare Shows that Canadians Can Do It. Ottawa, ON, [ http://www.healthcoalition.ca/history.html ]

Canadian Nurses Association: Succession Planning for Nursing Leadership. 2006, Ottawa, ON, [ http://www.cna-nurses.ca/CNA/practice/leadership/default_e.aspx ]

Hannah KJ: Health informatics and nursing in Canada. Healthcare Information Management and Communications Canada. 2005, 14: 3-[ http://hcccinc.qualitygroup.com/hcccinc2/pdf/Vol_XIX_No_3/Vol_XIX_No_3_7.pdf ]

Manojlovich M, Ketefian S: The effects of organizational culture on nursing professionalism: Implications for health resource planning. The Canadian Journal of Nursing Research. 2002, 33: 15-34.

CAS   PubMed   Google Scholar  

Health Canada: Investing in Health Care Providers. 2003, Ottawa, ON, [ http://www.hc-sc.gc.ca/english/pdf/romanow/pdfs/HCC_Chapter_4.pdf ]

Romanow RJ: Building on Values: The Future of Health Care in Canada. 2002, Commission on the Future of Health Care in Canada. Ottawa, ON

Kirby MJL: The Health of Canadians – The Federal Role. 2002, The Senate of the Government of Canada. Ottawa, ON: Government of Canada, 4: 111-

National Coalition on Health Care: Building a Better Health Care System: Specifications for Reform. Report from the National Coalition on Health Care. Washington, DC. 2004, 5-12. [ http://www.nchc.org/materials/studies/reform.pdf ]

The Texas Department of Insurance: Health Maintenance Organizations. 2005, Austin, TX, [ http://www.tdi.state.tx.us/consumer/cbo69.html ]

Centers for Medicare and Medicaid Services: Medicare and You. 2006, Baltimore, Maryland, [ http://www.medicare.gov/publications/pubs/pdf/10050.pdf ]

Malat J: Social distance and patient's ratings of health care providers. Journal of Health and Social Behavior. 2001, 42: 360-72. 10.2307/3090184.

Article   CAS   PubMed   Google Scholar  

Anson BR: Taking charge in a volatile health care marketplace. Human Resource Planning. 2003, 23 (4): 21-34.

Jones DA: Repositioning human resources: a case study. Human Resources Planning. 1996, 19 (1): 51-54.

World Health Organization Regional Office for Europe: Highlights on Health. Germany. Copenhagen. 2004, [ http://www.euro.who.int/eprise/main/who/progs/chhdeu/system/20050311_1 ]

The Library of Congress: A Country Study: Germany. Washington, DC. 1995, [ http://lcweb2.loc.gov/frd/cs/detoc.html ]

Organization for Economic Co-operation and Development: OECD Health Data 2005. How Does Germany Compare. Paris. 2005, [ http://www.oecd.org/dataoecd/16/6/34970073.pdf ]

Organization for Economic Co-operation and Development: OECD Health Data 2005. How Does the United States Compare. Paris. 2005, [ http://www.oecd.org/dataoecd/15/23/34970246.pdf ]

Organization for Economic Co-operation and Development: OECD Health Data 2005. How Does Canada Compare. Paris. 2005, [ http://www.oecd.org/dataoecd/16/9/34969633.pdf ]

Bundesministerium für Gesundheit: Information on Medical Training in the Federal Republic of Germany. 2005, Kohn, GDR, [ http://www.bmg.bund.de/cln_041/nn_617014/EN/Health/health-node,param=.html_nnn=true ]

Medknowledge: Working Formalities for Foreign Physicians in Germany. Munster. 2000, [ http://www.medknowledge.de/germany/ ]

National Coalition on Health Care: Health Care in Germany. Washington, DC. 1999, [ http://www.nchc.org/facts/Germany.pdf ]

Vujicic M, Zurn P, Diallo K, Orvill A, Dal Poz MR: The role of wages in the migration of health care professionals from developing countries. Human Resources for Health. 2004, 2: 3-10.1186/1478-4491-2-3. [ http://www.human-resources-health.com/content/2/1/3 ]

Gupta N, Zurn P, Diallo K, Dal Poz MR: Uses of population census data for monitoring geographical imbalance in the health workforce: snapshots from three developing countries. International Journal for Equity in Health. 2003, 2: 11-10.1186/1475-9276-2-11. [ http://www.equityhealthj.com/content/2/1/11 ]

Dussault G, Franceschini M: Not enough here, too many there: understanding geographical imbalances in the distribution of the health workforce. Washington, DC: The World Bank Institute. 2003, [ http://www.lachsr.org/observatorio/eng/pdfs/Geographical Imbalances05-13-03.pdf ]

Findlay J: Doctors with Borders: Struggles Facing Foreign Physicians in Canada. 2005, New Media Journalism. University of Western Ontario. London, ON, [ http://www.fims.uwo.ca/newmedia2005/default.asp?id=166 ]

Findlay J: Facts on Foreign Doctors. 2005, New Media Journalism, University of Western Ontario. London, ON, [ http://www.fims.uwo.ca/newmedia2005/default.asp?id=175 ]

Barney J: Gaining and Sustaining Competitive Advantage. 1997, Reading, MASS: Addison-Wesley Publishing Co.

Download references

Acknowledgements

The authors are grateful to Valerie Sloby from PCHealthcare for her editorial assistance and helping in reviewing the manuscript.

Author information

Authors and affiliations.

Management and Organizational Studies, The University of Western Ontario, London, Ontario, Canada

Stefane M Kabene, Mark A Soriano & Raymond Leduc

Schulich School of Medicine, The University of Western Ontario, London, Ontario, Canada

John M Howard

School of Nursing, The University of Western Ontario, London, Ontario, Canada

Stefane M Kabene & Carole Orchard

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Stefane M Kabene .

Additional information

Competing interests.

The author(s) declare that they have no competing interests.

Authors' contributions

SK conceived the paper, worked on research design, did data analysis and led the writing of the paper. CO, JH, MS and RL all actively participated in data analysis, manuscript writing and review. All authors read and approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2, authors’ original file for figure 3, authors’ original file for figure 4, authors’ original file for figure 5, rights and permissions.

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Kabene, S.M., Orchard, C., Howard, J.M. et al. The importance of human resources management in health care: a global context. Hum Resour Health 4 , 20 (2006). https://doi.org/10.1186/1478-4491-4-20

Download citation

Received : 13 April 2006

Accepted : 27 July 2006

Published : 27 July 2006

DOI : https://doi.org/10.1186/1478-4491-4-20

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Human Resource
• Health Care System
• Gross Domestic Product
• Human Resource Management
• Health Care Delivery

Human Resources for Health

ISSN: 1478-4491

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

New study highlights the benefit of touch on mental and physical health

Through a large-scale analysis, researchers at the Netherlands Institute for Neuroscience have uncovered the ways in which consensual touch can benefit a person's physical and mental wellbeing.

You might recognize the comforting feeling when someone offers you a hug at the end of a stressful day or strokes your shoulder when you're feeling down. But the question remains: can touch really help you feel better, and does it matter who it's from or how they touch you? To explore these questions, researchers from the Social Brain Lab at the Netherlands Institute for Neuroscience and the University Hospital Essen conducted a large-scale analysis of studies exploring touch interventions.

The benefits of touch on mental and physical health

Does touch truly improve someone's wellbeing? It is an easy question to ask but more complicated to answer. Individual studies often only focus on specific instances and may contradict each other. Combining all these studies together for a large-scale analysis offers a clearer answer: yes, touch substantially improves both physical and mental wellbeing, for example via reduction of pain, anxiety, depression, and stress in adults. But in fact, those with physical or mental health problems (and therefore most in need of support) benefit even more from touch than healthy adults. "This is especially relevant considering how often touch interventions are overlooked" Packheiser, first author, adds.

"A key question of our study is to leverage the hundreds of individual studies out there to identify what type of touch works best," adds professor Keysers, director of the Social Brain Lab. "What if you don't have a friend or partner close by to hug you? Would touch from a stranger or even a machine also help? And how often? The study clearly shows that touch can indeed be optimized, but the most important factors are not necessarily those we suspect."

Interestingly, the person touching you, how they touch you, and the duration of their touch doesn't make a difference in terms of impact. A long-lasting massage by a therapist could therefore be just as effective as a quick hug offered by a friend. That is, until the frequency of the intervention is considered. The more often a touch intervention is offered, the greater the impact. A quick hug could therefore be even more impactful than a massage if it is offered more frequently.

Human or non-human touch?

The next question was whether touch intervention needs to be human at all. As it turns out, object or robot interventions can be equally effective at improving physical wellbeing. "There are lots of people in need of wellbeing improvements, perhaps because they're lonely but also because they may be inflicted by clinical conditions. These results indicate that a touch-robot, or even a simple weighted blanket has the potential to help those people," last author Frédéric Michon explains. However, the benefits of robot and object interventions are less effective for mental wellbeing. Mental health disorders like anxiety or depression might therefore require human touch after all, "perhaps suggestive of the importance for an emotional component associated with the touch," Michon point out.

While the researchers were equally curious about human-to-animal contact, studies exploring this question are still lacking. "It would be useful to see whether an animal's or pet's touch could improve wellbeing, and inversely if they also benefit from it, but unfortunately there simply aren't enough studies, or properly controlled ones, for us to draw any general conclusions on these topics," Michon clarifies.

Touch interventions across ages

When the team looked into the impact of touch on newborns, they found out that newborns also benefited significantly from touch. However, the person conducting the touch intervention was more important: the benefits of touch are higher when done by a parent instead of a healthcare worker. "This finding could be impactful," Packheiser adds. "Death rates due to premature births are high in some countries and the knowledge that a baby benefits more from the touch of their own parent offers another easily implementable form of support for the baby's health."

Due to a lack of studies, it proved difficult to draw conclusions about children and teenagers. "Large scale studies like this help us draw more general conclusions but they also help us identify where research is lacking," Michon explains. "We hope that our findings can steer future research to explore lesser-known questions. This includes animal touch, but also touch across ages, and in specific clinical settings like autistic patients, another category that has not been explored extensively."

• Alternative Medicine
• Mental Health Research
• Chronic Illness
• Mental Health
• Disorders and Syndromes
• Evidence-based medicine
• Hyperactivity
• Philosophy of mind
• Psychotherapy
• Psychiatric service dog

Story Source:

Materials provided by Netherlands Institute for Neuroscience - KNAW . Note: Content may be edited for style and length.

Journal Reference :

• Julian Packheiser, Helena Hartmann, Kelly Fredriksen, Valeria Gazzola, Christian Keysers, Frédéric Michon. A systematic review and multivariate meta-analysis of the physical and mental health benefits of touch interventions . Nature Human Behaviour , 2024; DOI: 10.1038/s41562-024-01841-8

Cite This Page :

Explore More

• Quantum Effects in Electron Waves
• Star Trek's Holodeck Recreated Using ChatGPT
• Cloud Engineering to Mitigate Global Warming
• Detecting Delayed Concussion Recovery
• Genes for Strong Muscles: Healthy Long Life
• Brightest Gamma-Ray Burst
• Stellar Winds of Three Sun-Like Stars Detected
• Fences Causing Genetic Problems for Mammals
• Ozone Removes Mating Barriers Between Fly ...
• Parkinson's: New Theory On Origins and Spread

Trending Topics

Strange & offbeat.

This paper is in the following e-collection/theme issue:

Published on 11.4.2024 in Vol 26 (2024)

Patients’ Experiences With Digitalization in the Health Care System: Qualitative Interview Study

Authors of this article:

Original Paper

• Christian Gybel Jensen 1 * , MA   ; 
• Frederik Gybel Jensen 1 * , MA   ; 
• Mia Ingerslev Loft 1, 2 * , MSc, PhD  

1 Department of Neurology, Rigshospitalet, Copenhagen, Denmark

2 Institute for People and Technology, Roskilde University, Roskilde, Denmark

*all authors contributed equally

Corresponding Author:

Mia Ingerslev Loft, MSc, PhD

Department of Neurology

Rigshospitalet

Inge Lehmanns Vej 8

Phone: 45 35457076

Email: [email protected]

Background: The digitalization of public and health sectors worldwide is fundamentally changing health systems. With the implementation of digital health services in health institutions, a focus on digital health literacy and the use of digital health services have become more evident. In Denmark, public institutions use digital tools for different purposes, aiming to create a universal public digital sector for everyone. However, this digitalization risks reducing equity in health and further marginalizing citizens who are disadvantaged. Therefore, more knowledge is needed regarding patients’ digital practices and experiences with digital health services.

Objective: This study aims to examine digital practices and experiences with public digital health services and digital tools from the perspective of patients in the neurology field and address the following research questions: (1) How do patients use digital services and digital tools? (2) How do they experience them?

Methods: We used a qualitative design with a hermeneutic approach. We conducted 31 semistructured interviews with patients who were hospitalized or formerly hospitalized at the department of neurology in a hospital in Denmark. The interviews were audio recorded and subsequently transcribed. The text from each transcribed interview was analyzed using manifest content analysis.

Results: The analysis provided insights into 4 different categories regarding digital practices and experiences of using digital tools and services in health care systems: social resources as a digital lifeline, possessing the necessary capabilities, big feelings as facilitators or barriers, and life without digital tools. Our findings show that digital tools were experienced differently, and specific conditions were important for the possibility of engaging in digital practices, including having access to social resources; possessing physical, cognitive, and communicative capabilities; and feeling motivated, secure, and comfortable. These prerequisites were necessary for participants to have positive experiences using digital tools in the health care system. Those who did not have these prerequisites experienced challenges and, in some cases, felt left out.

Conclusions: Experiences with digital practices and digital health services are complex and multifaceted. Engagement in digital practices for the examined population requires access to continuous assistance from their social network. If patients do not meet requirements, digital health services can be experienced as exclusionary and a source of concern. Physical, cognitive, and communicative difficulties might make it impossible to use digital tools or create more challenges. To ensure that digitalization does not create inequities in health, it is necessary for developers and institutions to be aware of the differences in digital health literacy, focus on simplifying communication with patients and next of kin, and find flexible solutions for citizens who are disadvantaged.

Introduction

In 2022, the fourth most googled question in Denmark was, “Why does MitID not work?” [ 1 ]. MitID (My ID) is a digital access tool that Danes use to enter several different private and public digital services, from bank accounts to mail from their municipality or the state. MitID is a part of many Danish citizens’ everyday lives because the public sector in Denmark is digitalized in many areas. In recent decades, digitalization has changed how governments and people interact and has demonstrated the potential to change the core functions of public sectors and delivery of public policies and services [ 2 ]. When public sectors worldwide become increasingly digitalized, this transformation extends to the public health sectors as well, and some studies argue that we are moving toward a “digital public health era” that is already impacting the health systems and will fundamentally change the future of health systems [ 3 ]. While health systems are becoming more digitalized, it is important that both patients and digitalized systems adapt to changes in accordance with each other. Digital practices of people can be understood as what people do with and through digital technologies and how people relate to technology [ 4 ]. Therefore, it is relevant to investigate digital practices and how patients perceive and experience their own use of digital tools and services, especially in relation to existing digital health services. In our study, we highlight a broad perspective on experiences with digital practices and particularly add insight into the challenges with digital practices faced by patients who have acute or chronic illness, with some of them also experiencing physical, communicative, or cognitive difficulties.

An international Organization for Economic Cooperation and Development report indicates that countries are digitalized to different extents and in different ways; however, this does not mean that countries do not share common challenges and insights into the implementation of digital services [ 2 ].

In its global Digital Government Index, Denmark is presented as one of the leading countries when it comes to public digitalization [ 2 ]. Recent statistics indicate that approximately 97% of Danish families have access to the internet at home [ 5 ]. The Danish health sector already offers many different digital services, including web-based delivery of medicine, e-consultations, patient-related outcome questionnaires, and seeking one’s own health journal or getting test results through; “Sundhed” [ 6 ] (the national health portal) and “Sundhedsjournalen” (the electronic patient record); or the apps “Medicinkortet” (the shared medication record), “Minlæge” (My Doctor, consisting of, eg, communication with the general practitioner), or “MinSP” (My Health Platform, consisting of, eg, communication with health care staff in hospitals) [ 6 - 8 ].

The Danish Digital Health Strategy from 2018 aims to create a coherent and user-friendly digital public sector for everyone [ 9 ], but statistics indicate that certain groups in society are not as digitalized as others. In particular, the older population uses digital services the least, with 5% of people aged 65 to 75 years and 18% of those aged 75 to 89 years having never used the internet in 2020 [ 5 ]. In parts of the literature, it has been problematized how the digitalization of the welfare state is related to the marginalization of older citizens who are socially disadvantaged [ 10 ]. However, statistics also indicate that the probability of using digital tools increases significantly as a person’s experience of using digital tools increases, regardless of their age or education level [ 5 ].

Understanding the digital practices of patients is important because they can use digital tools to engage with the health system and follow their own health course. Researching experiences with digital practices can be a way to better understand potential possibilities and barriers when patients use digital health services. With patients becoming more involved in their own health course and treatment, the importance of patients’ health literacy is being increasingly recognized [ 11 ]. The World Health Organization defines health literacy as the “achievement of a level of knowledge, personal skills and confidence to take action to improve personal and community health by changing personal lifestyles and living conditions” [ 12 ]. Furthermore, health literacy can be described as “a person’s knowledge and competencies to meet complex demands of health in modern society, ” and it is viewed as a critical step toward patient empowerment [ 11 , 12 ]. In a digitalized health care system, this also includes the knowledge, capabilities, and resources that individuals require to use and benefit from eHealth services, that is, “digital health literacy (eHealth literacy)” [ 13 ]. An eHealth literacy framework created by Norgaard et al [ 13 ] identified that different aspects, for example, the ability to process information and actively engage with digital services, can be viewed as important facets of digital health literacy. This argument is supported by studies that demonstrate how patients with cognitive and communicative challenges experience barriers to the use of digital tools and require different approaches in the design of digital solutions in the health sector [ 14 , 15 ]. Access to digital services and digital literacy is becoming increasingly important determinants of health, as people with digital literacy and access to digital services can facilitate improvement of health and involvement in their own health course [ 16 ].

The need for a better understanding of eHealth literacy and patients’ capabilities to meet public digital services’ demands as well as engage in their own health calls for a deeper investigation into digital practices and the use of digital tools and services from the perspective of patients with varying digital capabilities. Important focus areas to better understand digital practices and related challenges have already been highlighted in various studies. They indicate that social support, assessment of value in digital services, and systemic assessment of digital capabilities are important in the use and implementation of digital tools, and they call for better insight into complex experiences with digital services [ 13 , 17 , 18 ]. Therefore, we aimed to examine digital practices and experiences with public digital health services and digital tools from the perspective of patients, addressing the following research questions: how do patients use digital services and digital tools, and how do they experience them?

We aimed to investigate digital practices and experiences with digital health services and digital tools; therefore, we used a qualitative design and adopted a hermeneutic approach as the point of departure, which means including preexisting knowledge of digital practices but also providing room for new comprehension [ 19 ]. Our interpretive approach is underpinned by the philosophical hermeneutic approach by Gadamer et al [ 19 ], in which they described the interpretation process as a “hermeneutic circle,” where the researcher enters the interpretation process with an open mind and historical awareness of a phenomenon (preknowledge). We conducted semistructured interviews using an interview guide. This study followed the COREQ (Consolidated Criteria for Reporting Qualitative Research) checklist [ 20 ].

Setting and Participants

To gain a broad understanding of experiences with public digital health services, a purposive sampling strategy was used. All 31 participants were hospitalized or formerly hospitalized patients in a large neurological department in the capital of Denmark ( Table 1 ). We assessed whether including patients from the neurological field would give us a broad insight into the experiences of digital practices from different perspectives. The department consisted of, among others, 8 inpatient units covering, for example, acute neurology and stroke units, from which the patients were recruited. Patients admitted to a neurological department can have both acute and transient neurological diseases, such as infections in the brain, stroke, or blood clot in the brain from which they can recover completely or have persistent physical and mental difficulties, or experience chronic neurological and progressive disorders such as Parkinson disease and dementia. Some patients hospitalized in neurological care will have communicative and cognitive difficulties because of their neurological disorders. Nursing staff from the respective units helped the researchers (CGJ, FGJ, and MIL) identify patients who differed in terms of gender, age, and severity of neurological illness. Some patients (6/31, 19%) had language difficulties; however, a speech therapist assessed them as suitable participants. We excluded patients with severe cognitive difficulties and those who were not able to speak the Danish language. Including patients from the field of neurology provided an opportunity to study the experience of digital health practice from various perspectives. Hence, the sampling strategy enabled the identification and selection of information-rich participants relevant to this study [ 21 ], which is the aim of qualitative research. The participants were invited to participate by either the first (CGJ) or last author (MIL), and all invited participants (31/31, 100%) chose to participate.

All 31 participants were aged between 40 to 99 years, with an average age of 71.75 years ( Table 1 ). Out of the 31 participants, 10 (32%) had physical disabilities or had cognitive or communicative difficulties due to sequela in relation to neurological illness or other physical conditions.

Data Collection

The 31 patient interviews were conducted over a 2-month period between September and November 2022. Of the 31 patients, 20 (65%) were interviewed face-to-face at the hospital in their patient room upon admission and 11 (35%) were interviewed on the phone after being discharged. The interviews had a mean length of 20.48 minutes.

We developed a semistructured interview guide ( Table 2 ). The interview questions were developed based on the research aim, findings from our preliminary covering of literature in the field presented in the Introduction section, and identified gaps that we needed to elaborate on to be able to answer our research question [ 22 ]. The semistructured interview guide was designed to support the development of a trusting relationship and ensure the relevance of the interviews’ content [ 22 ]. The questions served as a prompt for the participants and were further supported by questions such as “please tell me more” and “please elaborate” throughout the interview, both to heighten the level of detail and to verify our understanding of the issues at play. If the participant had cognitive or communicative difficulties, communication was supported using a method called Supported Communication for Adults with Aphasia [ 23 ] during the interview.

The interviews were performed by all authors (CGJ, FGJ, and MIL individually), who were skilled in conducting interviews and qualitative research. The interviewers are not part of daily clinical practice but are employed in the department of neurology from where the patients were recruited. All interviews were audio recorded and subsequently transcribed verbatim by all 3 authors individually.

a PRO: patient-related outcome.

Data Analysis

The text from each transcribed interview was analyzed using manifest content analysis, as described by Graneheim and Lundman [ 24 ]. Content analysis is a method of analyzing written, verbal, and visual communication in a systematic way [ 25 ]. Qualitative content analysis is a structured but nonlinear process that requires researchers to move back and forth between the original text and parts of the text during the analysis. Manifest analysis is the descriptive level at which the surface structure of the text central to the phenomenon and the research question is described. The analysis was conducted as a collaborative effort between the first (CGJ) and last authors (MIL); hence, in this inductive circular process, to achieve consistency in the interpretation of the text, there was continued discussion and reflection between the researchers. The transcriptions were initially read several times to gain a sense of the whole context, and we analyzed each interview. The text was initially divided into domains that reflected the lowest degree of interpretation, as a rough structure was created in which the text had a specific area in common. The structure roughly reflected the interview guide’s themes, as guided by Graneheim and Lundman [ 24 ]. Thereafter, the text was divided into meaning units, condensed into text-near descriptions, and then abstracted and labeled further with codes. The codes were categorized based on similarities and differences. During this process, we discussed the findings to reach a consensus on the content, resulting in the final 4 categories presented in this paper.

Ethical Considerations

The interviewees received oral and written information about the study and its voluntary nature before the interviews. Written informed consent was obtained from all participants. Participants were able to opt of the study at any time. Data were anonymized and stored electronically on locked and secured servers. The Ethics Committee of the Capitol Region in Denmark was contacted before the start of the study. This study was registered and approved by the ethics committee and registered under the Danish Data Protection Agency (number P2021-839). Furthermore, the ethical principles of the Declaration of Helsinki were followed for this study.

The analysis provided insights into 4 different categories regarding digital practices and experiences of using digital tools and services in health care systems: social resources as a digital lifeline, possessing the necessary capabilities, big feelings as facilitators or barriers, and life without digital tools.

Social Resources as a Digital Lifeline

Throughout the analysis, it became evident that access to both material and social resources was of great importance when using digital tools. Most participants already possessed and had easy access to a computer, smartphone, or tablet. The few participants who did not own the necessary digital tools told us that they did not have the skills needed to use these tools. For these participants, the lack of material resources was tied particularly to a lack of knowledge and know-how, as they expressed that they would not know where to start after buying a computer—how to set it up, connect it to the internet, and use its many systems.

However, possessing the necessary material resources did not mean that the participants possessed the knowledge and skill to use digital tools. Furthermore, access to material resources was also a question of having access to assistance when needed. Some participants who had access to a computer, smartphone, and tablet and knew how to use these tools still had to obtain help when setting up hardware, updating software, or getting a new device. These participants were confident in their own ability to use digital devices but also relied on family, friends, and neighbors in their everyday use of these tools. Certain participants were explicitly aware of their own use of social resources when expressing their thoughts on digital services in health care systems:

I think it is a blessing and a curse. I think it is both. I would say that if I did not have someone around me in my family who was almost born into the digital world, then I think I would be in trouble. But I feel sorry for those who do not have that opportunity, and I know quite a few who do not. They get upset, and it’s really frustrating. [Woman, age 82 years]

The participants’ use of social resources indicates that learning skills and using digital tools are not solely individual tasks but rather continuously involve engagement with other people, particularly whenever a new unforeseen problem arises or when the participants want a deeper understanding of the tools they are using:

If tomorrow I have to get a new ipad...and it was like that when I got this one, then I had to get XXX to come and help me move stuff and he was sweet to help with all the practical stuff. I think I would have cursed a couple of times (if he hadn’t been there), but he is always helpful, but at the same time he is also pedagogic so I hope that next time he showed me something I will be able to do it. [Man, age 71 years]

For some participants, obtaining assistance from a more experienced family member was experienced as an opportunity to learn, whereas for other participants, their use of public digital services was even tied directly to assistance from a spouse or family member:

My wife, she has access to mine, so if something comes up, she can just go in and read, and we can talk about it afterwards what (it is). [Man, age 85 years]

The participants used social resources to navigate digital systems and understand and interpret communication from the health care system through digital devices. Another example of this was the participants who needed assistance to find, answer, and understand questionnaires from the health care department. Furthermore, social resources were viewed as a support system that made participants feel more comfortable and safer when operating digital tools. The social resources were particularly important when overcoming unforeseen and new challenges and when learning new skills related to the use of digital tools. Participants with physical, cognitive, and communicative challenges also explained how social resources were of great importance in their ability to use digital tools.

Possessing the Necessary Capabilities

The findings indicated that possessing the desire and knowing how to use digital tools are not always enough to engage with digital services successfully. Different health issues can carry consequences for motor skills and mobility. Some of these consequences were visibly affecting how our participants interacted with digital devices, and these challenges were somewhat easy to discover. However, our participants revealed hidden challenges that posed difficulties. In some specific cases, cognitive and communicative inabilities can make it difficult to use digital tools, and this might not always be clear until the individual tries to use a device’s more complex functions. An example of this is that some participants found it easy to turn on a computer and use it to write but difficult to go through security measures on digital services or interpret and understand digital language. Remembering passwords and logging on to systems created challenges, particularly for those experiencing health issues that directly affect memory and cognitive abilities, who expressed concerns about what they were able to do through digital tools:

I think it is very challenging because I would like to use it how I used to before my stroke; (I) wish that everything (digital skills) was transferred, but it just isn’t. [Man, age 80 years]

Despite these challenges, the participants demonstrated great interest in using digital tools, particularly regarding health care services and their own well-being. However, sometimes, the challenges that they experienced could not be conquered merely by motivation and good intentions. Another aspect of these challenges was the amount of extra time and energy that the participants had to spend on digital services. A patient diagnosed with Parkinson disease described how her symptoms created challenges that changed her digital practices:

Well it could for example be something like following a line in the device. And right now it is very limited what I can do with this (iPhone). Now I am almost only using it as a phone, and that is a little sad because I also like to text and stuff, but I also find that difficult (...) I think it is difficult to get an overview. [Woman, age 62 years]

Some participants said that after they were discharged from the hospital, they did not use the computer anymore because it was too difficult and too exhausting , which contributed to them giving up . Using digital tools already demanded a certain amount of concentration and awareness, and some diseases and health conditions affected these abilities further.

Big Feelings as Facilitators or Barriers

The findings revealed a wide range of digital practices in which digital tools were used as a communication device, as an entertainment device, and as a practical and informative tool for ordering medicine, booking consultations, asking health-related questions, or receiving email from public institutions. Despite these different digital practices, repeating patterns and arguments appeared when the participants were asked why they learned to use digital tools or wanted to improve their skills. A repeating argument was that they wanted to “follow the times, ” or as a participant who was still not satisfied with her digital skills stated:

We should not go against the future. [Woman, age 89 years]

The participants expressed a positive view of the technological developments and possibilities that digital devices offered, and they wanted to improve their knowledge and skills related to digital practice. For some participants, this was challenging, and they expressed frustration over how technological developments “moved too fast ,” but some participants interpreted these challenges as a way to “keep their mind sharp. ”

Another recurring pattern was that the participants expressed great interest in using digital services related to the health care system and other public institutions. The importance of being able to navigate digital services was explicitly clear when talking about finding test answers, written electronic messages, and questionnaires from the hospital or other public institutions. Keeping up with developments, communicating with public institutions, and taking an interest in their own health and well-being were described as good reasons to learn to use digital tools.

However, other aspects also affected these learning facilitators. Some participants felt alienated while using digital tools and described the practice as something related to feelings of anxiety, fear, and stupidity as well as something that demanded “a certain amount of courage. ” Some participants felt frustrated with the digital challenges they experienced, especially when the challenges were difficult to overcome because of their physical conditions:

I get sad because of it (digital challenges) and I get very frustrated and it takes a lot of time because I have difficulty seeing when I look away from the computer and have to turn back again to find out where I was and continue there (...) It pains me that I have to use so much time on it. [Man, age 71 years]

Fear of making mistakes, particularly when communicating with public institutions, for example, the health care system, was a common pattern. Another pattern was the fear of misinterpreting the sender and the need to ensure that the written electronic messages were actually from the described sender. Some participants felt that they were forced to learn about digital tools because they cared a lot about the services. Furthermore, fears of digital services replacing human interaction were a recurring concern among the participants. Despite these initial and recurring feelings, some participants learned how to navigate the digital services that they deemed relevant. Another recurring pattern in this learning process was repetition, the practice of digital skills, and consistent assistance from other people. One participant expressed the need to use the services often to remember the necessary skills:

Now I can figure it out because now I’ve had it shown 10 times. But then three months still pass... and then I think...how was it now? Then I get sweat on my forehead (feel nervous) and think; I’m not an idiot. [Woman, age 82 years]

For some participants, learning how to use digital tools demanded time and patience, as challenges had to be overcome more than once because they reappeared until the use of digital tools was more automatized into their everyday lives. Using digital tools and health services was viewed as easier and less stressful when part of everyday routines.

Life Without Digital Tools: Not a Free Choice

Even though some participants used digital tools daily, other participants expressed that it was “too late for them.” These participants did not view it as a free choice but as something they had to accept that they could not do. They wished that they could have learned it earlier in life but did not view it as a possibility in the future. Furthermore, they saw potential in digital services, including digital health care services, but they did not know exactly what services they were missing out on. Despite this lack of knowledge, they still felt sad about the position they were in. One participant expressed what she thought regarding the use of digital tools in public institutions:

Well, I feel alright about it, but it is very, very difficult for those of us who do not have it. Sometimes you can feel left out—outside of society. And when you do not have one of those (computers)...A reference is always made to w and w (www.) and then you can read on. But you cannot do that. [Woman, age 94 years]

The feeling of being left out of society was consistent among the participants who did not use digital tools. To them, digital systems seemed to provide unfair treatment based on something outside of their own power. Participants who were heavily affected by their medical conditions and could not use digital services also felt left out because they saw the advantages of using digital tools. Furthermore, a participant described the feelings connected to the use of digital tools in public institutions:

It is more annoying that it does not seem to work out in my favour. [Woman, age 62 years]

These statements indicated that it is possible for individuals to want to use digital tools and simultaneously find them too challenging. These participants were aware that there are consequences of not using digital tools, and that saddens them, as they feel like they are not receiving the same treatment as other people in society and the health care system.

Principal Findings

The insights from our findings demonstrated that our participants had different digital practices and different experiences with digital tools and services; however, the analysis also highlighted patterns related to how digital services and tools were used. Specific conditions were important for the possibility of digital practice, including having access to social resources; possessing the necessary capabilities; and feeling motivated, secure, and comfortable . These prerequisites were necessary to have positive experiences using digital tools in the health care system, although some participants who lived up to these prerequisites were still skeptical toward digital solutions. Others who did not live up to these prerequisites experienced challenges and even though they were aware of opportunities, this awareness made them feel left out. A few participants even viewed the digital tools as a threat to their participation in society. This supports the notion of Norgaard et al [ 13 ] that the attention paid to digital capability demands from eHealth systems is very important. Furthermore, our findings supported the argument of Hjeltholt and Papazu [ 17 ] that it is important to better understand experiences related to digital services. In our study, we accommodate this request and bring forth a broad perspective on experiences with digital practices; we particularly add insight into the challenges with digital practices for patients who also have acute or chronic illness, with some of them also experiencing physical, communicative, and cognitive difficulties. To our knowledge, there is limited existing literature focusing on digital practices that do not have a limited scope, for example, a focus on perspectives on eHealth literacy in the use of apps [ 26 ] or intervention studies with a focus on experiences with digital solutions, for example, telemedicine during the COVID-19 pandemic [ 27 ]. As mentioned by Hjeltholt et al [ 10 ], certain citizens are dependent on their own social networks in the process of using and learning digital tools. Rasi et al [ 28 ] and Airola et al [ 29 ] argued that digital health literacy is situated and should include the capabilities of the individual’s social network. Our findings support these arguments that access to social resources is an important condition; however, the findings also highlight that these resources can be particularly crucial in the use of digital health services, for example, when interpreting and understanding digital and written electronic messages related to one’s own health course or when dealing with physical, cognitive, and communicative disadvantages. Therefore, we argue that the awareness of the disadvantages is important if we want to understand patients’ digital capabilities, and the inclusion of the next of kin can be evident in unveiling challenges that are unknown and not easily visible or when trying to reach patients with digital challenges through digital means.

Studies by Kayser et al [ 30 ] and Kanoe et al [ 31 ] indicated that patients’ abilities to interpret and understand digital health–related services and their benefits are important for the successful implementation of eHealth services—an argument that our findings support. Health literacy in both digital and physical contexts is important if we want to understand how to better design and implement services. Our participants’ statements support the argument that communication through digital means cannot be viewed as similar to face-to-face communication and that an emphasis on digital health literacy demonstrates how health systems are demanding different capabilities from the patients [ 13 ]. We argue that it is important to communicate the purposes of digital services so that both the patient and their next of kin know why they participate and how it can benefit them. Therefore, it is important to make it as clear as possible that digital health services can benefit the patient and that these services are developed to support information, communication, and dialogue between patients and health professionals. However, our findings suggest that even after interpreting and understanding the purposes of digital health services, some patients may still experience challenges when using digital tools.

Therefore, it is important to understand how and why patients learn digital skills, particularly because both experience with digital devices and estimation of the value of digital tools have been highlighted as key factors for digital practices [ 5 , 18 ]. Our findings indicate that a combination of these factors is important, as recognizing the value of digital tools was not enough to facilitate the necessary learning process for some of our participants. Instead, our participants described the use of digital tools as complex and continuous processes in which automation of skills, assistance from others, and time to relearn forgotten knowledge were necessary and important facilitators for learning and understanding digital tools as well as becoming more comfortable and confident in the use of digital health services. This was particularly important, as it was more encouraging for our participants to learn digital tools when they felt secure, instead of feeling afraid and anxious, a point that Bailey et al [ 18 ] also highlighted. The value of digital solutions and the will to learn were greater when challenges were viewed as something to overcome and learn from instead of something that created a feeling of being stupid. This calls for attention on how to simplify and explain digital tools and services so that users do not feel alienated. Our findings also support the argument that digital health literacy should take into account emotional well-being related to digital practice [ 32 ].

The various perspectives that our participants provided regarding the use of digital tools in the health care system indicate that patients are affected by the use of digital health services and their own capabilities to use digital tools. Murray et al [ 33 ] argued that the use of digital tools in health sectors has the potential to improve health and health delivery by improving efficacy, efficiency, accessibility, safety, and personalization, and our participants also highlighted these positive aspects. However, different studies found that some patients, particularly older adults considered socially vulnerable, have lower digital health literacy [ 10 , 34 , 35 ], which is an important determinant of health and may widen disparities and inequity in health care [ 16 ]. Studies on older adult populations’ adaptation to information and communication technology show that engaging with this technology can be limited by the usability of technology, feelings of anxiety and concern, self-perception of technology use, and the need for assistance and inclusive design [ 36 ]. Our participants’ experiences with digital practices support the importance of these focus areas, especially when primarily older patients are admitted to hospitals. Furthermore, our findings indicate that some older patients who used to view themselves as being engaged in their own health care felt more distanced from the health care system because of digital services, and some who did not have the capabilities to use digital tools felt that they were treated differently compared to the rest of society. They did not necessarily view themselves as vulnerable but felt vulnerable in the specific experience of trying to use digital services because they wished that they were more capable. Moreover, this was the case for patients with physical and cognitive difficulties, as they were not necessarily aware of the challenges before experiencing them. Drawing on the phenomenological and feministic approach by Ahmed [ 37 ], these challenges that make patients feel vulnerable are not necessarily visible to others but can instead be viewed as invisible institutional “walls” that do not present themselves before the patient runs into them. Some participants had to experience how their physical, cognitive, or communicative difficulties affected their digital practice to realize that they were not as digitally capable as they once were or as others in society. Furthermore, viewed from this perspective, our findings could be used to argue that digital capabilities should be viewed as a privilege tied to users’ physical bodies and that digital services in the health care system are indirectly making patients without this privilege vulnerable. This calls for more attention to the inequities that digital tools and services create in health care systems and awareness that those who do not use digital tools are not necessarily indifferent about the consequences. Particularly, in a context such as the Danish one, in which the digital strategy is to create an intertwined and user-friendly public digital sector for everyone, it needs to be understood that patients have different digital capabilities and needs. Although some have not yet had a challenging experience that made them feel vulnerable, others are very aware that they receive different treatment and feel that they are on their own or that the rest of the society does not care about them. Inequities in digital health care, such as these, can and should be mitigated or prevented, and our investigation into the experiences with digital practices can help to show that we are creating standards and infrastructures that deliberately exclude the perspectives of those who are most in need of the services offered by the digital health care system [ 8 ]. Therefore, our findings support the notions that flexibility is important in the implementation of universal public digital services [ 17 ]; that it is important to adjust systems in accordance with patients’ eHealth literacy and not only improve the capabilities of individuals [ 38 ]; and that the development and improvement of digital health literacy are not solely an individual responsibility but are also tied to ways in which institutions organize, design, and implement digital tools and services [ 39 ].

Limitations

This qualitative study provided novel insights into the experiences with public digital health services from the perspective of patients in the Danish context, enabling a deeper understanding of how digital health services and digital tools are experienced and used. This helps build a solid foundation for future interventions aimed at digital health literacy and digital health interventions. However, this study has some limitations. First, the study was conducted in a country where digitalization is progressing quickly, and people, therefore, are accustomed to this pace. Therefore, readers must be aware of this. Second, the study included patients with different neurological conditions; some of their digital challenges were caused or worsened by these neurological conditions and are, therefore, not applicable to all patients in the health system. However, the findings provided insights into the patients’ digital practices before their conditions and other challenges not connected to neurological conditions shared by patients. Third, the study was broad, and although a large number of informants was included, from a qualitative research perspective, we would recommend additional research in this field to develop interventions that target digital health literacy and the use of digital health services.

Conclusions

Experiences with digital tools and digital health services are complex and multifaceted. The advantages in communication, finding information, or navigating through one’s own health course work as facilitators for engaging with digital tools and digital health services. However, this is not enough on its own. Furthermore, feeling secure and motivated and having time to relearn and practice skills are important facilitators. Engagement in digital practices for the examined population requires access to continuous assistance from their social network. If patients do not meet requirements, digital health services can be experienced as exclusionary and a source of concern. Physical, cognitive, and communicative difficulties might make it impossible to use digital tools or create more challenges that require assistance. Digitalization of the health care system means that patients do not have the choice to opt out of using digital services without having consequences, resulting in them receiving a different treatment than others. To ensure digitalization does not create inequities in health, it is necessary for developers and the health institutions that create, design, and implement digital services to be aware of differences in digital health literacy and to focus on simplifying communication with patients and next of kin through and about digital services. It is important to focus on helping individuals meet the necessary conditions and finding flexible solutions for those who do not have the same privileges as others if the public digital sector is to work for everyone.

Acknowledgments

The authors would like to thank all the people who gave their time to be interviewed for the study, the clinical nurse specialists who facilitated interviewing patients, and the other nurses on shift who assisted in recruiting participants.

Conflicts of Interest

None declared.

• Year in search 2022. Google Trends. URL: https://trends.google.com/trends/yis/2022/DK/ [accessed 2024-04-02]
• Digital government index: 2019. Organisation for Economic Cooperation and Development. URL: https://www.oecd-ilibrary.org/content/paper/4de9f5bb-en [accessed 2024-04-02]
• Azzopardi-Muscat N, Sørensen K. Towards an equitable digital public health era: promoting equity through a health literacy perspective. Eur J Public Health. Oct 01, 2019;29(Supplement_3):13-17. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Digital practices. Umeå University. URL: https://www.umu.se/en/humlab/research/digital-practice/ [accessed 2024-04-02]
• It-anvendelse i befolkningen 2020. Danmarks Statistik. URL: https://www.dst.dk/da/Statistik/nyheder-analyser-publ/Publikationer/VisPub?cid=29450 [accessed 2024-04-02]
• Sundhed.dk homepage. Sundhed.dk. URL: https://www.sundhed.dk/borger/ [accessed 2024-04-02]
• Nøhr C, Bertelsen P, Vingtoft S, Andersen SK. Digitalisering af Det Danske Sundhedsvæsen. Odense, Denmark. Syddansk Universitetsforlag; 2019.
• Eriksen J, Ebbesen M, Eriksen KT, Hjermitslev C, Knudsen C, Bertelsen P, et al. Equity in digital healthcare - the case of Denmark. Front Public Health. Sep 6, 2023;11:1225222. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Digital health strategy. Sundhedsdatastyrelsen. URL: https://sundhedsdatastyrelsen.dk/da/english/digital_health_solutions/digital_health_strategy [accessed 2024-04-02]
• Hjelholt M, Schou J, Bojsen LB, Yndigegn SL. Digital marginalisering af udsatte ældre: arbejdsrapport 2. IT-Universitetet i København. 2018. URL: https://egv.dk/images/Projekter/Projekter_2018/EGV_arbejdsrapport_2.pdf [accessed 2024-04-02]
• Sørensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z, et al. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health. Jan 25, 2012;12(1):80. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Improving health literacy. World Health Organization. URL: https://www.who.int/activities/improving-health-literacy [accessed 2024-04-02]
• Norgaard O, Furstrand D, Klokker L, Karnoe KA, Batterham R, Kayser L, et al. The e-health literacy framework: a conceptual framework for characterizing e-health users and their interaction with e-health systems. Knowl Manag E Learn. 2015;7(4). [ CrossRef ]
• Kramer JM, Schwartz A. Reducing barriers to patient-reported outcome measures for people with cognitive impairments. Arch Phys Med Rehabil. Aug 2017;98(8):1705-1715. [ CrossRef ] [ Medline ]
• Menger F, Morris J, Salis C. Aphasia in an internet age: wider perspectives on digital inclusion. Aphasiology. 2016;30(2-3):112-132. [ CrossRef ]
• Richardson S, Lawrence K, Schoenthaler AM, Mann D. A framework for digital health equity. NPJ Digit Med. Aug 18, 2022;5(1):119. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hjelholt M, Papazu I. "De har fået NemID, men det er ikke nemt for mig” - Digital rum(me)lighed i den danske velfærdsstat. Social Kritik. 2021;2021-2(163). [ FREE Full text ]
• Bailey C, Sheehan C. Technology, older persons’ perspectives and the anthropological ethnographic lens. Alter. 2009;3(2):96-109. [ CrossRef ]
• Gadamer HG, Weinsheimer HG, Marshall DG. Truth and Method. New York, NY. Crossroad Publishing Company; 1991.
• Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. Dec 16, 2007;19(6):349-357. [ CrossRef ] [ Medline ]
• Polit DF, Beck CT. Nursing Research: Generating and Assessing Evidence for Nursing Practice. Philadelphia, PA. Lippincott Williams & Wilkins, Inc; 2012.
• Kvale S, Brinkmann S. InterViews: Learning the Craft of Qualitative Research Interviewing. Thousand Oaks, CA. SAGE Publications; 2009.
• Kagan A. Supported conversation for adults with aphasia: methods and resources for training conversation partners. Aphasiology. Sep 1998;12(9):816-830. [ CrossRef ]
• Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. Feb 2004;24(2):105-112. [ CrossRef ] [ Medline ]
• Krippendorff K. Content Analysis: An Introduction to Its Methodology. Thousand Oaks, CA. SAGE Publications; 1980.
• Klösch M, Sari-Kundt F, Reibnitz C, Osterbrink J. Patients' attitudes toward their health literacy and the use of digital apps in health and disease management. Br J Nurs. Nov 25, 2021;30(21):1242-1249. [ CrossRef ] [ Medline ]
• Datta P, Eiland L, Samson K, Donovan A, Anzalone AJ, McAdam-Marx C. Telemedicine and health access inequalities during the COVID-19 pandemic. J Glob Health. Dec 03, 2022;12:05051. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Rasi P, Lindberg J, Airola E. Older service users’ experiences of learning to use eHealth applications in sparsely populated healthcare settings in Northern Sweden and Finland. Educ Gerontol. Nov 24, 2020;47(1):25-35. [ CrossRef ]
• Airola E, Rasi P, Outila M. Older people as users and non-users of a video conferencing service for promoting social connectedness and well-being – a case study from Finnish Lapland. Educ Gerontol. Mar 29, 2020;46(5):258-269. [ CrossRef ]
• Kayser L, Kushniruk A, Osborne RH, Norgaard O, Turner P. Enhancing the effectiveness of consumer-focused health information technology systems through eHealth literacy: a framework for understanding users' needs. JMIR Hum Factors. May 20, 2015;2(1):e9. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Karnoe A, Furstrand D, Christensen KB, Norgaard O, Kayser L. Assessing competencies needed to engage with digital health services: development of the ehealth literacy assessment toolkit. J Med Internet Res. May 10, 2018;20(5):e178. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Nielsen AS, Hanna L, Larsen BF, Appel CW, Osborne RH, Kayser L. Readiness, acceptance and use of digital patient reported outcome in an outpatient clinic. Health Informatics J. Jun 03, 2022;28(2):14604582221106000. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med. Nov 2016;51(5):843-851. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chesser A, Burke A, Reyes J, Rohrberg T. Navigating the digital divide: a systematic review of eHealth literacy in underserved populations in the United States. Inform Health Soc Care. Feb 24, 2016;41(1):1-19. [ CrossRef ] [ Medline ]
• Chesser AK, Keene Woods N, Smothers K, Rogers N. Health literacy and older adults: a systematic review. Gerontol Geriatr Med. Mar 15, 2016;2:2333721416630492. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mitra S, Singh A, Rajendran Deepam S, Asthana MK. Information and communication technology adoption among the older people: a qualitative approach. Health Soc Care Community. Nov 21, 2022;30(6):e6428-e6437. [ CrossRef ] [ Medline ]
• Ahmed S. How not to do things with words. Wagadu. 2016. URL: https://sites.cortland.edu/wagadu/wp-content/uploads/sites/3/2017/02/v16-how-not-to-do-ahmed.pdf [accessed 2024-04-02]
• Monkman H, Kushniruk AW. eHealth literacy issues, constructs, models, and methods for health information technology design and evaluation. Knowl Manag E Learn. 2015;7(4). [ CrossRef ]
• Brørs G, Norman CD, Norekvål TM. Accelerated importance of eHealth literacy in the COVID-19 outbreak and beyond. Eur J Cardiovasc Nurs. Aug 15, 2020;19(6):458-461. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by A Mavragani; submitted 14.03.23; peer-reviewed by G Myreteg, J Eriksen, M Siermann; comments to author 18.09.23; revised version received 09.10.23; accepted 27.02.24; published 11.04.24.

©Christian Gybel Jensen, Frederik Gybel Jensen, Mia Ingerslev Loft. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 11.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

TrinityHaus

View the contact page for more contact and location information

• Cities & Health Publication

Cities & Health article 'Bringing brain health home', including TrinityHaus Research Fellow, Tom Grey

For more information, read the article: https://www.tandfonline.com/doi/full/10.1080/23748834.2024.2327815

• Share full article

There’s an Explosion of Plastic Waste. Big Companies Say ‘We’ve Got This.’

Big brands like Procter & Gamble and Nestlé say a new generation of plants will help them meet environmental goals, but the technology is struggling to deliver.

Recycled polypropylene pellets at a PureCycle Technologies plant in Ironton, Ohio. Credit... Maddie McGarvey for The New York Times

Supported by

By Hiroko Tabuchi

• Published April 5, 2024 Updated April 8, 2024

By 2025, Nestle promises not to use any plastic in its products that isn’t recyclable. By that same year, L’Oreal says all of its packaging will be “refillable, reusable, recyclable or compostable.”

And by 2030, Procter & Gamble pledges that it will halve its use of virgin plastic resin made from petroleum.

To get there, these companies and others are promoting a new generation of recycling plants, called “advanced” or “chemical” recycling, that promise to recycle many more products than can be recycled today.

So far, advanced recycling is struggling to deliver on its promise. Nevertheless, the new technology is being hailed by the plastics industry as a solution to an exploding global waste problem.

The traditional approach to recycling is to simply grind up and melt plastic waste. The new, advanced-recycling operators say they can break down the plastic much further, into more basic molecular building blocks, and transform it into new plastic.

PureCycle Technologies, a company that features prominently in Nestlé, L’Oréal, and Procter & Gamble’s plastics commitments, runs one such facility, a $500 million plant in Ironton, Ohio. The plant was originally to start operating in 2020 , with the capacity to process as much as 182 tons of discarded polypropylene, a hard-to-recycle plastic used widely in single-use cups, yogurt tubs, coffee pods and clothing fibers, every day.

But PureCycle’s recent months have instead been filled with setbacks: technical issues at the plant, shareholder lawsuits, questions over the technology and a startling report from contrarian investors who make money when a stock price falls. They said that they had flown a drone over the facility that showed that the plant was far from being able to make much new plastic.

PureCycle, based in Orlando, Fla., said it remained on track. “We’re ramping up production,” its chief executive, Dustin Olson, said during a recent tour of the plant, a constellation of pipes, storage tanks and cooling towers in Ironton, near the Ohio River. “We believe in this technology. We’ve seen it work,” he said. “We’re making leaps and bounds.”

Nestlé, Procter & Gamble and L’Oréal have also expressed confidence in PureCycle. L’Oréal said PureCycle was one of many partners developing a range of recycling technologies. P.&G. said it hoped to use the recycled plastic for “numerous packaging applications as they scale up production.” Nestlé didn’t respond to requests for comment, but has said it is collaborating with PureCycle on “groundbreaking recycling technologies.”

PureCycle’s woes are emblematic of broad trouble faced by a new generation of recycling plants that have struggled to keep up with the growing tide of global plastic production, which scientists say could almost quadruple by midcentury .

A chemical-recycling facility in Tigard, Ore., a joint venture between Agilyx and Americas Styrenics, is in the process of shutting down after millions of dollars in losses. A plant in Ashley, Ind., that had aimed to recycle 100,000 tons of plastic a year by 2021 had processed only 2,000 tons in total as of late 2023, after fires, oil spills and worker safety complaints.

At the same time, many of the new generation of recycling facilities are turning plastic into fuel, something the Environmental Protection Agency doesn’t consider to be recycling, though industry groups say some of that fuel can be turned into new plastic .

Overall, the advanced recycling plants are struggling to make a dent in the roughly 36 million tons of plastic Americans discard each year, which is more than any other country. Even if the 10 remaining chemical-recycling plants in America were to operate at full capacity, they would together process some 456,000 tons of plastic waste, according to a recent tally by Beyond Plastics , a nonprofit group that advocates stricter controls on plastics production. That’s perhaps enough to raise the plastic recycling rate — which has languished below 10 percent for decades — by a single percentage point.

For households, that has meant that much of the plastic they put out for recycling doesn’t get recycled at all, but ends up in landfills. Figuring out which plastics are recyclable and which aren’t has turned into, essentially, a guessing game . That confusion has led to a stream of non-recyclable trash contaminating the recycling process, gumming up the system.

“The industry is trying to say they have a solution,” said Terrence J. Collins, a professor of chemistry and sustainability science at Carnegie Mellon University. “It’s a non-solution.”

‘Molecular washing machine’

It was a long-awaited day last June at PureCycle’s Ironton facility: The company had just produced its first batch of what it describes as “ultra-pure” recycled polypropylene pellets.

That milestone came several years late and with more than $350 million in cost overruns. Still, the company appeared to have finally made it. “Nobody else can do this,” Jeff Kramer, the plant manager, told a local news crew .

PureCycle had done it by licensing a game-changing method — developed by Procter & Gamble researchers in the mid-2010s, but unproven at scale — that uses solvent to dissolve and purify the plastic to make it new again. “It’s like a molecular washing machine,” Mr. Olson said.

There’s a reason Procter & Gamble, Nestlé and L’Oréal, some of the world’s biggest users of plastic, are excited about the technology. Many of their products are made from polypropylene, a plastic that they transform into a plethora of products using dyes and fillers. P.&G. has said it uses more polypropylene than any other plastic, more than a half-million tons a year.

But those additives make recycling polypropylene more difficult.

The E.P.A. estimates that 2.7 percent of polypropylene packaging is reprocessed. But PureCycle was promising to take any polypropylene — disposable beer cups, car bumpers, even campaign signs — and remove the colors, odors, and contaminants to transform it into new plastic.

Soon after the June milestone, trouble hit.

On Sept. 13, PureCycle disclosed that its plant had suffered a power failure the previous month that had halted operations and caused a vital seal to fail. That meant the company would be unable to meet key milestones, it told lenders.

Then in November, Bleecker Street Research — a New York-based short-seller, an investment strategy that involves betting that a company’s stock price will fall — published a report asserting that the white pellets that had rolled off PureCycle’s line in June weren’t recycled from plastic waste. The short-sellers instead claimed that the company had simply run virgin polypropylene through the system as part of a demonstration run.

Mr. Olson said PureCycle hadn’t used consumer waste in the June 2023 run, but it hadn’t used virgin plastic, either. Instead it had used scrap known as “post industrial,” which is what’s left over from the manufacturing process and would otherwise go to a landfill, he said.

Bleecker Street also said it had flown heat-sensing drones over the facility and said it found few signs of commercial-scale activity. The firm also raised questions about the solvent PureCycle was using to break down the plastic, calling it “a nightmare concoction” that was difficult to manage.

PureCycle is now being sued by other investors who accuse the company of making false statements and misleading investors about its setbacks.

Mr. Olson declined to describe the solvent. Regulatory filings reviewed by The New York Times indicate that it is butane, a highly flammable gas, stored under pressure. The company’s filing described the risks of explosion, citing a “worst case scenario” that could cause second-degree burns a half-mile away, and said that to mitigate the risk the plant was equipped with sprinklers, gas detectors and alarms.

Chasing the ‘circular economy’

It isn’t unusual, of course, for any new technology or facility to experience hiccups. The plastics industry says these projects, once they get going, will bring the world closer to a “circular” economy, where things are reused again and again.

Plastics-industry lobbying groups are promoting chemical recycling. At a hearing in New York late last year, industry lobbyists pointed to the promise of advanced recycling in opposing a packaging-reduction bill that would eventually mandate a 50 percent reduction in plastic packaging. And at negotiations for a global plastics treaty , lobby groups are urging nations to consider expanding chemical recycling instead of taking steps like restricting plastic production or banning plastic bags.

A spokeswoman for the American Chemistry Council, which represents plastics makers as well as oil and gas companies that produce the building blocks of plastic, said that chemical recycling potentially “complements mechanical recycling, taking the harder-to-recycle plastics that mechanical often cannot.”

Environmental groups say the companies are using a timeworn strategy of promoting recycling as a way to justify selling more plastic, even though the new recycling technology isn’t ready for prime time. Meanwhile, they say, plastic waste chokes rivers and streams, piles up in landfills or is exported .

“These large consumer brand companies, they’re out over their skis,” said Judith Enck, the president of Beyond Plastics and a former regional E.P.A. administrator. “Look behind the curtain, and these facilities aren’t operating at scale, and they aren’t environmentally sustainable,” she said.

The better solution, she said, would be, “We need to make less plastic.”

Touring the plant

Mr. Olson recently strolled through a cavernous warehouse at PureCycle’s Ironton site, built at a former Dow Chemical plant. Since January, he said, PureCycle has been processing mainly consumer plastic waste and has produced about 1.3 million pounds of recycled polypropylene, or about 1 percent of its annual production target.

“This is a bag that would hold dog food,” he said, pointing to a bale of woven plastic bags. “And these are fruit carts that you’d see in street markets. We can recycle all of that, which is pretty cool.”

The plant was dealing with a faulty valve discovered the day before, so no pellets were rolling off the line. Mr. Olson pulled out a cellphone to show a photo of a valve with a dark line ringing its interior. “It’s not supposed to look like that,” he said.

The company later sent video of Mr. Olson next to white pellets once again streaming out of its production line.

PureCycle says every kilogram of polypropylene it recycles emits about 1.54 kilograms of planet-warming carbon dioxide. That’s on par with a commonly used industry measure of emissions for virgin polypropylene. PureCycle said that it was improving on that measure.

Nestlé, L’Oréal and Procter & Gamble continue to say they’re optimistic about the technology. In November, Nestlé said it had invested in a British company that would more easily separate out polypropylene from other plastic waste.

It was “just one of the many steps we are taking on our journey to ensure our packaging doesn’t end up as waste,” the company said.

Hiroko Tabuchi covers the intersection of business and climate for The Times. She has been a journalist for more than 20 years in Tokyo and New York. More about Hiroko Tabuchi

Learn More About Climate Change

Have questions about climate change? Our F.A.Q. will tackle your climate questions, big and small .

“Buying Time,” a new series from The New York Times, looks at the risky ways  humans are starting to manipulate nature  to fight climate change.

Big brands like Procter & Gamble and Nestlé say a new generation of recycling plants will help them meet environmental goals, but the technology is struggling to deliver .

The Italian energy giant Eni sees future profits from collecting carbon dioxide and pumping it  into natural gas fields that have been exhausted.

New satellite-based research reveals how land along the East Coast is slumping into the ocean, compounding the danger from global sea level rise . A major culprit: the overpumping of groundwater.

Did you know the ♻ symbol doesn’t mean something is actually recyclable ? Read on about how we got here, and what can be done.

Advertisement

Read our research on: Gun Policy | International Conflict | Election 2024

Regions & Countries

What public k-12 teachers want americans to know about teaching.

Illustrations by Hokyoung Kim

At a time when most teachers are feeling stressed and overwhelmed in their jobs, we asked 2,531 public K-12 teachers this open-ended question:

If there’s one thing you’d want the public to know about teachers, what would it be?

We also asked Americans what they think about teachers to compare with teachers’ perceptions of how the public views them.

Related: What’s It Like To Be a Teacher in America Today?

Pew Research Center conducted this analysis to better understand what public K-12 teachers would like Americans to know about their profession. We also wanted to learn how the public thinks about teachers.

For the open-end question, we surveyed 2,531 U.S. public K-12 teachers from Oct. 17 to Nov. 14, 2023. The teachers surveyed are members of RAND’s American Teacher Panel, a nationally representative panel of public K-12 school teachers recruited through MDR Education. Survey data is weighted to state and national teacher characteristics to account for differences in sampling and response to ensure they are representative of the target population.

Overall, 96% of surveyed teachers provided an answer to the open-ended question. Center researchers developed a coding scheme categorizing the responses, coded all responses, and then grouped them into the six themes explored in the data essay.

For the questions for the general public, we surveyed 5,029 U.S. adults from Nov. 9 to Nov. 16, 2023. The adults surveyed are members of the Ipsos KnowledgePanel, a nationally representative online survey panel. Panel members are randomly recruited through probability-based sampling, and households are provided with access to the Internet and hardware if needed. To ensure that the results of this survey reflect a balanced cross section of the nation, the data is weighted to match the U.S. adult population by gender, age, education, race and ethnicity and other categories.

Here are the questions used for this analysis , along with responses, the teacher survey methodology and the general public survey methodology .

Most of the responses to the open-ended question fell into one of these six themes:

Teaching is a hard job

About half of teachers (51%) said they want the public to know that teaching is a difficult job and that teachers are hardworking. Within this share, many mentioned that they have roles and responsibilities in the classroom besides teaching, which makes the job stressful. Many also talked about working long hours, beyond those they’re contracted for.

“Teachers serve multiple roles other than being responsible for teaching curriculum. We are counselors, behavioral specialists and parents for students who need us to fill those roles. We sacrifice a lot to give all of ourselves to the role as teacher.”

– Elementary school teacher

“The amount of extra hours that teachers have to put in beyond the contractual time is ridiculous. Arriving 30 minutes before and leaving an hour after is just the tip of the iceberg. … And as far as ‘having summers off,’ most of August is taken up with preparing materials for the upcoming school year or attending three, four, seven days’ worth of unpaid development training.”

– High school teacher

Teachers care about their students

The next most common theme: 22% of teachers brought up how fulfilling teaching is and how much teachers care about their students. Many gave examples of the hardships of teaching but reaffirmed that they do their job because they love the kids and helping them succeed. 

“We are passionate about what we do. Every child we teach is important to us and we look out for them like they are our own.”

– Middle school teacher

“We are in it for the kids, and the most incredible moments are when children make connections with learning.”

Teachers are undervalued and disrespected

Some 17% of teachers want the public to know that they feel undervalued and disrespected, and that they need more public support. Some mentioned that they are well-educated professionals but are not treated as such. And many teachers in this category responded with a general plea for support from the public, which they don’t feel they’re getting now.

“We feel undervalued. The public and many parents of my students treat me and my peers as if we do not know as much as they do, as if we are uneducated.”

“The public attitudes toward teachers have been degrading, and it is making it impossible for well-qualified teachers to be found. People are simply not wanting to go into the profession because of public sentiments.”

Teachers are underpaid

A similar share of teachers (15%) want the public to know that teachers are underpaid. Many teachers said their salary doesn’t account for the effort and care they put into their students’ education and believe that their pay should reflect this.

“We are sorely underpaid for the amount of hours we work and the education level we have attained.”

Teachers need support and resources from government and administrators

About one-in-ten teachers (9%) said they need more support from the government, their administrators and other key stakeholders. Many mentioned working in understaffed schools, not having enough funding and paying for supplies out of pocket. Some teachers also expressed that they have little control over the curriculum that they teach.

“The world-class education we used to be proud of does not exist because of all the red tape we are constantly navigating. If you want to see real change in the classroom, advocate for smaller class sizes for your child, push your district to cap class sizes at a reasonable level and have real, authentic conversations with your child’s teacher about what is going on in the classroom if you’re curious.”

Teachers need more support from parents

Roughly the same share of teachers (8%) want the public to know that teachers need more support from parents, emphasizing that the parent-teacher relationship is strained. Many view parents as partners in their child’s education and believe that a strong relationship improves kids’ overall social and emotional development.

“Teachers help students to reach their potential. However, that job is near impossible if parents/guardians do not take an active part in their student’s education.”

How the U.S. public views teachers

While the top response from teachers in the open-ended question is that they want the public to know that teaching is a hard job, most Americans already see it that way. Two-thirds of U.S. adults say being a public K-12 teacher is harder than most other jobs, with 33% saying it’s a lot harder.

And about three-quarters of Americans (74%) say teachers should be paid more than they are now, including 39% who say teachers should be paid a lot more.

Americans are about evenly divided on whether the public generally looks up to (32%) or down on (30%) public K-12 teachers. Some 37% say Americans neither look up to or down on public K-12 teachers.

In addition to the open-ended question about what they want the public to know about them, we asked teachers how much they think most Americans trust public K-12 teachers to do their job well. We also asked the public how much they trust teachers. Answers differ considerably.

Nearly half of public K-12 teachers (47%) say most Americans don’t trust teachers much or at all. A third say most Americans trust teachers some, and 18% say the public trusts teachers a great deal or a fair amount.

In contrast, a majority of Americans (57%) say they do trust public K-12 teachers to do their job well a great deal or a fair amount. About a quarter (26%) say they trust teachers some, and 17% say they don’t trust teachers much or at all.

Related: About half of Americans say public K-12 education is going in the wrong direction

How the public’s views differ by party

There are sizable party differences in Americans’ views of teachers. In particular, Democrats and Democratic-leaning independents are more likely than Republicans and Republican leaners to say:

• They trust teachers to do their job well a great deal or a fair amount (70% vs. 44%)
• Teaching is a lot or somewhat harder when compared with most other jobs (77% vs. 59%)
• Teachers should be paid a lot or somewhat more than they are now (86% vs. 63%)

In their own words

Below, we have a selection of quotes that describe what teachers want the public to know about them and their profession.

Social Trends Monthly Newsletter

Sign up to to receive a monthly digest of the Center's latest research on the attitudes and behaviors of Americans in key realms of daily life

About Pew Research Center Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of The Pew Charitable Trusts .