case study positive and negative

10 Case Study Advantages and Disadvantages

A case study in academic research is a detailed and in-depth examination of a specific instance or event, generally conducted through a qualitative approach to data.

The most common case study definition that I come across is is Robert K. Yin’s (2003, p. 13) quote provided below:

“An empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident.”

Researchers conduct case studies for a number of reasons, such as to explore complex phenomena within their real-life context, to look at a particularly interesting instance of a situation, or to dig deeper into something of interest identified in a wider-scale project.

While case studies render extremely interesting data, they have many limitations and are not suitable for all studies. One key limitation is that a case study’s findings are not usually generalizable to broader populations because one instance cannot be used to infer trends across populations.

Case Study Advantages and Disadvantages

1. in-depth analysis of complex phenomena.

Case study design allows researchers to delve deeply into intricate issues and situations.

By focusing on a specific instance or event, researchers can uncover nuanced details and layers of understanding that might be missed with other research methods, especially large-scale survey studies.

As Lee and Saunders (2017) argue,

“It allows that particular event to be studies in detail so that its unique qualities may be identified.”

This depth of analysis can provide rich insights into the underlying factors and dynamics of the studied phenomenon.

2. Holistic Understanding

Building on the above point, case studies can help us to understand a topic holistically and from multiple angles.

This means the researcher isn’t restricted to just examining a topic by using a pre-determined set of questions, as with questionnaires. Instead, researchers can use qualitative methods to delve into the many different angles, perspectives, and contextual factors related to the case study.

We can turn to Lee and Saunders (2017) again, who notes that case study researchers “develop a deep, holistic understanding of a particular phenomenon” with the intent of deeply understanding the phenomenon.

3. Examination of rare and Unusual Phenomena

We need to use case study methods when we stumble upon “rare and unusual” (Lee & Saunders, 2017) phenomena that would tend to be seen as mere outliers in population studies.

Take, for example, a child genius. A population study of all children of that child’s age would merely see this child as an outlier in the dataset, and this child may even be removed in order to predict overall trends.

So, to truly come to an understanding of this child and get insights into the environmental conditions that led to this child’s remarkable cognitive development, we need to do an in-depth study of this child specifically – so, we’d use a case study.

4. Helps Reveal the Experiences of Marginalzied Groups

Just as rare and unsual cases can be overlooked in population studies, so too can the experiences, beliefs, and perspectives of marginalized groups.

As Lee and Saunders (2017) argue, “case studies are also extremely useful in helping the expression of the voices of people whose interests are often ignored.”

Take, for example, the experiences of minority populations as they navigate healthcare systems. This was for many years a “hidden” phenomenon, not examined by researchers. It took case study designs to truly reveal this phenomenon, which helped to raise practitioners’ awareness of the importance of cultural sensitivity in medicine.

5. Ideal in Situations where Researchers cannot Control the Variables

Experimental designs – where a study takes place in a lab or controlled environment – are excellent for determining cause and effect . But not all studies can take place in controlled environments (Tetnowski, 2015).

When we’re out in the field doing observational studies or similar fieldwork, we don’t have the freedom to isolate dependent and independent variables. We need to use alternate methods.

Case studies are ideal in such situations.

A case study design will allow researchers to deeply immerse themselves in a setting (potentially combining it with methods such as ethnography or researcher observation) in order to see how phenomena take place in real-life settings.

6. Supports the generation of new theories or hypotheses

While large-scale quantitative studies such as cross-sectional designs and population surveys are excellent at testing theories and hypotheses on a large scale, they need a hypothesis to start off with!

This is where case studies – in the form of grounded research – come in. Often, a case study doesn’t start with a hypothesis. Instead, it ends with a hypothesis based upon the findings within a singular setting.

The deep analysis allows for hypotheses to emerge, which can then be taken to larger-scale studies in order to conduct further, more generalizable, testing of the hypothesis or theory.

7. Reveals the Unexpected

When a largescale quantitative research project has a clear hypothesis that it will test, it often becomes very rigid and has tunnel-vision on just exploring the hypothesis.

Of course, a structured scientific examination of the effects of specific interventions targeted at specific variables is extermely valuable.

But narrowly-focused studies often fail to shine a spotlight on unexpected and emergent data. Here, case studies come in very useful. Oftentimes, researchers set their eyes on a phenomenon and, when examining it closely with case studies, identify data and come to conclusions that are unprecedented, unforeseen, and outright surprising.

As Lars Meier (2009, p. 975) marvels, “where else can we become a part of foreign social worlds and have the chance to become aware of the unexpected?”

Disadvantages

1. not usually generalizable.

Case studies are not generalizable because they tend not to look at a broad enough corpus of data to be able to infer that there is a trend across a population.

As Yang (2022) argues, “by definition, case studies can make no claims to be typical.”

Case studies focus on one specific instance of a phenomenon. They explore the context, nuances, and situational factors that have come to bear on the case study. This is really useful for bringing to light important, new, and surprising information, as I’ve already covered.

But , it’s not often useful for generating data that has validity beyond the specific case study being examined.

2. Subjectivity in interpretation

Case studies usually (but not always) use qualitative data which helps to get deep into a topic and explain it in human terms, finding insights unattainable by quantitative data.

But qualitative data in case studies relies heavily on researcher interpretation. While researchers can be trained and work hard to focus on minimizing subjectivity (through methods like triangulation), it often emerges – some might argue it’s innevitable in qualitative studies.

So, a criticism of case studies could be that they’re more prone to subjectivity – and researchers need to take strides to address this in their studies.

3. Difficulty in replicating results

Case study research is often non-replicable because the study takes place in complex real-world settings where variables are not controlled.

So, when returning to a setting to re-do or attempt to replicate a study, we often find that the variables have changed to such an extent that replication is difficult. Furthermore, new researchers (with new subjective eyes) may catch things that the other readers overlooked.

Replication is even harder when researchers attempt to replicate a case study design in a new setting or with different participants.

Comprehension Quiz for Students

Question 1: What benefit do case studies offer when exploring the experiences of marginalized groups?

a) They provide generalizable data. b) They help express the voices of often-ignored individuals. c) They control all variables for the study. d) They always start with a clear hypothesis.

Question 2: Why might case studies be considered ideal for situations where researchers cannot control all variables?

a) They provide a structured scientific examination. b) They allow for generalizability across populations. c) They focus on one specific instance of a phenomenon. d) They allow for deep immersion in real-life settings.

Question 3: What is a primary disadvantage of case studies in terms of data applicability?

a) They always focus on the unexpected. b) They are not usually generalizable. c) They support the generation of new theories. d) They provide a holistic understanding.

Question 4: Why might case studies be considered more prone to subjectivity?

a) They always use quantitative data. b) They heavily rely on researcher interpretation, especially with qualitative data. c) They are always replicable. d) They look at a broad corpus of data.

Question 5: In what situations are experimental designs, such as those conducted in labs, most valuable?

a) When there’s a need to study rare and unusual phenomena. b) When a holistic understanding is required. c) When determining cause-and-effect relationships. d) When the study focuses on marginalized groups.

Question 6: Why is replication challenging in case study research?

a) Because they always use qualitative data. b) Because they tend to focus on a broad corpus of data. c) Due to the changing variables in complex real-world settings. d) Because they always start with a hypothesis.

Lee, B., & Saunders, M. N. K. (2017). Conducting Case Study Research for Business and Management Students. SAGE Publications.

Meir, L. (2009). Feasting on the Benefits of Case Study Research. In Mills, A. J., Wiebe, E., & Durepos, G. (Eds.). Encyclopedia of Case Study Research (Vol. 2). London: SAGE Publications.

Tetnowski, J. (2015). Qualitative case study research design.  Perspectives on fluency and fluency disorders ,  25 (1), 39-45. ( Source )

Yang, S. L. (2022). The War on Corruption in China: Local Reform and Innovation . Taylor & Francis.

Yin, R. (2003). Case Study research. Thousand Oaks, CA: Sage.

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 5 Top Tips for Succeeding at University
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 50 Durable Goods Examples
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 100 Consumer Goods Examples
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd/ 30 Globalization Pros and Cons

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

Home » Pros and Cons » 12 Case Study Method Advantages and Disadvantages

12 Case Study Method Advantages and Disadvantages

A case study is an investigation into an individual circumstance. The investigation may be of a single person, business, event, or group. The investigation involves collecting in-depth data about the individual entity through the use of several collection methods. Interviews and observation are two of the most common forms of data collection used.

The case study method was originally developed in the field of clinical medicine. It has expanded since to other industries to examine key results, either positive or negative, that were received through a specific set of decisions. This allows for the topic to be researched with great detail, allowing others to glean knowledge from the information presented.

Here are the advantages and disadvantages of using the case study method.

List of the Advantages of the Case Study Method

1. it turns client observations into useable data..

Case studies offer verifiable data from direct observations of the individual entity involved. These observations provide information about input processes. It can show the path taken which led to specific results being generated. Those observations make it possible for others, in similar circumstances, to potentially replicate the results discovered by the case study method.

2. It turns opinion into fact.

Case studies provide facts to study because you’re looking at data which was generated in real-time. It is a way for researchers to turn their opinions into information that can be verified as fact because there is a proven path of positive or negative development. Singling out a specific incident also provides in-depth details about the path of development, which gives it extra credibility to the outside observer.

3. It is relevant to all parties involved.

Case studies that are chosen well will be relevant to everyone who is participating in the process. Because there is such a high level of relevance involved, researchers are able to stay actively engaged in the data collection process. Participants are able to further their knowledge growth because there is interest in the outcome of the case study. Most importantly, the case study method essentially forces people to make a decision about the question being studied, then defend their position through the use of facts.

4. It uses a number of different research methodologies.

The case study method involves more than just interviews and direct observation. Case histories from a records database can be used with this method. Questionnaires can be distributed to participants in the entity being studies. Individuals who have kept diaries and journals about the entity being studied can be included. Even certain experimental tasks, such as a memory test, can be part of this research process.

5. It can be done remotely.

Researchers do not need to be present at a specific location or facility to utilize the case study method. Research can be obtained over the phone, through email, and other forms of remote communication. Even interviews can be conducted over the phone. That means this method is good for formative research that is exploratory in nature, even if it must be completed from a remote location.

6. It is inexpensive.

Compared to other methods of research, the case study method is rather inexpensive. The costs associated with this method involve accessing data, which can often be done for free. Even when there are in-person interviews or other on-site duties involved, the costs of reviewing the data are minimal.

7. It is very accessible to readers.

The case study method puts data into a usable format for those who read the data and note its outcome. Although there may be perspectives of the researcher included in the outcome, the goal of this method is to help the reader be able to identify specific concepts to which they also relate. That allows them to discover unusual features within the data, examine outliers that may be present, or draw conclusions from their own experiences.

List of the Disadvantages of the Case Study Method

1. it can have influence factors within the data..

Every person has their own unconscious bias. Although the case study method is designed to limit the influence of this bias by collecting fact-based data, it is the collector of the data who gets to define what is a “fact” and what is not. That means the real-time data being collected may be based on the results the researcher wants to see from the entity instead. By controlling how facts are collected, a research can control the results this method generates.

2. It takes longer to analyze the data.

The information collection process through the case study method takes much longer to collect than other research options. That is because there is an enormous amount of data which must be sifted through. It’s not just the researchers who can influence the outcome in this type of research method. Participants can also influence outcomes by given inaccurate or incomplete answers to questions they are asked. Researchers must verify the information presented to ensure its accuracy, and that takes time to complete.

3. It can be an inefficient process.

Case study methods require the participation of the individuals or entities involved for it to be a successful process. That means the skills of the researcher will help to determine the quality of information that is being received. Some participants may be quiet, unwilling to answer even basic questions about what is being studied. Others may be overly talkative, exploring tangents which have nothing to do with the case study at all. If researchers are unsure of how to manage this process, then incomplete data is often collected.

4. It requires a small sample size to be effective.

The case study method requires a small sample size for it to yield an effective amount of data to be analyzed. If there are different demographics involved with the entity, or there are different needs which must be examined, then the case study method becomes very inefficient.

5. It is a labor-intensive method of data collection.

The case study method requires researchers to have a high level of language skills to be successful with data collection. Researchers must be personally involved in every aspect of collecting the data as well. From reviewing files or entries personally to conducting personal interviews, the concepts and themes of this process are heavily reliant on the amount of work each researcher is willing to put into things.

These case study method advantages and disadvantages offer a look at the effectiveness of this research option. With the right skill set, it can be used as an effective tool to gather rich, detailed information about specific entities. Without the right skill set, the case study method becomes inefficient and inaccurate.

Related Posts:

• 25 Best Ways to Overcome the Fear of Failure
• Monroe's Motivated Sequence Explained [with Examples]
• 21 Most Effective Bundle Pricing Strategies with Examples
• Force Field Analysis Explained with Examples

• Communication
• Recreational

Pros and Cons of case studies

• Post author: admin
• Post published: March 27, 2020
• Post category: Education
• Post comments: 0 Comments

Case studies are research methodologies that are used and analyzed in order to depict principles; they have been usually used in social sciences. They are research strategies and experiential inquiries that seek to examine various phenomena within a real-life context. Case studies seek to explain and give details in the analysis of people and events. There are several pros that back case studies and there are cons too that criticize them. The pros and cons are listed below.

1 . They show client observations- Since case studies are strategies that are used and analyzed in order to describe principles therefore it seeks to show indeed the client investigated and experienced a particular phenomenon.

2 . Makes practical improvements- Case studies present facts that categorically describe particular people or events in order to make some of the necessary improvements. Case studies data is what supports a particular belief.

3 . They are an influential way of portraying something- If a researcher wants to prove a particular principle to be true, he or she must back it by case studies in order to make the other people and the naysayers believe.

4 . They turn opinions into facts- Case studies present real data on a particular phenomenon. Since facts about various things are presented then it can be verified through this kind of data if the information presented is in the positive or negative development of opinion.

5 . It is relevant to all the parties that are involved- Case studies help the researchers in actively focusing on the data collection process and the participants’ knowledge is bettered. At the end of the process, everybody is able to defend his position through facts.

6 . A number of different research methodologies can be used in case the studies- Case study method goes beyond the interview and direct observations. Secondary data can be obtained from various historical sources that can be used to back the method.

7 . Case studies can be done remotely- It is not essential for a researcher to be present in the specific location of the study in order to effectively use the case study method. Other forms of communication come in to cover that gap for the researcher.

8 . It has a very high cost- If you put this research method in comparison to the others, this one seems more expensive because the cost of accessing data is very high.

9 . Readers can access data from this method very easily- The The format in which case studies present their data is very useful to the readers and easily note the outcomes of the same.

10 . Collects data that cannot be collected by another method- The type of data collected by case studies is much richer and greater in-depth than that of the other experimental methods.

1 . Data collected cannot be generalized- The data collected by the case study method was collected from a smaller population it cannot be generalized to the wider population.

2 . Some of the case studies are not scientific- The weakness of the data collected in some of the case studies that are not scientific is that it cannot be generalized.

3 . It is very difficult to draw a definite cause/effect from case studies- The the kind of data that case studies present cannot be used to draw a definite cause-effect relationship.

4 . Case studies concentrate on one experiment- The problem associated with concentrating on one experiment or a specific group of people is that the data presented might contain some kind of bias.

5 . It takes a lot of time to analyze the data- This process takes longer to analyze the data because there is a very large amount of data that must be collected. Participants might take a lot of time in giving answers or giving inaccurate information.

6 . Case studies can be inefficient processes- Sometimes the researchers are not present in the study areas which means they will not be able to notice whether the information provided is accurate or not terming the whole process inefficient.

7 . Case study method can only be effective with a small sample size- If a very large sample size is involved in the case study it is likely for it to become inefficient because the method requires a small sample size to get the data and analyze it.

8 . The method requires a lot of labor in data collection- The researcher is seriously needed in the data collection of this method. They have to be personally involved in order to be able to identify the quality of the data provided.

9 . There are factors that can influence the data- The method of data collection is meant to collect fact-based data but the power to determine what fact is and what is not is the person who is collecting the data.

10 . There is no right answer in case studies- Case studies do not present any specific answer that is right, the problem arises in the validation of solutions because there is more than one way of looking at things.

You Might Also Like

Pros and Cons of Traditional Education System

Pros and Cons of Essa

Pros and Cons of Theory Y

Leave a reply cancel reply.

Save my name, email, and website in this browser for the next time I comment.

This site uses Akismet to reduce spam. Learn how your comment data is processed .

Resilience and Well-Being: Case Studies of Four Individuals Who Have Undergone Adversities

• First Online: 08 March 2022

Cite this chapter

• Shikha Soni 3 &
• Amrita Deb 3  

1311 Accesses

Individuals frequently confront various kinds of losses during their lifetime such as death of loved ones, financial losses, damage to health, among others. These challenges can have short-term or long-term impacts. Hence, adapting successfully to the impact of these challenges is an inevitable and sometimes an ongoing process in life. While some individuals display maladaptive behavior when faced with such adversities, others display resilience in adapting to them and eventually establish a state of well-being. This chapter presents four case studies of resilient individuals who have faced different challenges such as physical health, relationship loss, domestic violence, and child sexual abuse. There were two male and two female participants with a mean age of 32. The objective of the study was to explore resilience and well-being outcomes among individuals who have experienced adversity. A case study approach was considered suitable as it allows participants to report their subjective experiences, an opportunity that quantitative methods do not allow. Interviews were conducted using (McAdams, 1985 ) life story approach which covers factors ranging from childhood experiences to personality traits and peak experiences. The findings revealed that all participants eventually established a state of optimal functioning through resilience derived from internal and external protective factors. Major themes identified were negative experiences and emotions, positive emotions, benefit finding, presence of significant others, and pro-social behavior. Well-being was exhibited through positive consequences such as empathy, self-belief, and gratitude after facing hardship. The authors recommend using the case study approach in exploring resilience as individuals have unique coping mechanisms that are difficult to capture through quantitative measures. Additionally, case study approaches may be useful in understanding well-being outcomes in a variety of contexts which are otherwise difficult through preset questions. Finally, implications of the study and suggestions for future research have been proposed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info
• Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Anthony, E. J. (1974). Introduction: The syndrome of the psychologically vulnerable child. In E. J. Anthony & C. Koupernik (Eds.). The child in his family: Children at Psychiatric risk (Vol 3, pp. 3–10). Wiley.

Google Scholar  

Babson, K., & Feldner, M. (2010). Temporal relations between sleep problems and both traumatic event exposure and PTSD: A critical review of the empirical literature. Journal of Anxiety Disorders, 24 (1), 1–15. https://doi.org/10.1016/j.janxdis.2009.08.002

Article   Google Scholar  

Baker, J. C., Williams, J. K., Witvliet, C. V., & Hill, P. C. (2017). Positive reappraisals after an offense: Event-related potentials and emotional effects of benefit-finding and compassion. The Journal of Positive Psychology, 12 (4), 373–384. https://doi.org/10.1080/17439760.2016.1209540

Boals, A., Banks, J. B., Hathaway, L. M., & Schuettler, D. (2011). Use of cognitive words in stressful narratives reflects the meaning making process. Journal of Social and Clinical Psychology, 30 , 378–403. https://doi.org/10.1521/jscp.2011.30.4.378

Boehm, J. K., & Kubzansky, L. D. (2012). The heart’s content: The association between positive psychological well-being and cardiovascular health. Psychological Bulletin, 138 (4), 655–691. https://doi.org/10.1037/a0027448

Bonanno, G. A. (2004). Loss, trauma, and human resilience: Have we underestimated the human capacity to thrive after extremely aversive events? American Psychologist, 59 (1), 20–28. https://doi.org/10.1037/0003-066X.59.1.20

Bonanno, G. A., & Keltner, D. (1997). Facial expressions of emotion and the course of conjugal bereavement. Journal of Abnormal Psychology, 106 (1), 126–137. https://doi.org/10.1037/0021-843X.106.1.126

Bonanno, A., Moskowitz, J. T., Papa, A., & Folkman, S. (2005). Resilience to loss in bereaved spouses, bereaved parents, and bereaved gay men. Journal of Personality and Social Psychology, 88 (5), 827–843. https://doi.org/10.1037/0022-3514.88.5.827

Bonanno, G. A., Papa, A., & O’Neill, K. (2001). Loss and human resilience. Applied and Preventive Psychology, 10 (3), 193–206. https://doi.org/10.1016/S0962-1849(01)80014-7

Bonanno, G. A., Wortman, C. B., Lehman, D. R., Tweed, R. G., Haring, M., Sonnega, J., Carr, D., & Nesse, R. M. (2002). Resilience to loss and chronic grief: A prospective study from preloss to 18-months postloss. Journal of Personality and Social Psychology, 83 (5), 1150–1164. https://doi.org/10.1037/0022-3514.83.5.1150

Bonanno, G. A., Wortman, C. B., & Nesse, R. M. (2004). Prospective patterns of resilience and maladjustment during widowhood. Psychology and Aging, 19 (2), 260 271. https://doi.org/10.1037/0882-7974.19.2.260

Bowlby, J. (1960). Grief and mourning in infancy and early childhood. The Psychoanalytic Study of the Child, 15 (1), 9–52. https://doi.org/10.1080/00797308.1960.11822566

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3 (2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Brody, G., Yu, T., Chen, E., Miller, G., Kogan, S., & Beach, S. (2013). Is resilience only skin deep? Rural African Americans’ socioeconomic status-related risk and competence in preadolescence and psychological adjustment and allostatic load at age 19. Psychological Science, 24 (7), 1285–1293. https://doi.org/10.1177/0956797612471954

Buunk, B. P., Oldersma, F. L., & De Dreu, C. K. (2001). Enhancing satisfaction through downward comparison: The role of relational discontent and individual differences in social comparison orientation. Journal of Experimental Social Psychology, 37 (6), 452–467. https://doi.org/10.1006/jesp.2000.1465

Calhoun, L. G., & Tedeschi, R. G. (2004). The foundations of posttraumatic growth: New considerations. Psychological Inquiry, 15 (1), 102–193. https://doi.org/10.1207/s15327965pli1501_03

Cloninger, C. R. (2004). Feeling good: The science of well-being . Oxford University Press.

Carlton, B. S., Goebert, D. A., Miyamoto, R. H., Andrade, N. N., Hishinuma, E. S., Makini, G. K., Jr., Yuen, N. Y. C., Bell, C. K., McCubbin, L. D., Else, I. R. N., & Nishimura, S. T. (2006). Resilience, family adversity and well-being among Hawaiian and non-Hawaiian adolescents. International Journal of Social Psychiatry, 52 (4), 291–308. https://doi.org/10.1177/0020764006065136

Cohn, M. A., Fredrickson, B. L., Brown, S. L., Mikels, J. A., & Conway, A. M. (2009). Happiness unpacked: positive emotions increase life satisfaction by building resilience. Emotion, 9 (3), 361–368. https://doi.org/10.1037/a0015952

Diener, E. (1984). Subjective well-being. Psychological Bulletin, 95 (3), 542–575. https://doi.org/10.1037/0033-2909.95.3.542

Dodge, R., Daly, A., Huyton, J., & Sanders, L. (2012). The challenge of defining well-being. International Journal of Wellbeing, 2 (3), 222–235. https://doi.org/10.5502/ijw.v2i3.4

Duggal, D., Sacks-Zimmerman, A., & Liberta, T. (2016). The impact of hope and resilience on multiple factors in neurosurgical patients. Cureus, 8 (10), e849. https://doi.org/10.7759/cureus.849

Duchenne de Boulogne, G. B. (1862).  Mécanisme de la Physionomie Humaine: Texte. Renouard.

Erikson, E. H. (1963). Childhood and Society (2nd ed.). Norton.

Fredrickson, B. L., Cohn, M. A., Coffey, K. A., Pek, J., & Finkel, S. M. (2008). Open hearts build lives: positive emotions, induced through loving-kindness meditation, build consequential personal resources. Journal of Personality and Social Psychology, 95 (5), 1045–1062. https://doi.org/10.1037/a0013262

Fredrickson, B. L., Tugade, M. M., Waugh, C. E., & Larkin, G. R. (2003). What good are positive emotions in crisis? A prospective study of resilience and emotions following the terrorist attacks on the United States on September 11th, 2001. Journal of Personality and Social Psychology, 84 (2), 365–376. https://doi.org/10.1037/0022-3514.84.2.365

Frieswijk, N., Buunk, B. P., Steverink, N., & Slaets, J. P. (2004). The effect of social comparison information on the life satisfaction of frail older persons. Psychology and Aging, 19 (1), 183–190. https://doi.org/10.1037/0882-7974.19.1.183

Gable, S. L., & Haidt, J. (2005). What (and why) is positive psychology? Review of General Psychology, 9 (2), 103–110. https://doi.org/10.1037/1089-2680.9.2.103

Garmezy, N. (1974). The study of competence in children at risk for severe psychopathology. In E. J. Anthony & C. Koupernik (Eds.). The child in his family: Children at Psychiatric risk (Vol 3, pp. 77–97). Wiley.

Garmezy, N., & Masten, A. S. (1986). Stress, competence, and resilience: Common frontiers for therapist and psychopathologist. Behavior Therapy, 17 (5), 500–521. https://doi.org/10.1016/S0005-7894(86)80091-0

George, A. L., & Bennett, A. (2005). Case studies and theory development in social science . MIT Press.

Ginzburg, K., Ein-Dor, T., & Solomon, Z. (2010). Comorbidity of posttraumatic stress disorder, anxiety, and depression: A 20-year longitudinal study of war veterans. Journal of Affective Disorders, 123 (1–3), 249–257. https://doi.org/10.1016/j.jad.2009.08.006

Grossman, F. K., Sorsoli, L., & Kia‐Keating, M. (2006). A gale force wind: Meaning making by male survivors of childhood sexual abuse. American Journal of Orthopsychiatry, 76 (4),434–443. https://doi.org/10.1037/0002-9432.76.4.434

Helgeson, V. S., Reynolds, K. A., & Tomich, P. L. (2006). A meta-analytic review of benefit finding and growth. Journal of Consulting and Clinical Psychology, 74 (5), 797–816. https://doi.org/10.1037/0022-006X.74.5.797

Hess, U., Banse, R., & Kappas, A. (1995). The intensity of facial expression is determined by underlying affective state and social situation. Journal of Personality and Social Psychology, 69 (2), 280–288. https://doi.org/10.1037/0022-3514.69.2.280

Hobfoll, S. E. (2001). The influence of culture, community, and the nested-self in the stress process: Advancing conservation of resources theory. Applied Psychology, 50 (3), 337–421. https://doi.org/10.1111/1464-0597.00062

Hu, T., Zhang, D., & Wang, J. (2015). A meta-analysis of the trait resilience and mental health. Personality and Individual Differences, 76 , 18–27. https://doi.org/10.1016/j.paid.2014.11.039

Isen, A. M., Daubman, K. A., & Nowicki, G. P. (1987). Positive affect facilitates creative problem solving. Journal of Personality and Social Psychology, 52 (6), 1122–1131. https://doi.org/10.1037/0022-3514.52.6.1122

Jakobs, E., Manstead, A. S., & Fischer, A. H. (1999). Social motives and emotional feelings as determinants of facial displays: The case of smiling. Personality and Social Psychology Bulletin , 25 (4), 424–435. https://doi.org/10.1177/2F0146167299025004003

Kross, E., Bruehlman-Senecal, E., Park, J., Burson, A., Dougherty, A., Shablack, H., Bremner, R., Moser, J., & Ayduk, O. (2014). Self-talk as a regulatory mechanism: How you do it matters. Journal of Personality and Social Psychology, 106 (2), 304–324. https://doi.org/10.1037/a0035173

Lifton, R. J. (1993). The protean self: Human resilience in the age of fragmentation . Basic Books.

Lilgendahl, J. P., & McAdams, D. P. (2011). Constructing stories of self-growth: How individual differences in patterns of autobiographical reasoning relate to well-being in midlife. Journal of Personality, 79 (2), 391–428. https://doi.org/10.1111/j.1467-6494.2010.00688.x

Lim, D., & DeSteno, D. (2016). Suffering and compassion: The links among adverse life experiences, empathy, compassion, and prosocial behavior. Emotion, 16 (2), 175–182. https://doi.org/10.1037/emo0000144

Linley, P. A., & Joseph, S. (2004). Positive change following trauma and adversity: A review. Journal of Traumatic Stress, 17 (1), 11–21. https://doi.org/10.1023/B:JOTS.0000014671.27856.7e

Lundman, B., Aléx, L., Jonsén, E., Norberg, A., Nygren, B., Fischer, R. S., & Strandberg, G. (2010). Inner strength—A theoretical analysis of salutogenic concepts. International Journal of Nursing Studies, 47 (2), 251–260. https://doi.org/10.1016/j.ijnurstu.2009.05.020

Luthar, S. S. (2006). Resilience in development: A synthesis of research across five decades. In D. Cicchetti & D. J. Cohen (Eds.). Developmental psychopathology: Risk, disorder, and adaptation (pp. 740–795). Wiley. https://doi.org/10.1002/9780470939406.ch20 .

Marks, N., & Shah, H. (2004). A well-being manifesto for a flourishing society. Journal of Public Mental Health, 3 (4), 9–15. https://doi.org/10.1108/17465729200400023

Maslow, A. H. (1954). Motivation and personality . Harper and Row.

Masten, A. S., & Powell, J. L. (2003). A resilience framework for research, policy, and practice. In S. S. Luthar (Ed.),  Resilience and vulnerability: Adaptation in the context of childhood adversities ( pp. 1–25). Cambridge University Press. https://doi.org/10.1017/CBO9780511615788.003

McAdams, D. P. (1985). Power, intimacy, and the life story: Personological inquiries into identity . Dorsey Press.

McAdams, D. P. (1996). Personality, modernity, and the storied self: A contemporary framework for studying persons. Psychological Inquiry, 7 (4), 295–321. https://doi.org/10.1207/s15327965pli0704_1

Miller, E. D., & Omarzu, J. (1998). New directions in loss research. In J. H. Harvey (Ed.), Perspectives on loss: A sourcebook (pp. 3–20). Routledge. https://doi.org/10.4324/9781315803562 .

Morin, A., Duhnych, C., & Racy, F. (2018). Self-reported inner speech use in university students. Applied Cognitive Psychology, 32 (3), 376–382. https://doi.org/10.1002/acp.3404

Obradović, J., Shaffer, A., & Masten, A. S. (2012). Risk and adversity in developmental psychopathology: Progress and future directions . In L. C. Mayes & M. Lewis (Eds.), Cambridge handbooks in psychology. The Cambridge handbook of environment in human development (pp. 35–57). Cambridge University Press. https://doi.org/10.1017/CBO9781139016827.004 .

Ong, A. D., Bergeman, C. S., Bisconti, T. L., & Wallace, K. A. (2006). Psychological resilience, positive emotions, and successful adaptation to stress in later life. Journal of Personality and Social Psychology, 91 (4), 730–749. https://doi.org/10.1037/0022-3514.91.4.730

Perissinotto, C. M., Stijacic Cenzer, I., & Covinsky, K. E. (2012). Loneliness in older persons: A predictor of functional decline and death. Archives of Internal Medicine, 172 (14), 1078–1083. https://doi.org/10.1001/archinternmed.2012.1993

Peterson, C., & Park, N. (2003). Positive psychology as the even handed positive psychologist views it. Psychological Inquiry, 14 , 141–146. https://doi.org/10.1207/S15327965PLI1402_03

Picardi, A., Bartone, P. T., Querci, R., Bitetti, D., Tarsitani, L., Roselli, V., Flynn, B., et al. (2012). Development and validation of the Italian version of the 15-item dispositional resilience scale. Rivista Di Psichiatria, 47 (3), 231–237. https://doi.org/10.1708/1128.12446

Ponzetti, J. (2003). International encyclopedia of marriage and family (2nd ed.). Macmillan.

Rando, T. A. (1993). Treatment of complicated mourning . Research Press.

Reiber, G. E., McFarland, L. V., Hubbard, S., Maynard, C., Blough, D. K., Gambel, J. M., & Smith, D. G. (2010). Service members and veterans with major traumatic limb loss from Vietnam war and OIF/OEF conflicts: Survey methods, participants, and summary findings. Journal of Rehabilitation Research and Development, 47 (4), 299–316. https://doi.org/10.1682/JRRD.2010.01.0009

Rozin, P., & Royzman, E. B. (2001). Negativity bias, negativity dominance, and contagion. Personality and Social Psychology Review, 5 (4), 296–320. https://doi.org/10.1207/S15327957PSPR0504_2

Rutter, M. (1979). Protective factors in children’s responses to stress and disadvantage. In M. W. Kent & J. E. Rolf (Eds.). Primary prevention in psychopathology: Social competence in children. (Vol. 8, pp. 49–74). University Press of New England.

Rutter, M. (1987). Psychosocial resilience and protective mechanisms. American Journal of Orthopsychiatry, 57 (3), 316–331. https://doi.org/10.1111/j.1939-0025.1987.tb03541.x

Rutter, M. (2012). Resilience as a dynamic concept. Development and Psychopathology, 24 (2), 335–344. https://doi.org/10.1017/S0954579412000028

Ryff, C. D. (2014). Psychological well-being revisited: Advances in the science and practice of eudaimonia. Psychotherapy and Psychosomatics, 83 (1), 10–28. https://doi.org/10.1159/000353263

Sagone, E., & De Caroli, M. E. (2013). Relationships between resilience, self-efficacy, and thinking styles in Italian middle adolescents. Procedia-Social and Behavioral Sciences, 92 , 838–845. https://doi.org/10.1016/j.sbspro.2013.08.763

Schultze-Lutter, F., Schimmelmann, B. G., & Schmidt, S. J. (2016). Resilience, risk, mental health and well-being: Associations and conceptual differences [Editorial]. European Child and Adolescent Psychiatry, 25 (5), 459–466. https://doi.org/10.1007/s00787-016-0851-4

Seery, M. D. (2011). Resilience: A silver lining to experiencing adverse life events? Current Directions in Psychological Science, 20 (6), 390–394. https://doi.org/10.1177/0963721411424740

Sehmi, R., Maughan, B., Matthews, T., & Arseneault, L. (2019). No man is an island: Social resources, stress and mental health at mid-life. The British Journal of Psychiatry , 1–7. https://doi.org/10.1192/bjp.2019.25

Seligman, M. E. P., Steen, T. A., Park, N., & Peterson, C. (2005). Positive psychology progress: Empirical validation of interventions. American Psychologist, 60 (5), 410–421. https://doi.org/10.1037/0003-066X.60.5.410

Seligman, M. E. P., & Csikszentmihalyi, M. (2000). Positive psychology: An introduction. American Psychologist, 55 (1), 5–14. https://doi.org/10.1037/0003-066X.55.1.5

Simons, H. (2009). Case study research in practice. Sage Publications . https://doi.org/10.4135/9781446268322

Snyder, C. R., & Lopez, S. J. (2009). Oxford handbook of positive psychology . Oxford University Press.

Souri, H., & Hasanirad, T. (2011). Relationship between Resilience, Optimism and Psychological Well-being in Students of Medicine. Procedia-Social and Behavioral Sciences, 30 ,1541–1544. https://doi.org/10.1016/j.sbspro.2011.10.299

Staub, E., & Vollhardt, J. (2008). Altruism born of suffering: The roots of caring and helping after victimization and other trauma. American Journal of Orthopsychiatry., 78 (3), 267–280. https://doi.org/10.1037/a0014223

Tedeschi, R. G., & Calhoun, L. G. (2004). Posttraumatic growth: Conceptual foundations and empirical evidence. Psychological Inquiry, 15 (1), 1–18. https://doi.org/10.1207/s15327965pli1501_01

Tennen, H., & Affleck, G. (2002). Benefit-finding and benefit-reminding. In C. R. Snyder & S. J. Lopez (Eds.), Handbook of positive psychology (pp. 584–597). Oxford University Press.

Valle, M. F., Huebner, E. S., & Suldo, S. M. (2006). An analysis of hope as a psychological strength. Journal of School Psychology, 44 (5), 393–406. https://doi.org/10.1016/j.jsp.2006.03.005

Viorst, J. (1986). Necessary losses: The loves, illusions, dependencies and impossible expectations that all of us have to give up in order to grow. Fawcett Gold Medal.

Vollhardt, J. R. (2009). Altruism born of suffering and prosocial behavior following adverse life events: A review and conceptualization. Social Justice Research, 22 (1), 53–97. https://doi.org/10.1007/s11211-009-0088-1

Ward, M. C., & Higgs, P. (1998). Psychosocial aspects of adjusting to disability in older people. Reviews in Clinical Gerontology, 8 (3), 251–256. https://doi.org/10.1017/S0959259898008375

Waters, T. E., & Fivush, R. (2015). Relations between narrative coherence, identity, and psychological well‐being in emerging adulthood. Journal of Personality, 83 (4), 441–451. https://doi.org/10.1111/jopy.12120

Wayment, H. A. (2004). It could have been me: Vicarious victims and disaster-focused distress. Personality and Social Psychology Bulletin, 30 (4), 515–528. https://doi.org/10.1177%0146167203261892

Werner, E., & Smith, R. (1982). Vulnerable but invincible: A study of resilient children. McGraw-Hill.

White, R. E., Kross, E., & Duckworth, A. L. (2015). Spontaneous self-distancing and adaptive self-reflection across adolescence. Child Development, 86 (4), 1272–1281. https://doi.org/10.1111/cdev.12370

Wills, T. A. (1981). Downward comparison principles in social psychology. Psychological Bulletin, 90 (2), 245–271. https://doi.org/10.1037/0033-2909.90.2.245

Wilmoth, J. M. (2000). Unbalanced social exchanges and living arrangement transitions among older adults. The Gerontologist, 40 (1), 64–74. https://doi.org/10.1093/geront/40.1.64

Witvliet, C. V. O., Knoll, R. W., Hinman, N. G., & DeYoung, P. A. (2010). Compassion-focused reappraisal, benefit-focused reappraisal, and rumination after an interpersonal offense: Emotion-regulation implications for subjective emotion, linguistic responses, and physiology. The Journal of Positive Psychology, 5 (3), 226–242. https://doi.org/10.1080/17439761003790997

Wright, M. O. D., Masten, A. S., & Narayan, A. J. (2013). Resilience processes in development: Four waves of research on positive adaptation in the context of adversity. In S. Goldstein & R. B. Brooks (Eds.). Handbook of resilience in children (pp. 15–37). Springer. https://doi.org/10.1007/978-1-4614-3661-4_2 .

Yates, T. M., & Grey, I. K. (2012). Adapting to aging out: Profiles of risk and resilience among emancipated foster youth. Development and Psychopathology, 24 (2), 475–492. https://doi.org/10.1017/S0954579412000107

Download references

Author information

Authors and affiliations.

Department of Liberal Arts, Indian Institute of Technology, Hyderabad, India

Shikha Soni & Amrita Deb

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Shikha Soni .

Editor information

Editors and affiliations.

Sriperumbudur, Tamil Nadu, India

Sibnath Deb

Florida, FL, USA

Brian A. Gerrard

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Soni, S., Deb, A. (2022). Resilience and Well-Being: Case Studies of Four Individuals Who Have Undergone Adversities. In: Deb, S., Gerrard, B.A. (eds) Handbook of Health and Well-Being. Springer, Singapore. https://doi.org/10.1007/978-981-16-8263-6_28

Download citation

DOI : https://doi.org/10.1007/978-981-16-8263-6_28

Published : 08 March 2022

Publisher Name : Springer, Singapore

Print ISBN : 978-981-16-8262-9

Online ISBN : 978-981-16-8263-6

eBook Packages : Behavioral Science and Psychology Behavioral Science and Psychology (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Study Design 101

• Helpful formulas
• Finding specific study types

Some helpful formulas

• Meta- Analysis
• Systematic Review
• Practice Guideline
• Randomized Controlled Trial
• Cohort Study
• Case Control Study
• Case Reports

Sensitivity and Specificity

Test result

b = false negative c = false positive

To estimate sensitivity

The number of positive test results for the presence of an outcome (a) divided by the total presence of an outcome (a+b) Sensitivity = a / (a+b)

To estimate specificity

Number of negative test results for the absence of an outcome (d) divided by the total absences of an outcome (c + d) Specificity = d / (c+d)

False Positive and False Negative rates

Test Result

To calculate rate of false positives

The number of false positive test results for an outcome (c) divided by the total number of absences of an outcome (c+d) Rate of false positives = c / (c+d)

To calculate the rate of false negatives

The number of false negative test results for an outcome (b) divided by the total number of presences of an outcome (a+b) Rate of false negatives = b / (a+b)

Positive Predictive Value and Negative Predictive Value

To estimate positive predictive value

The number of positive test results for the presence of an outcome (a) divided by the total number of positive test results (a+c). Positive predictive value = a / (a+c)

To estimate negative predictive value

The number of negative test results for the absence of an outcome (d) divided by the total number of negative test results (b+d). Negative predictive value = d / (b+d)

Note: the formulas for positive predictive value and negative predictive value are accurate if the prevalence of the outcome (presences) is known.

Relative Risk

Relative Risk  = (a / a+b) / (c / c+d)

Smelly Shoes Example

In this example, 9 of the 10 pairs of sneakers that were worn without socks were smelly, and 2 of the 10 pairs of sneakers worn with socks were smelly. The relative risk would be (9/10) / (2/10), or 4.5. Therefore, the data suggest it is four times more likely to have smelly shoes if shoes are worn without socks.

Things to note about this formula:

• If the relative risk < 1 the exposure/incidence is protective: it lowers the risk for expressing the outcome.
• If the relative risk = 1 there is no association between an exposure that delineates the cohorts and the outcome.
• If the relative risk > 1 there is an association between an exposure that delineates the cohorts and the outcome (as seen in the example).

Attributable Risk

To calculate attributable risk

Subtract the outcome incidence rate of the control group from the outcome incidence rate of the experimental group. Attributable risk = (a-c) Attributable risk is helpful in showing to what extent the exposure to the variable of interest relates to the outcome studied.

In our smelly shoe example, attributable risk would be 7. This is interpreted as: "The risk of smelly shoes can be attributed to wearing shoes without socks in seven cases."

To calculate the odds ratio

The number of people in the "variable present" cohort that experiences an outcome (a) divided by the number of people in the reference cohort that experiences the outcome (b) to the number of people in the "variable present" cohort that experiences no outcome (c) divided by the number of people in the reference cohort that experiences no outcome (d). Odds ratio = (a/b) / (c/d)

Helpful hint: This formula can be simplified to ad/bc.

Odds Ratio in an unmatched study

Odds Ratio  = (a/c) / (b/d) = ad /bc

An Odds Ratio of unity means that cases are no more likely to be exposed to the risk factor than controls.

Odds ratio in a matched study

In a 1:1 matching, a case is paired with a control based on a similar characteristic (e.g. age), and the exposure is assessed in this pair.

f = a pair in which the control is not exposed and the case is exposed g = a pair in which the control is exposed and the case is not exposed Odds ratio  = f / g

← Previous Next →

© 2011-2019, The Himmelfarb Health Sciences Library Questions? Ask us .

• Himmelfarb Intranet
• Privacy Notice
• Terms of Use
• GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Brief Communication
• Open access
• Published: 12 August 2023

The need for a clinical case definition in test-negative design studies estimating vaccine effectiveness

• Sheena G. Sullivan   ORCID: orcid.org/0000-0002-0856-0294 1 , 2 ,
• Arseniy Khvorov 1 ,
• Xiaotong Huang 3 ,
• Can Wang 3 ,
• Kylie E. C. Ainslie   ORCID: orcid.org/0000-0001-5419-7206 3 , 4 ,
• Joshua Nealon   ORCID: orcid.org/0000-0003-1538-4636 3 ,
• Bingyi Yang 3 ,
• Benjamin J. Cowling   ORCID: orcid.org/0000-0002-6297-7154 3 , 5   na1 &
• Tim K. Tsang 3   na1  

npj Vaccines volume  8 , Article number:  118 ( 2023 ) Cite this article

1786 Accesses

4 Citations

10 Altmetric

Metrics details

• Epidemiology
• Influenza virus

Test negative studies have been used extensively for the estimation of COVID-19 vaccine effectiveness (VE). Such studies are able to estimate VE against medically-attended illness under certain assumptions. Selection bias may be present if the probability of participation is associated with vaccination or COVID-19, but this can be mitigated through use of a clinical case definition to screen patients for eligibility, which increases the likelihood that cases and non-cases come from the same source population. We examined the extent to which this type of bias could harm COVID-19 VE through systematic review and simulation. A systematic review of test-negative studies was re-analysed to identify studies ignoring the need for clinical criteria. Studies using a clinical case definition had a lower pooled VE estimate compared with studies that did not. Simulations varied the probability of selection by case and vaccination status. Positive bias away from the null (i.e., inflated VE consistent with the systematic review) was observed when there was a higher proportion of healthy, vaccinated non-cases, which may occur if a dataset contains many results from asymptomatic screening in settings where vaccination coverage is high. We provide an html tool for researchers to explore site-specific sources of selection bias in their own studies. We recommend all groups consider the potential for selection bias in their vaccine effectiveness studies, particularly when using administrative data.

Similar content being viewed by others

Bias assessment of a test-negative design study of COVID-19 vaccine effectiveness used in national policymaking

Mapping global acceptance and uptake of COVID-19 vaccination: A systematic review and meta-analysis

Unrepresentative big surveys significantly overestimated US vaccine uptake

Introduction.

Since the initial roll-out of COVID-19 vaccines, the test-negative design has been frequently applied to enable timely monitoring of COVID-19 vaccine effectiveness (VE) 1 . This design has been extensively used for estimation of influenza VE 2 , for which studies have often leveraged sentinel surveillance systems where patients presenting with a particular clinical case definition are enroled from ambulatory or inpatient medical facilities, regardless of their vaccination status, and tested for the pathogen of interest. Those patients testing positive are identified as cases, while those testing negative are identified as non-cases. VE is estimated from the odds ratio comparing the odds of vaccination among the cases versus non-cases, adjusting for important confouders 3 , 4 . Here, the term “non-cases” is deliberately used because case status is not known at the time of enrolment, and no sampling frame is used to guide recruitment of cases and non-cases, which differentiates the test-negative design from the traditional case-control study.

The test-negative design has been extensively validated for influenza 4 , 5 , 6 , 7 , 8 , 9 , usually under the scenario described above. We have previously reviewed its application to other pathogens and have cautioned that its suitability needs to be re-examined for each new use 2 . The applicability of the test-negative design for monitoring COVID-19 VE was not examined until after widespread use and several possible weaknesses were highlighted 10 .

Here, we focus on one key design feature of the test-negative design that has been variously implemented: the restriction of participants to those meeting a clinical case definition. Prior to COVID-19, laboratory tests for confirmation of infection were typically only conducted on people with clinical symptoms. However, given the pre-symptomatic transmission potential of COVID-19 cases, laboratory tests were conducted on many people without symptoms, so some studies using the test-negative design may include participants that would not meet a clinical case definition. Notwithstanding other sources of bias, the use of a clinical case definition is an attempt to ensure that cases and non-cases are derived from the same source population; i.e., patients who would have presented for care with the disease of interest and been enroled as cases had they tested positive for the pathogen of interest. The causal model is depicted in Fig. 1 .

In ( a ), health-care-seeking behaviour HS confounds the relationship between vaccination V, SARS-CoV-2 infection status SI (e.g., by influencing engagement in risk behaviours), and COVID-19 status C19 (e.g., because of other healthy behaviours that modify disease severity). Only patients who are tested for SARS-CoV-2 are selected into the study S  = 1. An individual’s health-care-seeking behaviour HS and COVID-19 status C19 influence whether they present for care, are tested and selected into the study S  = 1, resulting in collider bias. In ( b ), the test-negative design by restricting participants to those who present to sentinel sites and meet particular clinical criteria HS = 1, the collider bias introduced by S  = 1 is blocked enabling unbiased estimation of the V-C19 effect.

Clinical restriction underscores two key features of test-negative studies. First, in this design, VE is not estimated against infection per se, but estimates the vaccine’s effectiveness at preventing medically-attended illness (or hospitalised illness, if enrolment is in hospitals). Second, failure to restrict the population in this way breaks the assumption that cases and non-cases are derived from the same source population 10 . This problem relates to the selection bias that might be induced by differential health seeking between cases and non-cases 6 , 8 , 10 . Lewnard et al. explored this problem and noted that in scenarios where healthcare seeking is correlated with vaccination, ignoring it inflates VE estimates 10 .

Studies using health services databases may be at greatest risk of this selection bias. These studies typically use data collected for administrative purposes rather than for the study in question. They may assimilate results on a broad range of individuals tested for a variety of reasons. For example, administrative datasets may include a high proportion of people tested asymptomatically as part of screening programmes, close contacts tested to clear isolation, or the worried well. The pool of negative test results may be over-represented by people whose degree of risk was associated with their vaccination status (e.g., because their workplace requires both asymptomatic screening and vaccination), which can result in a higher proportion of unvaccinated cases leading to higher VE estimates.

Evidence from a systematic review

To demonstrate the problem, we explored VE estimates extracted as part of a systematic review 1 of test-negative design studies that estimated VE against medically attended COVID-19 illness and severe disease (hospitalisation, admission to intensive care unit and/or death) for a primary course of vaccination. Full details are provided elsewhere 1 , but briefly, papers were included if the authors described the study as a test-negative design or all participants included in the analysis had been tested for SARS-CoV-2, irrespective of clinical criteria. Data were extracted using a standard data collection form, which included whether or not the study used clinical criteria for enrolment.

The search was last updated 11 July 2022 and identified 66 studies that met our inclusion criteria (Supplementary Table 1 ). Forty-one studies used clinical criteria for enrolment, while 25 did not (Supplementary Tables 1 and 2 ). Pooled VE was estimated using random effects meta-analysis. VE against medically-attended illness from studies that did not use clinical criteria was higher (VE: 87%; 95% CI: 83%, 90%) than studies that used clinical criteria (VE: 81%; 95% CI: 78%, 83%; Fig. 2 ), representing a ratio of odds ratios (ROR), 1.44 (95% CI: 1.08, 1.91). VE against severe disease was also higher in studies that did not use clinical criteria (VE: 93%; 95% CI: 91%, 95% versus VE: 87%; 95% CI: 84%, 90%; Fig. 2 ), corresponding to an ROR of 1.92 (95% CI: 1.30, 2.85). In meta-regression these ratios were recalculated adjusting for whether the study included participants with prior infection, the predominant SARS-CoV-2 circulating variant and the type of vaccine used. These adjustments reduced the RORs to 1.17 (95% CI: 0.95, 1.46) for medically-attended illness and 1.48 (95% CI: 1.08, 2.04) for severe illness, suggesting that clinical criteria may be more important for studies of severe disease.

a shows VE against medically-attended illness, while ( b ) shows VE against severe disease. Points indicate the VE point estimate from each study without confidence intervals. Black points with lines show the pooled estimate from the random-effect meta-analysis with 95% confidence intervals. Shaded area is the violin plot, which is the smoothed density of the VE point estimates.

We note that some studies using administrative data have restricted the study sample to individuals with certain discharge codes to approximate a clinical case definition 11 . However, discharge diagnoses are assigned after testing, so this approach may still fail to achieve exchangeability between cases and non-cases in terms of their clinical indications for testing. Moreover, such an approach is contingent on assuming that testing was not influenced by the patient’s vaccination status. When test-negative studies are run prospectively, participating providers can be reminded to remain impartial about vaccination status when sampling patients.

Evidence from simulations

We also sought to demonstrate the impact of this form of selection bias using a simple simulation. The associated R script is provided in the Supplementary Information and at https://github.com/khvorov45/casedef . We assumed cases are all people with a positive test result, which includes people infected with SARS-CoV-2 who have symptoms (e.g., identified through symptomatic testing) and people infected with SARS-CoV-2 who do not have symptoms (e.g., identified through asymptomatic screening). Non-cases are all people with a negative test result, some of whom have symptoms and are infected with anything other than SARS-CoV-2, and some of whom have no symptoms and are not infected with SARS-CoV-2 (we will call them “healthy” to differentiate them from people who have an infection). Table 1 shows the default simulation parameters under which the VE estimate from a test-negative study is unbiased.

We first explored the scenario where the asymptomatic proportion was allowed to vary by case status but did not vary by vaccination status. The bias in this situation is negligible (diagonals in Fig. 3a, b ).

Expected bias (estimated VE minus true VE) is shown at different values of proportion of asymptomatic (healthy) people who are part of the study as non-cases and proportion of asymptomatically people who are included as cases (proportion is the same for the vaccinated and the unvaccinated). It show the bias when the proportion asymptomatic is differential by vaccination status in non-cases ( a ) and cases ( b ). The non-differential case is also shown along the diagonal in ( a ) and ( b ) and while non-zero is negligible and not visible on the plot. Note that for ( b ) this is because the proportion of asymptomatic infections among all infections is the same for the vaccinated and the unvaccinated in the simulation under the default parameter set. c It shows selected values exploring the bias at different asymptomatic proportions by both vaccination and case status. Axis labels are understood as follows: “25%V 75%UV” indicates that for the vaccinated the proportion asymptomatic is set to 25%, for the unvaccinated it is set to 75%. For all plots, the percent bias indicates the difference in VE estimate compared with the default value of 60%; e.g., a value of −17% means the estimated value is VE = 47%. All parameters other than the ones in the X and Y axes are set to their default values as per Table 1 .

Next, we examined the situation where some proportion of non-cases are healthy and would not be included if using a clinical case definition. This scenario might occur if the dataset includes people from workplaces that conduct asymptomatic screening. Figure 3a shows the effect of this bias. If those same workplaces also require vaccination, then the proportion of healthy vaccinated non-cases may be greater than the proportion of healthy unvaccinated non-cases. In this scenario, the expected VE estimate is biased positively away from the null (i.e., bottom right half of Fig. 3a ; VE is overestimated). When the proportion of healthy individuals is lower among the vaccinated compared with unvaccinated non-cases, which might occur if eligibility for travel or entertainment entrance is contingent on testing for the unvaccinated, VE is biased towards the null and can even be negative (i.e., top left half of Fig. 3a ; VE is underestimated). This bias is negligible at low disease prevalence even in the extreme case of both proportions being 100% (this would be equivalent to a standard case-control study).

The converse scenario showing bias that occurs when the asymptomatic proportion among the cases is varied is shown in Fig. 3b . When vaccination reduces symptoms severity 11 , and the proportion of asymptomatic cases is higher among the vaccinated, the estimate is biased towards the null (i.e., bottom right half of Fig. 3b ; VE is underestimated). This might occur if the dataset includes people working or resident in settings where vaccination is high (e.g., aged care) and testing identifies a high proportion of asymptomatic cases through screening during an outbreak. Note, however, that the scenarios in Fig. 3b result in less bias than those depicted in Fig. 3a .

If the asymptomatic proportions among cases and non-cases are not the same for the vaccinated and the unvaccinated, a compounding effect is observed (Fig. 3c ). For example, if the proportion asymptomatic in cases is greater in the vaccinated, we know from Fig. 3b the bias will be negative. If the proportion healthy in non-cases is greater in the unvaccinated, we know from Fig. 3a the bias will be negative. When both are true, the bias becomes more negative and pulls estimates further from their true value. In some scenarios, the bias may cancel out, such as when the proportion asymptomatic in cases is greater in the vaccinated, and proportion healthy in non-cases is greater in the vaccinated. To realise the inflated VE seen in the systematic review, the most likely scenario is one where the healthy proportion among vaccinated non-cases is higher than among unvaccinated non-cases (i.e., columns marked 50%V 0%UV or 50%V 10%UV), irrespective of the asymptomatic proportion among the cases. However, there are numerous possible scenarios and the degree of bias will change under different default parameter values. Further combinations of parameter values can be explored using an html tool available at https://github.com/khvorov45/casedef .

Conclusions

Rapid VE estimation, especially estimation that leverages administrative data and can therefore be done less expensively than studies which follow a sampling framework, is an attractive option. However, research groups and policy makers need to understand the pitfalls of this approach.

The application of a clinical case definition in test-negative studies provides some reassurance that the non-case group reflects the source population of the cases 12 . While this requirement increases the likelihood that the non-cases have a similar risk of exposure to the SARS-CoV-2 virus, it does not guarantee it. Some non-cases may still fail to meet the exposure necessity assumption 12 ; i.e., some non-cases may not, in fact, have been exposed to the virus and were therefore never at risk of COVID-19 illness. Moreover, the use of clinical criteria seeks to address internal validity; generalisability is limited to the healthcare seeking population 13 . In some special cases, it may be possible to estimate VE in the whole population; for example, when participants are recruited through point-prevalence surveys 14 or in studies that limit participants to close contacts of a case such as household transmission studies 15 . However, those approaches may still suffer from participation bias 13 .

Salvaging internal validity, at a minimum, is important for public health decision making. In VE studies, generalising to the healthcare-seeking population may be satisfactory since it is the burden on our health systems we wish to mitigate with vaccination. Where selection processes fail to ensure the study sample represents the source population, various methods exist to correct the resultant selection bias, but may require additional information unavailable to the researcher 16 , 17 , 18 , 19 . We recommend that all research groups perform an assessment of the degree to which VE is biased under selection scenarios relevant to their setting. The tool we have provided can help with this assessment.

When conducted with a clinical case definition in mind, test-negative studies may be able to provide valid estimates of VE against a specific syndrome of medically-attended disease. When the indications for testing are ignored, the resulting VE is unbiased only when the asymptomatic proportions included into cases and non-cases are the same for the vaccinated and the unvaccinated, which is rare. It is otherwise unclear what the VE estimate represents, but it is unlikely to be a measure of VE against infection, nor medically-attended illness, and is instead some hybrid, the public health implications of which are unclear (and possibly unhelpful). If the goal is to estimate VE against infection, not disease, the test-negative design is not the best design choice, and those choosing it need to acknowledge fully its limitations. The tool we have provided in the supplementary information can help researchers assess the potential for bias under scenarios most plausible for their population.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

Papers included in systematic review are listed in the appendix. Any further data extracted from reviewed articles can be provided upon request to Tim K. Tsang [email protected].

Code availability

R and html scripts used in simulations are available https://github.com/khvorov45/casedef . In addition, the R scripts for the simulations are provided with the Supplementary Information. R scripts used for meta-analysis and meta-regression can be provided upon request to [email protected].

Tsang, T. K. et al. Prior infections and effectiveness of SARS-CoV-2 vaccine in test-negative study: a systematic review and meta-analysis. medRxiv, 2022.2011.2003.22281925, https://doi.org/10.1101/2022.11.03.22281925 (2022).

Chua, H. et al. The use of test-negative controls to monitor vaccine effectiveness: a systematic review of methodology. Epidemiology 31 , 43–64 (2020).

Article   PubMed   PubMed Central   Google Scholar  

Sullivan, S. G. & Cowling, B. J. "Crude vaccine effectiveness" is a misleading term in test-negative studies of influenza vaccine effectiveness. Epidemiology 26 , e60 (2015).

Sullivan, S. G., Feng, S. & Cowling, B. J. Potential of the test-negative design for measuring influenza vaccine effectiveness: a systematic review. Expert Rev. Vaccines 13 , 1571–1591 (2014).

Article   PubMed   PubMed Central   CAS   Google Scholar  

Sullivan, S. G., Tchetgen Tchetgen, E. J. & Cowling, B. J. Theoretical basis of the test-negative study design for assessment of influenza vaccine effectiveness. Am. J. Epidemiol. 184 , 345–353 (2016).

Ainslie, K. E. C., Haber, M. & Orenstein, W. A. Bias of influenza vaccine effectiveness estimates from test-negative studies conducted during an influenza pandemic. Vaccine 37 , 1987–1993 (2019).

Foppa, I. M., Haber, M., Ferdinands, J. M. & Shay, D. K. The case test-negative design for studies of the effectiveness of influenza vaccine. Vaccine 31 , 3104–3109 (2013).

Article   PubMed   Google Scholar  

Jackson, M. L. & Nelson, J. C. The test-negative design for estimating influenza vaccine effectiveness. Vaccine 31 , 2165–2168 (2013).

Ainslie, K. E. C., Haber, M. & Orenstein, W. A. Challenges in estimating influenza vaccine effectiveness. Expert Rev. Vaccines 18 , 615–628 (2019).

Lewnard, J. A. et al. Theoretical framework for retrospective studies of the effectiveness of SARS-CoV-2 vaccines. Epidemiology 32 , 508–517 (2021).

Thompson, M. G. et al. Effectiveness of a third dose of mRNA vaccines against COVID-19-associated emergency department and urgent care encounters and hospitalizations among adults during periods of delta and omicron variant predominance - VISION Network, 10 States, August 2021-January 2022. MMWR Morb. Mortal. Wkly. Rep. 71 , 139–145 (2022).

Stensrud, M. J. Identification of vaccine effects when exposure status is unknown. Epidemiology 34 , 216–224 (2023).

Infante-Rivard, C. & Cusson, A. Reflection on modern methods: selection bias-a review of recent developments. Int. J. Epidemiol. 47 , 1714–1722 (2018).

Chadeau-Hyam, M. et al. SARS-CoV-2 infection and vaccine effectiveness in England (REACT-1): a series of cross-sectional random community surveys. Lancet Respir. Med. 10 , 355–366 (2022).

Halloran, M. E., Longini, I. M. Jr. & Struchiner, C. J. Design and interpretation of vaccine field studies. Epidemiol. Rev. 21 , 73–88 (1999).

Article   PubMed   CAS   Google Scholar  

Geneletti, S., Mason, A. & Best, N. Adjusting for selection effects in epidemiologic studies: why sensitivity analysis is the only "solution". Epidemiology 22 , 36–39 (2011).

Geneletti, S., Richardson, S. & Best, N. Adjusting for selection bias in retrospective, case-control studies. Biostatistics 10 , 17–31 (2009).

Greenland, S. Basic methods for sensitivity analysis of biases. Int. J. Epidemiol. 25 , 1107–1116 (1996).

Westreich, D. Berkson’s bias, selection bias, and missing data. Epidemiology 23 , 159–164 (2012).

Download references

Acknowledgements

This project was supported by the National Institute of General Medical Sciences (BJC; R01 GM139926), the National Institute of Allergy and Infectious Diseases (S.G.S.; R01 AI141534), and the Theme-based Research Scheme (B.J.C., Project No. T11–712/19-N) of the Research Grants Council of the Hong Kong SAR Government. B.J.C. is supported by an RGC Senior Research Fellowship (HKU SRFS2021–7S03) and the AIR@innoHK program of the Innovation and Technology Commission of the Hong Kong SAR Government. The WHO Collaborating Centre for Reference and Research on Influenza is funded by the Australian Government Department of Health and Aged Care.

Author information

These authors contributed equally: Benjamin J. Cowling, Tim K. Tsang.

Authors and Affiliations

WHO Collaborating Centre for Reference and Research on Influenza, Royal Melbourne Hospital, and Department of Infectious Diseases, University of Melbourne, at the Peter Doherty Institute for Infection and Immunity, Melbourne, VIC, Australia

Sheena G. Sullivan & Arseniy Khvorov

Department of Epidemiology, Fielding School of Public Health, University of California Los Angeles, Los Angeles, CA, USA

Sheena G. Sullivan

WHO Collaborating Centre for Infectious Disease Epidemiology and Control, School of Public Health, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong Special Administrative Region, China

Xiaotong Huang, Can Wang, Kylie E. C. Ainslie, Joshua Nealon, Bingyi Yang, Benjamin J. Cowling & Tim K. Tsang

Centre for Infectious Disease Control, National Institute for Public Health and the Environment, Bilthoven, the Netherlands

Kylie E. C. Ainslie

Laboratory of Data Discovery for Health Limited, Hong Kong Science and Technology Park, New Territories, Hong Kong Special Administrative Region, China

Benjamin J. Cowling

You can also search for this author in PubMed   Google Scholar

Contributions

S.G.S. conceptualised the study, led manuscript development and submission; B.J.C. and T.K.T. conceived and designed the systematic review; X.H., C.W. and T.K.T. reviewed papers and extracted data; T.K.T. led analysis of the extracted data; A.K. developed simulations in R and html; K.E.C.A., J.N., B.Y. contributed to interpretation and development of the manuscript.

Corresponding author

Correspondence to Sheena G. Sullivan .

Ethics declarations

Competing interests.

BJC reports honoraria from AstraZeneca, Fosun Pharma, GSK, Haleon, Moderna, Novavax, Pfizer, Roche and Sanofi Pasteur. SGS reports honoraria from CSL Seqirus, Evo Health, Moderna and Pfizer.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, reporting summary, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Sullivan, S.G., Khvorov, A., Huang, X. et al. The need for a clinical case definition in test-negative design studies estimating vaccine effectiveness. npj Vaccines 8 , 118 (2023). https://doi.org/10.1038/s41541-023-00716-9

Download citation

Received : 13 March 2023

Accepted : 01 August 2023

Published : 12 August 2023

DOI : https://doi.org/10.1038/s41541-023-00716-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Open access
• Published: 21 March 2023

Changing positive and negative affects through music experiences: a study with university students

• José Salvador Blasco-Magraner 1 ,
• Gloria Bernabé-Valero 2 ,
• Pablo Marín-Liébana 1 &
• Ana María Botella-Nicolás 1  

BMC Psychology volume  11 , Article number:  76 ( 2023 ) Cite this article

10k Accesses

1 Citations

4 Altmetric

Metrics details

Currently, there are few empirical studies that demonstrate the effects of music on specific emotions, especially in the educational context. For this reason, this study was carried out to examine the impact of music to identify affective changes after exposure to three musical stimuli.

The participants were 71 university students engaged in a music education course and none of them were musicians. Changes in the affective state of non-musical student teachers were studied after listening to three pieces of music. An inter-subject repeated measures ANOVA test was carried out using the Positive and Negative Affect Schedule (PANAS) to measure their affective state.

The results revealed that: (i) the three musical experiences were beneficial in increasing positive affects and reducing negative affects, with significant differences between the interaction of Music Experiences × Moment (pre-post); (ii) listening to Mahler’s sad fifth symphony reduced more negative affects than the other experimental conditions; (iii) performing the blues had the highest positive effects.

Conclusions

These findings provide applied keys aspects for music education and research, as they show empirical evidence on how music can modify specific affects of personal experience.

Peer Review reports

Introduction

The studies published on the benefits of music have been on the increase in the last two decades [ 1 , 2 , 3 ] and have branched out into different areas of research such as psychology [ 4 , 5 , 6 , 7 , 8 ], education [ 1 , 9 , 10 ] and health [ 11 , 12 ] providing ways of using music as a resource for people’s improvement.

The publication in 1996 of the famous report “Education Hides a Treasure” submitted to the UNESCO by the International Commission was an important landmark in the educational field. This report pointed out the four basic pillars of twenty-first century education: learning to know, learning to do, learning to live together, and learning to be [ 13 ]. The two last ones clearly refer to emotional education. This document posed a challenge to Education in terms of both academically and emotionally development at all levels from kindergarten to university. In this regard, there has been a notable increase in the number of studies that have shown the strong impact of music on the emotions in the different stages of education and our lives. For example, from childhood to adolescence, involving primary, secondary and university education, music is especially relevant for its beneficial effects on developing students’ emotional intelligence and prosocial skills [ 1 , 14 ]. In adults, music benefits emotional self-regulation [ 15 ], while in old age it helps to maintain emotional welfare and to experience and express spirituality [ 16 ]. This underlines the importance of providing empirical evidence on the emotional influence of music.

Influence of music on positive affects

Numerous studies have used the Positive and Negative Affect Schedule (PANAS) to evaluate the emotional impact of music [ 17 ]. This scale is valid and effective for measuring the influence of positive and negative effects of music on listeners and performers [ 10 , 18 , 19 ]. Thus, for example, empirical evidence shows that exposure to a musical stimulus favours the increase of positive affects [ 20 , 21 ] found a significant increase in three positive affects in secondary school students after listening to music, and the same results has been found after listening to diverse musical styles. These results are consistent with Schubert [ 22 ], who demonstrated that music seems to improve or maintain well-being by means of positive valence emotions (e. g. happiness, joy and calm). Other research studied extreme metal fans aged between 18 and 34 years old and found statements of physiological excitement together with increased positive affects [ 21 ]. Positive outcomes after listening to sad music have also been found [ 23 ], who played Samuel Barbers’ Adagio for Strings , described by the BBC as the world’s saddest piece of classical music, to 20 advanced music students and 20 advanced psychology students with no musical background and subsequently found that the music only had positive affects on both groups.

Several experimental designs that used sad music on university students noticed that they experienced both sadness and positive affects [ 24 , 25 ] and also found that music labeled as “happy” increased positive affects while the one labeled as “sad” reduced both positive and negative affects [ 26 ]. For other authors the strongest and most pleasant responses to sad music are associated with empathy [ 27 ]. Moreover, listening to sad music had benefits since attributes of empathy were intensified [ 27 , 28 ]. In relation to musical performances, empirical evidence found a significant increase in positive affects [ 29 ]. Thus, music induces listeners to experience positive affects, which could turn music into an instrument for personal development.

Following on from Fredrickson’s ‘broaden‐and‐build’ framework of positive emotions [ 30 ], positive affects cause changes in cognitive activities which, in turn, can cause behaviour changes. They can also expand the possibilities for action and improve physical resources. According to Fredrickson [ 30 ], positive affects trigger three sequential effects: (1) amplification of the scope for thought and action; (2) construction of personal resources to deal with difficult simplifications; (3) personal transformation by making one more creative, with a better understanding of situations, better able to face up to difficulties and better socially integrated. This leads to an “upward spiral” in which even more positive affects are experienced. A resource such as music that can increase positive affects, can therefore be considered as a step forward in personal transformation. Thus, music teachers could have a powerful tool to help students enhance their personal development.

Influence of music on negative affects

There is a great deal of controversy as regards the influence of music on negative affects. Blasco and Calatrava [ 20 ] found a significant reduction of five negative affects in secondary school students after listening to Arturo Marquez’s typically happy Danzón N O 2. Different results were found in an experiment in which the change in participants ‘affects was assessed after listening the happy "Eye of the Tiger" by Survivor and the sad "Everybody Hurts" by REM [ 26 ]. They found that the happy piece only increased the positive affects but did not reduce the negative ones, while the sad piece reduced both positive and negative affects. However, neither of these findings agree with Miller and Au [ 31 ], who carried out an experiment to compare the influence of sad and happy music on undergraduates ‘mood arousal and found that listening to both types had no significant changes on negative affects. Shulte [ 32 ] conducted a study with 30 university students to examine the impact that nostalgic music has on affects, and found that after listening to different songs, negative affects decreased. Matsumoto [ 33 ] found that sad music reduced sad feelings in deeply sad university students, while Vuoskoski and Eerola [ 34 ] showed that sad music could produce changes in memory and emotional judgements related to emotions and that experiencing music-induced sadness is intrinsically more pleasant than sad memories. It therefore seems that reducing negative affects has mostly been studied with sad and nostalgic musical stimuli. In this way, if music can reduce negative affects, it can also be involved in educational and psychological interventions focused on improving the emotional-affective sphere. Thus, for example, one study examined the effects of a wide range of music activities and found that it would be necessary to specify exactly what types of music activity lead to what types of outcomes [ 2 ]. Moore [ 3 ] also found that certain music experiences and characteristics had both desirable and undesirable effects on the neural activation patterns involved in emotion regulation. Furthermore, recent research on university students shows that music could be used to assess mood congruence effects, since these effects are reactions to the emotions evoked by music [ 35 ].

These studies demonstrate that emotional experience can be actively driven by music. Moreover, they synthesize the efforts to find ways in which music can enhance affective emotional experience by increasing positive affects and reducing the negative ones (e. g. hostility, nervousness and irritability). Although negative emotions have a great value for personal development and are necessary for psychological adjustment, coping with them and self-regulation capacities are issues that have concerned psychology. For example, Emotional Intelligence [ 36 ], which has currently been established in the educational field, constitutes a fundamental conceptual framework to increase well-being when facing negative emotions, providing keys for greater control and management of emotional reactions. It also establishes how to decrease the intensity and frequency of negative emotional states [ 37 ], providing techniques such as mindfulness meditation that have proven their effectiveness in reducing negative emotional experiences and increasing the positive ones [ 38 ]. The purpose of this research is to find whether music can be part of the varied set of resources that can be used by a teacher to modify students’ emotional experience.

Thus, although empirical evidence of the effects of music on the emotional sphere is still incipient. It seems that they can increase positive effects, but it is not clear their impact on the negative ones, since diverse and contradictory results (no change and reduction of negative affects after listening to music) were found. In addition, the effects of the type of musical piece (e.g. happy or sad music) need further investigation as different effects were found. Moreover, previous studies do not compare between the effects of listening to versus performing music. Such an approach could provide keys to highlight the importance of performing within music education. Therefore, this study aims to contribute to this scientific field, providing experimental evidence on the effects of listening to music as compared to performing music, as well as determining the effects of different types of music on positive and negative affects.

To this end, the effects of three different types of music experiences were compared: (1) listening to a sad piece, (2) listening to an epic and solemn piece, and (3) performing of a rhythm and a blues piece, to determine whether positive and negative affects were modified after exposure to these experimental situations. In particular, two hypotheses guided this study: (1) After exposure to each musical experience (listening to a sad piece; listening to a solemn piece and playing a blues), all participants will improve their emotional experience, increasing their positive affects and reducing their negative ones; and (2) the music performance will induce a greater change as compared to the listening conditions.

Participants

A total of 71 students were involved in this study, 6 men and 65 women between the ages of 20 and 40, who were studying a Teaching Grade. These students were enrolled in the "Music Education" program as part of their university degree’s syllabus. None of them had special music studies from conservatories, academies or were self-taught; thus, all had similar musical knowledge. None of them had previously listened to music in an instructional context nor had performed music with their fellow students. In addition, none of them had listening before to the musical pieces selected for this experiment.

All signed an informed consent form before participating and no payment was given for taking part in the study. As the experiment was carried out in the context of a university course, they were assured that their participation and responses would be anonymous and would have no impact on their qualifications. The research was approved by the ethical committee at the Universidad Católica de Valencia San Vicente Mártir: UCV2017- 18-28 code.

Questionnaire

To assess emotional states, the Positive and Negative Affective States scales (PANAS), was administered [ 39 ]. In particular, the Spanish version of the scale [ 17 ], whose study shows a high degree of internal consistency; in males 0.89 in positive affects and 0.91 in negative affects; in women 0.87 in positive affects and 0.89 in negative affects. In this study, good reliability level in each experimental condition was obtained (0.836–0.913 for positive affects and 0.805–0.917 for negative affects (see Table 1 for more information on Cronbach’s α for each experimental condition).

The PANAS consists of 20 items which describe different dimensions of emotional experience. Participants must answer them regarding to their current affective state. The scale is composed of 20 items; 10 positive affects (PA) and 10 negative affects (NA). Answers are graded in a 5-options (Likert scale), with reversed items, ranging from extremely (1) to very slightly or not at all (5).

Musical pieces

The musical pieces choice stemmed from the analysis of some of the music elements that most influence the perception of emotions: mode, melody and intervals. Within the melody, range and melodic direction were distinguished. The range or amplitude of the melodic line is commonly divided into wide or narrow, while the melodic direction is often classified as ascending or descending. Chang and Hoffman [ 10 ] associated narrow amplitude melodies with sadness, while Schimmark and Grob [ 40 ] related melodic amplitude with highly activated emotions. Regarding the melodic direction, Gerardi and Gerken [ 41 ] found a relationship between ascending direction and happiness and heroism, and between descending direction and sadness.

In relation to the mode, Tizón [ 42 ] stated that the major one is completely happy, while the minor one represents sadness. Thompson and Robitaille [ 43 ] considered that, in order to cause emotions such as happiness, solemnity or joy, composers use tonal melodies, while to obtain negative emotions, they use atonality and chromaticism.

In this research, the selected pieces (“Adagietto” from Gustav Mahler's Fifth Symphony, MML; and “Titans” from Alexander The Great from Vangelis, VML) are representative examples of the melodic, intervallic and modal characteristics previously exposed. Mahler's and Vangelis's pieces completely differ in modes and melodic amplitude (sad vs. heroism). Likewise, Mahler's piece is much more chromatic than Vangelis' one, which has a broader melody made up of third, fourth and fifth intervals, often representative of heroism. Those features justify the fact that they have been used as soundtracks in two films belonging to the epic genre (Alexander The Great, 2004) and drama (Death in Venice, 1971).

The musical piece that was performed by the students was chosen in order to be easy to learn in a few sessions, since they were not musicians. So, three musical pieces were used for the experimental conditions, the first two musical pieces were recordings in a CD, while the third one was performed by the subjects.

The three chosen pieces are described below:

Condition 1 (MML): “Adagietto” from Gustav Mahler’s Fifth Symphony (9:01 min), performed by the Berlin Philharmonic conducted by Claudio Abbado [ 44 ]. This is a sad, melancholic and dramatic piece that Luchino Visconti used in the film Death in Venice, made in 1971 and based on the book by Thomas Mann.

Condition 2 (VML): “Titans Theme” from Alexander the Great (3:59 min), directed by Oliver Stone and premiered in 2004, whose music was composed, produced and performed by Vangelis [ 45 ]. It has a markedly epic character with large doses of heroism and solemnity.

Condition 3 (BP): “Rhythm’s Blues” composed and played by Ana Bort (4:00 min). This is a popular African-American piece of music with an insistent rhythm and harmonically sustained by tonal degrees. This piece was performed by the participants using percussion instruments (carillons and a range of xylophones and metallophones).

The sample was divided into two groups (N 1  = 36 and N 2  = 35) that participated separately in all the phases of the study. The first two conditions (MML and VML) were carried out in each group's classroom, while the performance (BP) was developed in the musical instruments room. This room had 52 percussion instruments, including different types of chimes, xylophones and metallophones (soprano, alto and bass). It is a large space where there are only chairs and musical instruments and stands. The first group was distributed as follows: 6 chimes (3 soprano and 3 alto), 5 soprano xylophones, 5 alto xylophones, 5 bass xylophones, 5 soprano metallophones, 5 alto metallophones and 5 bass metallophones. The distribution of the second group was similar, but with one less alto metallophone.

Prior to the experiment, participants received two practical lessons in order to learn how to collectively perform the music score (third experimental condition). After the two practical lessons, during the next three sessions (leaving two weeks between each session), the experiment was carried out. In each session, an experimental condition was applied and PANAS was on-line administered online beforehand and afterwards (Pre-Post design). All participants were exposed to the three experimental conditions and completed the scale before and after listening to music.

In each of these three sessions, a different music condition was applied: MML in the first one, VML in the second one and BP in the third one.

As conditions VML and MML were listening to pieces of music, the instructions received by the subjects were: “You are going to listen to a musical piece, you ought to listen actively, avoiding distractions. You can close your eyes if you feel like to”. For the BP condition, they were said to play the musical sheet all together.

The aim of the study was to examine the effect of the music experience variable (with three levels: MML, VML and BP) in the Positive and Negative Affects subscales from the PANAS scale. The variable Moment was also studied to control biases and to analyze differences between the Pre and Post conditions.

The experiment was designed as a two-way repeated measure (RM) ANOVA with two dependent variables: Positive Affects and Negative Affects, one for each PANAS’ subscales.

The two repeated measures used in the experiment were the variables Musical Experience (ME), with three levels (MML, VML and BP) and the variable Moment, with two levels (PRE and POST). All participants were exposed to the three experimental conditions.

The design did not include a control group, similar to many other studies in the field of music psychology [ 27 , 30 ]. The control was carried out from the intra-subject pre-post measurement of all the participants. The rationale for this design lies in the complexity of the control condition (or placebo) design in psychology [ 46 ]. While placebos in pharmacological trials are sugar pills, in psychology it is difficult to establish an equivalent period of time similar to the musical pieces (e. g. 9 min) without activity, so that cognitive activity occurred during this period of time (e. g. daydreaming, reading a story, etc.) could bias and limit the generalization of results.

Additionally, one of the goals of this study was to compare the effects of listening to music compared to performance on affects. For this reason, two music listening experiences (MML and VML) and a musical performance experience (BP) were designed. In order to control potential biases, participants did not know the musical pieces in the experimental conditions and they had a low level of musical performance competence (musicians were excluded).

It was used SPSS statistics v.26 for the statistical analyzes.

Two ANOVA were performed. The first one, analyzed two dependent variables at the same time: Positive Affects (PA) and Negative Affects (NA).

In the second ANOVA, the 20 items of the PANAS scale were taken as dependent variables. The rest of the experimental design was similar to the first one, a two-way RM ANOVA with variables Musical Experience (ME) and Moment as repeated measures.

Examination of frequency distributions, histograms, and tests of homogeneity of variance and normality for the criterion measures indicated that the assumptions for the use of parametric statistics were met. Normality was met in all tests except for one, but the ANOVA is robust against this assumption violation. All the analyses presented were performed with the significance level (alpha) set at 0.05, two-tailed tests. Means and standard deviations for the 6 experimental conditions for both subscales, Positive Affects and Negative Affects, are presented in Table 1 .

Mauchly’s test of sphericity was statistically significant for Musical Experience and Musical Experience*Moment focusing on NA as the dependent variable ( p  < 0.05). The test only was significant for Musical Experience for PA as dependent variable ( p  < 0.05). The rest of the W’s Mauchly were not significant ( p  > 0.05), so we assumed sphericity for the non-mentioned variables and worked with the assumed sphericity univariate solution. For the variables which the W’s Mauchly was significant, the univariate solution was also taken, but choosing the corrected Greenhouse–Geisser epsilon approximation due to its conservativeness.

A significant principal effect of the Musical Experience variable F(1.710,119.691) = 22.505, p  < 0.05, η 2  = 0.243; the Moment variable F(1,70) = 45.291, p  < 0.05, η 2  = 0.393; and the Musical Experience*Moment interaction F(2,140) = 32.502, p  < 0.05, η 2  = 0.317 were found for PA.

Statistically significance was found for Moment F(1, 70) = 70.729, p  < 0.05, η 2  = 0.503 and Musical Experience*Moment interaction F(1.822, 127.555) = 8.594, p  < 0.05, η 2  = 0.109, but not for Musical Experience F(1.593, 111.540) = 2.713, p  < 0.05, η 2  = 0.037, for the other dependent variable, NA.

Table 2 shows pairwise comparisons between Musical Experience levels. Bonferroni’s correction was applied in order to control type I error. We only interpret the results for the Positive Affects because the Musical Experience effect was not statistically significant for Negative Affects. Results show that condition VML presents a significant higher punctuation in Positive Affects than the other two conditions ( p  < 0.05). It also shows that the musical condition MML is significantly above BP in Positive Affects ( p  < 0.05).

As regards Moment variable (Table 3 ), all but one Pre-Post differences were statistically significant ( p  < 0.05) for all the three conditions for both Positive and Negative Affects dependent variables. The Pre-Post difference found in Positive Affects for the VML Musical Experience did not reach the statistical level ( p  = 0.319).

Focusing on these statistically significant differences, we observe that conditions MML and BP, for PA, decreased from Pre to Post condition, indicating that positive emotions increased significantly between pre and post measures. On the other hand, for NA, all conditions increased from Pre to Post conditions, indicating that negative affects were decreased between pre and post conditions. Once again, one should bear in mind that items were reversed, thus, a higher scores in NA means a decrease in affects.

In order to measure the interaction effect, significant differences between simple effects were analysed.

The simple effect of Moment (level2-level1) in the first Music Experience condition (MML) in PA was compared with the simple effect of Moment (level2-level1) in the second Musical Experience condition (VML). Music Experience conditions 2–3 (VML-BP) and 1–3 (MML-BP) were compared in the same way. Thus, taking into account PA and NA variables, a total of 6 comparisons, 3 per dependent variable, were made.

The results of these comparisons are shown in Table 4 . Comparisons for PA range from T1 to T3 and comparisons for NA range from T4 to T6. All of them are significant ( p  < 0.05) which means that there are statistically significant differences between all the Musical Experience conditions when comparing the Moment (pre/post) simple effects.

In Table 5 , we can look at the differences’ values. As we said before the differences between Pre and Post conditions are significant when comparing the three musical conditions. The biggest difference for positive affects is between MML and BP (T3 = 8.443), and between VML and MML (T4 = − 6.887) for negative affects.

In this second part, the results obtained from the second two-way RM ANOVA with the 20 items as dependent variables are considered. Results of the descriptive analysis of each item: Interested, Excited, Strong, Enthusiastic, Proud, Alert, Inspired, Determined, Attentive, Active, Distressed, Upset, Guilty, Afraid, Hostile, Irritable, Ashamed, Nervous, Jittery, Scared ; in each musical condition: MML, VML and BP; and for the PRE and POST measurements, can be found in the Additional file 1 (Appendix A).

As regards the ANOVA test that compares the three experimental conditions in each mood, Mauchly’s Sphericity Test indicates that sphericity cannot be assumed for the musical experience in most of the variables of the items of effects, except for Interested, Alert, Inspired, Active and Irritable . For these items, the highest observed power index among Greenhouse–Geisser, Huynh–Feldt and Lower-bound epsilon corrections was taken for each variable. For the interaction Musical Experience*Moment, sphericity was not assumed for Distressed, Guilty, Hostile and Scared . For these items, the same above-cited criterion was followed.

Musical experience has a principal effect on all the positive affects, but only has it for 5 negative affects ( Nervous, Jittery, Scared, Hostile and Upset ) ( p  < 0.05). For more detail see Table S1 from Additional file 1 : Appendix B.

The principal effect of Moment is also statistically significant ( p  < 0.05) for all (positive and negative), but two items: Guilty ( p  = 0.073) and Hostile ( p  = 0.123). All the differences between Pre and Post for positive affects are positive, which means that scores in conditions Pre were significantly higher than in condition Post. The other way around occurs for negative affects, all the differences Pre-Post are negative, meaning that the Post condition is significantly higher than the Pre condition. For more detail, see Table S2 from Additional file 1 : Appendix B. In this way, Pre-post changes (Moment) improve affective states; the positive affects increase while the negative are reduced, except for Guilty ( p  = 0.073) and Hostile ( p  = 0.123).

Comparing the proportion of variance explained by the musical experienced and Moment (Tables s1 and s2 from the Additional file 1 : Appendix B), it is observed that most of the η 2 scores in musical experience are below 0.170, except Active and Alert , which are higher. On the other hand, the η 2 scores for Moment are close to 0.300. From these results we can state that, taking only one of the variables at a time, the proportion of the dependent variable’s variance explained by Moment is higher than the proportion of the dependent variable’s variance explained by Musical Experience.

The effect of interaction, shown in Table S3 from the Additional file 1 : Appendix B is significant in 7 positive moods ( Interested, Excited, Enthusiastic, Alert, Determined, Active and Proud ) and 4 negative moods ( Hostile , Irritable, Nervous , and Jittery ).

The pairwise comparisons of Musical Experience’s levels show a wide variety of patterns. Looking at Positive Affects, there is only one item ( Active ) which present significant differences between the three musical conditions. Items Concentrated and Decided do not present any significant difference between any musical conditions. The rest of the Positive items show at least one significant difference between conditions VML and BP. All differences are positive when comparing VML-MML, VML-BP MML-BP, except for Alert and Proud. So, in general, scores are higher for the first two conditions in relation to the third one, meaning that third musical condition presents the biggest increase for Positive Affects (remember items where reversed). For more detail see Additional file 1 : Appendix C.

As regard pairwise comparisons of Musical Experience’s for negative affects, only the items which had a significant principal effect of the variable Musical Experience are shown here. There is a significant difference between conditions VML and MML in item Nervous ; between VML and BP for Scared ( p  < 0.05). For Jittery ; all three conditions differed significantly from each other ( p  < 0.05). Conditions MML and BP differed significantly for Hostile ( p  < 0.05) and conditions VML and BP almost differed significantly for Upset item, but null hypothesis cannot be rejected as p  = 0.056. For more detail see Additional file 1 : Appendix C. All differences were negative when comparing VML-MML, VML-BP MML-BP, except for Nervous and Jittery . So, in general, scores are lower for the first and second condition in relation to the third one.

Positive effects increased significantly during the post phase of all the music experiences, showing that exposure to any of the three music stimuli improved positive affectivity. There were also significant differences between the three experiences in this phase, according to the following order of improvements in positive affectivity: (1) the rhythm and blues performance (BP), (2) listening to Mahler (MML) and (3) listening to Vangelis (VML). As regards the effects of the musical experience x Moment interaction , all the comparisons were significant, with bigger differences in the interpretation of the blues (BP) than in listening to Mahler (MML) and Vangelis (VML). However, the comparison between both experiences, although significant, was smaller. These results indicate that performing music is significantly effective in increasing positive effects. We will explain these results in greater detail below as regards the specific affective states.

As regards Negative Affects, the comparison of the simple effects showed that these decreased after the musical experiences, although in this first analysis the VML musical experience did not differ from the other two. However, the results of the effects of the interaction between musical experiencie x Moment showed that all the comparisons were significant, with a larger difference between MML and VML than the one between BP and each of the other experiences. Listening to Mahler (MML) was more effective in reducing negative affects, compared to both listening to Vangelis and interpreting the blues (BP). These results agree with previous studies [ 26 , 32 ], in which listening to sad music helped to reduce negative affectivity. In this study, it was the most effective condition, although exposure to all three musical experiences reduced negative affects.

The analysis of the specific affective states shows that most items that belong to Positive Affect scale are the most sensitive ones to the PRE-POST change, the different musical conditions and the interpretation of both effects. However, some items of the Negative Affect scale did not differ in the different music conditions or in the music experience × Moment interaction . For example, there were two items (Guilty and Hostile) that did not obtain significance. These results are consistent with the fact that music has certain limits as regards its impact on people’s affects and does not influence all equally. For example, Guilty has profound psychological implications that cannot be affected by simple exposure to certain musical experiences. This means we should be cautious in inferring that music alone can have therapeutical effects on complex emotional states whose treatment should include empirically validated methods. Also, emotional experiences are widely diverse so that any instrument used to measure them is limited as regards the affective/emotional state under study. These results suggest the importance of reviewing the items that compose the PANAS scale in musical studies to adapt it in order to include affective states more sensitive to musical experiences and eliminate the least relevant items.

The analysis of the results in the specific affective states, allows us to delve deeper into each experimental condition. Thus, regarding the results obtained in the complete scale of PANAS, listening to Mahler (MML), causes desirable changes by raising two positive affects ( Inspired and Attentive ) and reducing 10 negative affects ( Distressed, Upset, Afraid, Hostile, Irritable, Ashamed, Nervous, Jittery, and Scared ). This shows that this music condition had a greater effect on the negative affects than the other ones. These results agree with previous studies [ 26 , 32 ], which found that sad music could effectively reduce negative affects, although other studies came to the opposite conclusion. For instance, Miller and Au [ 31 ] found that sad music did not significantly change negative affects. Some authors [ 47 , 48 ] have argued that adults prefer to listen to sad music to regulate their feelings after a negative psychological experience in order to feel better. Taruffi and Koelsch [ 49 ] concluded that sad music could induce listeners to a wide range of positive effects, after a study with 772 participants. In order to contribute to this debate. It would be interesting to control personality variables that might explain these differences on the specific emotions evoked by sad music. In this study, it has been shown that a sad piece of music can be more effective in reducing negative affects than in increasing positive ones. Although the results come from undergraduate students, similar outcomes could be obtained from children and adolescents, although further research is required. In fact, Borella et al. [ 50 ] studied the influence of age on the effects of music and found that the emotional effects influenced cognitive performance (working memory) in such a way that the type of music (Mozart vs. Albinoni) had a stronger influence on young people than on adults. Kawakami and Hatahira [ 28 ], in a study on 84 primary schoolchildren, also found that exposure to sad music pleased them and their level of empathy correlated with their taste for sad music.

Listening to Vangelis (VML) increased 3 positive affects ( Excited, Inspired and Attentive ) and reduced 8 negative affects ( Distressed, Upset, Afraid, Irritable, Ashamed, Nervous, Jittery , and Scared ). Surprisingly, two positive affects were reduced in this experimental condition ( Alert and Attentive ). It could be explained due to the characteristic ostinato rhythm of this piece of music. It was found a similar effect in the study by Campbell et al., [ 26 ] in which sad music reduced both positive and negative affects. This musical condition also managed to modify negative affects more than positive ones.

Performing the blues (BP) increased all 10 positive affects, indicating that performing is more effective in increasing positive affects than listening. These results agree with the study by Dunbar et al. [ 29 ], who found that music performance significantly increased positive affects.

Performing the blues (BP) reduced 6 negative affects, although it was more effective in increasing positive affective states. Vigorous rhythmic music was also found to be positively associated with the use of all the forms of regulating emotions, which suggests that this type of music is especially useful for emotion modulation [ 51 ]. It was found an exception, since Jittery increased after the blues performance. It could be explained by the negative experience that is sometimes associated with music performance. Therefore, it should be taken into account that music performance could increase some negative effects. For example, Dimsdale et al. [ 52 ] found that a strong negative emotional response to a certain type of music in adolescents was related to risk behaviour, indicating that research into the repertory of music experiences needs to be broadened to diverse styles in different age groups to identify all the types of emotional response and their psychological consequences. However, this result should be taken with caution and further research should focus on whether the effect of increased agitation is usual after music performances.

To sum up, this study contributes to the scientific field on the following points: (1) all the musical experiences had significant effects on improving emotional states, increasing positive affects and decreasing the negative ones, which shows the importance of musical experiences on improving the affective sphere; (2) the specific affects that increased, decreased or did not change for each musical experience were identified, providing specific and useful keys for the design of future interventions; and (3) the differences between various types of musical experiences were analyzed, finding more improvements in the performing conditions than in the listening ones.

Limitations and future directions

Limitations.

The sample, made up of university students with a very homogeneous profile in terms of age and sociodemographic characteristics, could limit the generalization of the results. In addition, the low percentage of men in the sample could also affect the generalizability of the results, although no previous studies have reported gender-based differential effects on the positive and negative affects after musical experiences.

Besides, the choice of the pieces of music was based on theoretical criteria and students’ music preferences were not taken into account. This will be included in future research, since the specific choice of the pieces could affect the positive or negative valence of participants’ emotions. However, the goal of using pieces of music not chosen by participants was to elicit new musical experiences for them. Furthermore, no participant was a musician and none of them had previous knowledge of any of the pieces, which may lead to a bias in the results.

In relation to this, the huge amount of available pieces of music, all of them influenced by their cultural and historical context, make it difficult to generalize that certain music parameters correlate with specific emotions. It would be necessary a cross-cultural approach to reach that conclusion.

Future directions

It is recommended to introduce the variables of music preferences and music history to control their effect on the results and to be able to compare the different musical parameters of the pieces together with participants’ preferences.

Likewise, it would be interesting to identify the affects with a greater or lesser degree of influence by music, to adjust the psychological evaluation instrument to the characteristics of the experiment, including items of emotions that can be modified after exposure to a music experience.

The PANAS manual [ 39 ] indicates that a wide variety of affective states (60) and eight different temporal instructions were included in its construction, showing its great versatility. In further research, this instrument should be adapted to for a more specific application to music studies. For instance, by including other emotional states that could be related with the influence of music (e.g. Tranquility , Gratitude , Elevation ), in order to measure more exactly the effects of music on people’s affective experiences.

Accordingly, it would be interesting to evaluate participants' affective traits to establish a baseline and control personality variables, helping to delve into the different levels of the hierarchical structure of affectivity and its relationship with the various music parameters.

Finally, it is recommended that the psychology of music include objective psychophysiological measurements together with self-report evaluations, so that conclusions arising from the experiments have greater robustness and can increase the impact of the contribution to the scientific community.

This study have shown how different music experiences, such as listening and performing, influence the changes in positive and negative affects in student teachers. The results show that the three musical experiences studied are effective in improving the affects by comparing the emotional states before and after the music experiences. It was also showed that there are differences between the effects obtained in each of the music experiences. Besides, improving both types of affects will depend largely on the selected music for the purpose. Although further evidence is required, the results support the importance of music in education, since it provides tools to increase positive affects and to decrease the negative ones, which is important for emotional intelligence development [ 53 , 54 ].

The three music experiences studied are more effective in reducing negative emotional states than in increasing the positive ones. This finding provides useful clues for music teachers to provide strategies that favor emotional regulation. For instance, in order to reduce hostility, irritability and nervousness, students could be exposed to musical auditions of both sad and solemn pieces, choosing musical pieces with similar characteristics to those described in this study. These auditions will be a resource for stress management in the classroom, as well as a tool that students can adopt and generalize to other contexts. Moreover, it is highly likely that students have not heard this type of music before and this experience could increase their repertoire of musical preferences, enhancing their emotional regulation.

The blues performance had a greater impact on participants' positive affects than listening to the other two pieces so, if any teacher wants to increase them (e.g., enthusiasm, interest, etc.), students could be asked to perform simple pieces such as Rhythm's Blues. In this way, musical performance could increase students' resources, contributing to higher levels of motivation, concentration and interest, which promotes learning [ 55 , 56 , 57 , 58 ]. Likewise, it could be very useful for elementary and secondary music teachers, who will be able to contribute to socio-emotional improvement and personal development of their students. Particularly, musical experiences could be a valuable resource for secondary teachers, since music is important in adolescents' lives and can be an interesting tool for meeting their emotional needs [ 59 ]. This is supported by Kokotsaki and Hallam [ 60 ], who consider that performing music helps students feel like active agents of a group, develop a strong sense of belonging, gain popularity, make "like-minded" relationships, improve their social skills and foster a strong sense of self-esteem and satisfaction.

This study shows that experiencing with various unknown musical pieces can have positive effects on emotions. According to this finding, university professors of Teaching grade in music education should encourage future teachers to experience various musical styles, rhythms and tonalities, avoiding prejudices. Thereby, future music teachers will be able to use a diversity of musical experiences that broaden the emotional effects and fulfill the socio-emotional function of music education. In relation to Fredrickson's 'broaden‐and‐build' framework of positive emotions [ 30 ], music can become a mean of widening other positive emotional states, constructing personal resources and transforming people, and contribute to an upward spiral of positive emotions. Taking into account the underlying psychological mechanisms of the impact of music on the emotional states it will be possible to use it to improve emotional area and other aspects of the personal sphere, as Chang et al., [ 10 ] maintain. Therefore, music education is an important resource to improve the emotional development of students.

Availability of data and materials

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Blasco-Magraner JS, Bernabe-Valero G, Marín-Liébana P, Moret-Tatay C. Effects of the educational use of music on 3-to 12-year-old children’s emotional development: a systematic review. Int J Environ Res Public Health. 2021;18(7):1–29. https://doi.org/10.3390/ijerph18073668 .

Article   Google Scholar  

Fancourt D, Ockelford A, Belai A. Brain, behavior, and immunity the psychoneuroimmunological effects of music: a systematic review and a new model. BRAIN Behav Immun. 2013;36:15–26. https://doi.org/10.1016/j.bbi.2013.10.014 .

Article   PubMed   Google Scholar  

Moore KS. A systematic review on the neural effects of music on emotion regulation: implications for music therapy practice. J Music Therapy. 2013;50(3):198–242. https://doi.org/10.1093/jmt/50.3.198 .

Bonde LO, Beck BD. Imagining nature during music listening. An exploration of the meaning, sharing and therapeutic potential of nature imagery in guided imagery and music. In: Pfeifer E, editor. Natur in Psychotherapie und Künstlerischer Therapie: Theoretische, methodische und praktische Grundlagen (2 Bände). Psychosozial-Verlag; 2019. p. 147–68.

Google Scholar  

Cespedes-Guevara J, Eerola T. Music communicates affects, not basic emotions: a constructionist account of attribution of emotional meanings to music. Front Psychol. 2018;9(1):1–19. https://doi.org/10.3389/fpsyg.2018.00215 .

Cotter KN, Silvia PJ, Fayn K. What does feeling like crying when listening to music feel like? Psychol Aesth Creat Arts. 2018;2018(12):216–27. https://doi.org/10.1037/aca0000108 .

Sakka LS, Juslin PN. Emotional reactions to music in depressed individuals. Psychol Music. 2017;46(6):1–19. https://doi.org/10.1177/0305735617730425 .

Ter Bogt T, Canale N, Lenzi M, Vieno A, van den Eijnden R. Sad music depresses sad adolescents: a listener’s profile. Psychol Music. 2021;49(2):257–72. https://doi.org/10.1177/0305735619849622 .

Akkermans J, Schapiro R, Müllensiefen D, Jakubowski K, Shanahan D, Baker D, et al. Decoding emotions in expressive music performances: a multi-lab replication and extension study. Cogn Emot. 2019;33(6):1099–118. https://doi.org/10.1080/02699931.2018.1541312 .

Chang J, Lin P, Hoffman E. Music major, affects, and positive music listening experience. Psychol Music. 2021;49(4):841–54. https://doi.org/10.1177/0305735619901151 .

Kreuth G. Music students’ health problems and health-promoting behaviours. Med Probl Perform Art. 2008;23(1):3–11.

MacDonald R, Kreutz G, Mitchell L. Music, health, and wellbeing. Oxford: Oxford University Press; 2013.

Delors J. La Educación encierra un tesoro. Informe a la UNESCO de la Comisión Internacional sobre la Educación para el siglo XXI. Santillana. 1996.

Requena SO. Música y adolescencia: usos, funciones y consideraciones educativas. UT: Revista de Ciències de l’Educació. 2015;2:28–45.

Saarikallio S. Music as emotional self-regulation throughout adulthood. Psychol Music. 2011;39(3):307–27. https://doi.org/10.1177/0305735610374894 .

Hays T, Minichiello V. The meaning of music in the lives of older people: a qualitative study. Psychol Music. 2005;33(4):437–51. https://doi.org/10.1177/0305735605056160 .

Sandín B, Chorot P, Lostao L, Joiner TE, Santed MA, Valiente RM. Escalas PANAS de afecto positivo y negativo: validacion factorial y convergencia transcultural. Psicothema. 1999;11:37–51.

Arjmand HA, Hohagen J, Paton B, Rickard NS. Emotional responses to music: shifts in frontal brain asymmetry mark periods of musical change. Front Psychol. 2017;8(1):1–13. https://doi.org/10.3389/fpsyg.2017.02044 .

Miu AC, Baltes FR. Empathy manipulation impacts music-induced emotions: a psychophysiological study on opera. PLoS ONE. 2012;7(1):1–6. https://doi.org/10.1371/journal.pone.0030618 .

Blasco JS, Calatrava C. Influencia de la música en las emociones percibidas en el alumnado de educación secundaria y bachillerato. Espiral Cuad del Profr. 2020;13(27):180–91. https://doi.org/10.25115/ecp.v13i27.2909 .

Sharman L, Dingle GA. Extreme metal music and anger. Front Hum Neurosci. 2015;9(1):1–11. https://doi.org/10.3389/fnhum.2015.00272 .

Schubert E. A special class of experience: positive affect evoked by music and the arts. Int J Environ Res Public Health. 2022;19(8):4735. https://doi.org/10.3390/ijerph19084735 .

Article   PubMed   PubMed Central   Google Scholar  

Tasso F. Influencia de la empatía y la instrucción musical en el reconocimiento de emociones y estimación temporal en la música [Internet]. 2019. Available from: https://repositorio.uca.edu.ar/bitstream/123456789/10189/1/influencia-empatia-instruccion-musical.pdf .

Brattico E, Bogert B, Alluri V, Tervaniemi M, Eerola T, Jacobsen T. It’s sad but i like it: THE neural dissociation between musical emotions and liking in experts and laypersons. Front Hum Neurosci. 2016;9(JAN2016):1–21. https://doi.org/10.3389/fnhum.2015.00676 .

Kawakami A, Furukawa K, Katahira K, Okanoya K. Sad music induces pleasant emotion. Front Psychol. 2013;4(June):1–15. https://doi.org/10.3389/fpsyg.2013.00311 .

Campbell EA, Berezina E, Gill CMHD. The effects of music induction on mood and affect in an Asian context. Psychol Music. 2021;49(5):1132–44. https://doi.org/10.1177/0305735620928 .

Vuoskoski JK, Eerola T. The pleasure evoked by sad music is mediated by feelings of being moved. Front Psychol. 2017;8(March):1–11. https://doi.org/10.3389/fpsyg.2017.00439 .

Kawakami A, Katahira K. Influence of trait empathy on the emotion evoked by sad music and on the preference for it. Front Psychol. 2015;6(OCT):1–9. https://doi.org/10.3389/fpsyg.2015.01541 .

Dunbar RIM, Kaskatis K, MacDonald I, Barra V. Performance of music elevates pain threshold and positive affect: implications for the evolutionary function of music. Evol Psychol. 2012;10(4):688–702. https://doi.org/10.1177/147470491201000 .

Fredrickson BL, Tugade MM, Waugh CE, Larkin GR. What good are positive emotions in crises? A prospective study of resilience and emotions following the terrorist attacks on the United States on September 11th, 2001. J Personal Soc Psychol. 2003;84(2):365–76.

Miller S, Au A. The comparison of happy and sad music on mood and task-switching. In: Proceedings of the 37th Australasian Experimental Psychology Conference [Internet]. 2010. p. 8–10. Available from: https://www.psychology.org.au/Assets/Files/2010-Combined-Abstracts.pdf .

Schulte B, ScholarWorks at WMU Music Evoked Nostalgia and Mood States Music Evoked Nostalgia and Mood States. West Mich Uni 2018; https://scholarworks.wmich.edu/honors_theses/3085 .

Matsumoto J. Why people listen to sad music: effects of music on sad moods. Japanese J Educ Psychol. 2002;50(1):23–32. https://doi.org/10.5926/jjep1953.50.1_23 .

Vuoskoski JK, Eerola T. Can sad music really make you sad? Indirect measures of affective states induced by music and autobiographical memories. Front Aesth Arts. 2012;6(3):204–13. https://doi.org/10.1037/a0026937 .

Talamini F, Eller G, Vigl J, Zentner M. Musical emotions affect memory for emotional pictures. Sci Rep. 2022;12(1):1–8. https://doi.org/10.1038/s41598-022-15032-w .

Mayer JD, Salovey P, Caruso D. Models of emotional intelligence. In: Sternberg RJ, editor. The handbook of intelligence. New York: Cambridge University Press; 2000. p. 396–420.

Chapter   Google Scholar  

Fernández-Berrocal P, Extremera N, Ruiz-Aranda D, Cabello R. Inteligencia emocional, estilos de respuesta y depresión. Ansiedad y Estrés. 2006;12(3):191–205.

Eberth J, Sedlmeier P. The effects of mindfulness meditation: a meta-analysis. Mindfulness. 2012;3(3):174–89. https://doi.org/10.1007/s12671-012-0101-x .

Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Personal Soc Psychol. 1988;54(6):1063–70. https://doi.org/10.1037/0022-3514.54.6.1063 .

Schimmack U, Grob A. Dimensional models of core affect: A quantitative comparison by means of structural equation modeling. Eur J Pers. 2000;14(4):325–45.

Gerardi GM, Gerken L. The development of affective responses to modality and melodic contour. Music Percept. 1995;12(3):279–90.

Tizón Díaz MA. La influencia del estilo musical en la emoción percibida. 2015; Available from: https://burjcdigital.urjc.es/handle/10115/13620#.YtizDsdrafw.mendeley .

Thompson WF, Robitaille B. Can composers express emotions through music? Empir Stud Arts. 1992;10(1):79–89. https://doi.org/10.2190/NBNY-AKDK-GW58-M .

Mahler, G. Symphony nº 5. [CDROM recorded by Berliner Philharmoniker/Claudio Abbado]. Berlin: Deutsche Grammophon, 1993.

Vangelis, E. Alexander. [CDROM recorded by Vangelis]. Sony Classical, 2004.

Locher C, Gaab J, Blease C. When a placebo is not a placebo: Problems and solutions to the gold standard in psychotherapy research. Front Psychol. 2018;9(NOV):1–4. https://doi.org/10.3389/fpsyg.2018.02317 .

Saarikallio S, Erkkilä J. The role of music in adolescents’ mood regulation. Psychol Music. 2007;35(1):88–109. https://doi.org/10.1177/0305735607068889 .

Van Den Tol AJM, Edwards J. Exploring a rationale for choosing to listen to sad music when feeling sad. Psychol Music. 2013;41(4):440–65. https://doi.org/10.1177/0305735611430433 .

Taruffi L, Koelsch S. The paradox of music-evoked sadness: an online survey. PLoS ONE. 2014;9(10):1–17. https://doi.org/10.1371/journal.pone.0110490 .

Borella E, Carretti B, Grassi M, Nucci M, Sciore R. Are age-related differences between young and older adults in an affective working memory test sensitive to the music effects? Front Aging Neurosci. 2014;6(OCT):1–9. https://doi.org/10.3389/fnagi.2014.00298 .

Cook T, Roy ARK, Welker KM. Music as an emotion regulation strategy: an examination of genres of music and their roles in emotion regulation. Psychol Music. 2019;47(1):144–54. https://doi.org/10.1177/03057356177346 .

Roberts KR, Dimsdale J, East P, Ph D, Friedman L. Adolescent emotional response to music and its relationship to risk-taking behaviors. J Adolesc Health. 1998;1:49–54. https://doi.org/10.1016/S1054-139X(97)00267-X .

Park N, Peterson C, Seligman M. Strengths of character and well-being. J Social Clin Psychol. 2004;23(5):603–19. https://doi.org/10.1521/jscp.23.5.603.50748 .

Zysberg L, Raz S. Personality and Individual Differences Emotional intelligence and emotion regulation in self-induced emotional states: physiological evidence. Personal Individ Differ. 2019;139:202–7. https://doi.org/10.1016/j.paid.2018.11.027 .

Brown ED, Sax KL. Arts enrichment and preschool emotions for low-income children at risk. Early Child Res Q. 2013;28:337–46. https://doi.org/10.1016/j.ecresq.2012.08.002 .

Ramdane T, Souad M, Marusin R, Sidek, SS. The usefullness of music as a tool of teaching islamic education: Teachers’ perspective. Al-Shajarah J. Isl. 2018, 267–286.

Pimenta MA, Trevisan VL. Música e psicologia na escola: Mobilizando afetos na classe de recuperação. Psicol Esc Educ. 2018;22:17–25. https://doi.org/10.1590/2175-35392018019065 .

Rauduvaite A. The educational aspects of integrating popular music into lessons. Rural Environ Educ Personal. 2018;11:94–100. https://doi.org/10.22616/reep.2018.01153 .

North AC, Hargreaves DJ, Neill SAO. The importance of music to adolescents. British J Educ Psychol. 2000;70(2):255–72. https://doi.org/10.1348/000709900158083d .

Kokotsaki D, Hallam S. Higher education music students’ perceptions of the benefits of participative music making. Music Educ Res. 2007;9(1):93–109. https://doi.org/10.1080/14613800601127577 .

Download references

Acknowledgements

We should like to express our gratitude to the Valencia University student teachers for their disinterested and valuable contribution to this study.

Not applicable.

Author information

Authors and affiliations.

Department of Music Education, University of Valencia, Av. Dels Tarongers, 4, 46022, Valencia, Spain

José Salvador Blasco-Magraner, Pablo Marín-Liébana & Ana María Botella-Nicolás

Department of Occupational Sciences, Speech Therapy, Evolutionary and Educational Psychology, Catholic University of Valencia San Vicente Mártir, Av. De La Ilustración, 2, 46100, Burjassot, Valencia, Spain

Gloria Bernabé-Valero

You can also search for this author in PubMed   Google Scholar

Contributions

JSBM and GBV contributed to the study conception and design. Material preparation, data collection and analysis were performed by JSBM and GBV. The first draft of the manuscript was written by JSBM, GBV and PML. PML and ABN review, translate and editing the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to José Salvador Blasco-Magraner .

Ethics declarations

Ethics and consent to participate.

The study protocol was according to the declaration of Helsinki. The research was approved by the ethical committee at the Catholic University of Valencia San Vicente Mártir: UCV2017-18-28 code. Informed written consents were obtained from all participants in the present study.

Consent for publication

Competing interests.

The authors indicate that they have no conflict of interests that impacted this study.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

. Results obtained from item analyses.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Blasco-Magraner, J.S., Bernabé-Valero, G., Marín-Liébana, P. et al. Changing positive and negative affects through music experiences: a study with university students. BMC Psychol 11 , 76 (2023). https://doi.org/10.1186/s40359-023-01110-9

Download citation

Received : 12 November 2022

Accepted : 06 March 2023

Published : 21 March 2023

DOI : https://doi.org/10.1186/s40359-023-01110-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Positive affects
• Negative affects
• Music experiences
• University students

BMC Psychology

ISSN: 2050-7283

• General enquiries: [email protected]

Self-Efficacy Development of Female Secondary Students in an Assistive Co-robotics Project

Women are underrepresented in science, technology engineering and math (STEM) careers. This is particularly detrimental within the space of engineering and technology where the women can provide unique perspectives about design. People are more likely to choose careers in which they feel confident in their abilities. Therefore, this study examined the experiences of girls in high school engineering and technology programs who were in the process of making decisions about their future careers. It explored how their classroom experiences were related to the development of their self-efficacy in engineering. This study addressed the research question: How and in what ways do the classroom experiences of female secondary students during a co- robotics assistive technology project relate to their changes in engineering self-efficacy? This question was addressed through qualitative case study research. Data were collected through observation, focus group interviews, and review of design journals kept by the participants. The data were coded, and themes were developed as guided by Bandura’s four sources of self-efficacy. Findings from this study indicated that the high school girls relied in varying amounts on different sources of self-efficacy based on their initial self-efficacy, their interactions with their teammates during group work, and connections they made between the content and applications in their lives outside of the classroom. The girls in the study had improved or maintained self-efficacy because they were able to achieve their desired outcomes in the projects. Relatedly frustrations that the girls faced along the way were not detrimental because they ultimately achieved success. Positive experiences with teammates supported the girls’ self-efficacy development, and negative experiences deterred self-efficacy. Finally, when the girls made connections between the content they were learning and applications that held value for them, they were more motivated to engage in experiences that supported the development of their self-efficacy.

National Science Foundation Grant 2133028

Degree type.

• Master of Science
• Technology Leadership and Innovation

Campus location

• West Lafayette

Advisor/Supervisor/Committee Chair

Additional committee member 2, additional committee member 3, usage metrics.

• Secondary education
• Engineering education

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Hosted content
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 79, Issue 5
• Impact of post-COVID-19 condition on health status and activities of daily living: the PRIME post-COVID study
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0002-3333-6964 Maarten Van Herck 1 , 2 , 3 ,
• Demi M E Pagen 4 , 5 ,
• Céline J A van Bilsen 4 , 5 ,
• Stephanie Brinkhues 6 ,
• Kevin Konings 7 ,
• Casper D J den Heijer 4 , 5 ,
• Suhreta Mujakovic 4 ,
• Henriëtte L G ter Waarbeek 4 ,
• Chris Burtin 3 ,
• http://orcid.org/0000-0002-1827-9869 Daisy J A Janssen 1 , 8 ,
• Christian J P A Hoebe 4 , 5 ,
• http://orcid.org/0000-0003-3822-7430 Martijn A Spruit 1 , 2 ,
• Nicole H T M Dukers-Muijrers 4 , 9
• 1 Department of Research and Education , Ciro , Horn , The Netherlands
• 2 Department of Respiratory Medicine, School of Nutrition and Translational Research in Metabolism (NUTRIM), Faculty of Health, Medicine, and Life Sciences , Maastricht University Medical Centre+ , Maastricht , The Netherlands
• 3 REVAL, Rehabilitation Research Center, BIOMED – Biomedical Research Institute, Faculty of Rehabilitation Sciences , Hasselt University , Diepenbeek , Belgium
• 4 Department of Sexual Health, Infectious Diseases, and Environmental Health, Living Lab Public Health , South Limburg Public Health Service , Heerlen , The Netherlands
• 5 Department of Social Medicine, Faculty of Health, Medicine, and Life Sciences, Care and Public Health Research Institute (CAPHRI) , Maastricht University , Maastricht , The Netherlands
• 6 Department of Knowledge & Innovation , South Limburg Public Health Service , Heerlen , The Netherlands
• 7 Department of Process & Information Management, Communication & Automation , South Limburg Public Health Service , Heerlen , The Netherlands
• 8 Department of Health Services Research and Department of Family Medicine, Care and Public Health Research Institute, Faculty of Health, Medicine, and Life Sciences , Maastricht University , Maastricht , The Netherlands
• 9 Department of Health Promotion, Faculty of Health, Medicine, and Life Sciences, Care and Public Health Research Institute (CAPHRI) , Maastricht University , Maastricht , The Netherlands
• Correspondence to Maarten Van Herck, Department of Research and Education, Ciro, Horn, 6080, The Netherlands; maarten.vanherck{at}uhasselt.be

Objective To assess health and activities of daily living (ADL) in SARS-CoV-2-positive adults with and without post-COVID-19 condition (PCC) and compare this with negative tested individuals. Furthermore, different PCC case definitions were compared with SARS-CoV-2-negative individuals.

Methods All adults tested PCR positive for SARS-CoV-2 at the Public Health Service South Limburg (Netherlands) between June 2020 and November 2021 (n=41 780) and matched PCR negative individuals (2:1, on age, sex, year-quarter test, municipality; n=19 875) were invited by email. Health (five-level EuroQol five-dimension (EQ5D) index and EuroQol visual analogue scale (EQVAS)) and ADL impairment were assessed. PCC classification was done using the WHO case definition and five other common definitions.

Results In total, 8409 individuals (6381 SARS-CoV-2 positive; 53±15 years; 57% female; 9 (7–11) months since test) were included. 39.4% of positives had PCC by the WHO case definition (EQVAS: 71±20; EQ5D index: 0.800±0.191; ADL impairment: 30 (10–70)%) and perceived worse health and more ADL impairment than negatives, that is, difference of −8.50 points (95% CI −9.71 to −7.29; p<0.001) for EQVAS, which decreased by 1.49 points (95% CI 0.86 to 2.12; p<0.001) in individuals with PCC for each comorbidity present, and differences of −0.065 points (95% CI −0.074 to −0.056; p<0.001) for EQ5D index, and +16.72% (95% CI 15.01 to 18.43; p<0.001) for ADL impairment. Health and ADL impairment were similar in negatives and positives without PCC. Replacing the WHO case definition with other PCC definitions yielded comparable results.

Conclusions Individuals with PCC have substantially worse health and more ADL impairment than negative controls, irrespective of the case definition. Authorities should inform the public about the associated burden of PCC and enable adequate support.

Data availability statement

Data are available upon reasonable request. Data cannot be shared publicly because the data contains potentially identifying patient information. Data are available on request from the head of the data-archiving South Limburg Public Health Service (contact via [email protected]) for researchers who meet the criteria for access to confidential data.

https://doi.org/10.1136/thorax-2023-220504

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

A substantial proportion of SARS-CoV-2-infected people develop the post-COVID-19 condition (PCC).

PCC impacts health and activities of daily living (ADL).

Published studies were prone to selection bias, lacked controls and used different criteria to define the same condition, hampering comparison between studies and undermining the validity of the evidence.

WHAT THIS STUDY ADDS

PCC has a substantial impact on the health and performance of ADL compared with negative controls, irrespective of the case definition used.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

Irrespective of the case definition, there is an associated burden of PCC, which requires an adequate response by authorities in terms of informing the public and enabling support.

Introduction

A substantial proportion of SARS-CoV-2-infected people report lasting symptoms, 1 2 initially referred to as ‘long COVID’. 3 This was later updated by the WHO to ‘post-COVID-19 condition’ (PCC) which is defined as ‘ a history of probable or confirmed SARS-CoV-2 infection usually 3 months from onset of COVID-19 with symptoms that last for at least 2 months and cannot be explained by an alternative diagnosis’. 4

Several studies have reported on the impact of PCC. 5 6 Patients with PCC were found to have comparable health to that of people with chronic obstructive pulmonary disease and rheumatic arthritis. 7 Furthermore, a negative impact of PCC on quality of life 8 9 and activities of daily living (ADL) was found. 10 11 However, the study populations in these studies were often recruited via online support groups and therefore prone to selection bias. Additionally, studies often lack controls. 12 Consequently, the true burden of PCC on health and ADL remains poorly reported in the general population at the time of writing. Moreover, the lack of clear diagnostic criteria for PCC 4 and the use of different terminologies and criteria to define the same condition hamper comparison between studies and undermine the validity of the evidence. 13 In addition, little is known about other factors (such as sex, age and comorbidities) that could potentially affect the impact of PCC on ADL and health. 14

Therefore, the aims of the current study were (1) to assess experienced health and ADL in SARS-CoV-2-positive adults with and without PCC and compare this with SARS-CoV-2-negative individuals and (2) to compare these outcomes between SARS-CoV-2-negative individuals and various commonly used case definitions for PCC while accounting for potential confounding factors.

Data from the first wave of a longitudinal open cohort study on the prevalence, risk factors, and impact evaluation of post-COVID-19 condition (PRIME post-COVID) was used. The protocol of the study was published elsewhere. 15

Study design

In short, adults tested for SARS-CoV-2 at the Public Health Service (PHS) South Limburg in the Netherlands, with a valid PCR test result (positive or negative) between 1 June 2020 and 1 November 2021 and a valid email address were retrieved from test records in the registry. In November 2021, all PCR-positive individuals (n=41 780) and a group of PCR-negative individuals (who only had negative test result(s); n=19 875) matched (2:1 ratio) by age, sex, year-quarter of test and municipality of residence were invited for participation by email. Of note, when people tested negative multiple times, the last negative PCR test date was used. The online survey lasted 30–45 min and was available from 17 November 2021 to 9 January 2022. Digitally informed consent for the use and storage of data for research was asked prior to the start of the survey. The invitee could participate in the questionnaire after consent was provided on participation in the study and on the use of the data for research. 15

Population in the current analysis

Individuals were excluded from the current study when they were tested less than 3 months prior to participation in the survey, sex was not reported, PCR-negative individuals in the registry self-reported seropositivity for SARS-CoV-2 before vaccination (because of missing relevant infection-related information) or if the health section of the survey was not completed. Furthermore, PCR-negative individuals (ie, SARS-CoV-2-negative individuals in the registry) were attributed to the SARS-CoV-2-positive group if they self-reported a positive test not in the registry (eg, tested in a hospital, outside a geographical service area and rapid antigen testing). This was the case for 87 individuals. Besides that, 166 individuals gave no consent to match registry data with their questionnaire data. For these individuals, we used self-reported data only.

Main outcome variables

Experienced health eq5d index.

Experienced health was assessed using the five-level EuroQol five-dimensions version (EQ5D), which includes five health dimensions (mobility, self-care, usual activities, pain or discomfort and anxiety or depression), each on a five-level scale ((1) no problems, (2) slight problems, (3) moderate problems, (4) severe problems and (5) extreme problems or unable to). 16 An EQ5D index score was calculated by attaching weights to each level in each dimension. The attached weights were obtained by Versteegh and colleagues for the Dutch population via a standardised valuation study protocol by the EuroQol Group. 17 Index scores could range between −0.446 (worst health) and 1.000 points (best health), whereas 0 is the value of a health state equivalent to dead. 17

Experienced health EQVAS

Additionally, the EQ5D includes a vertical visual analogue scale (EQVAS) ranging from 0 (worst health you can imagine) to 100 points (best health you can imagine) to obtain the respondent’s current perceived health. 16

Impairment in ADL

Illness-related impairment in regular activities other than work (in the past 7 days) was assessed using a selected item of the Work Productivity and Activity Impairment questionnaire. Individuals were asked to indicate the degree to which their health affected productivity in regular unpaid activities using a 0 (no effect) to 10 (completely prevented me from doing my daily activities) scale. The degree of ADL impairment is expressed as a percentage, and a higher percentage indicates a higher overall impairment. 18

Case definitions for PCC

SARS-CoV-2-positive individuals were grouped in the ‘PCC’ group (yes or no) based on the WHO case definition (here used as the main PCC definition) and five alternatives, commonly used definitions, including an adapted WHO case definition to take into account advances in scientific knowledge regarding the WHO case definition. All SARS-CoV-2-positive individuals included in the analyses were at least 3 months after their initial SARS-CoV-2 infection. All case definitions are listed below, a more detailed description of the definitions and questions used can be found in online supplemental eMethods 1 .

Supplemental material

WHO case definition for PCC: fulfilling the current WHO case definition referring to a condition that occurs in individuals with a history of probable or confirmed SARS-CoV-2 infection, usually 3 months from the onset of COVID-19, with symptoms that last for at least 2 months and cannot be explained by an alternative diagnosis. 4

In concrete terms, this means experiencing ≥1 symptom; symptom(s) is/are present for ≥1 month; the time since the test is longer than or equal to the presence of symptom(s), and no new diagnoses have been confirmed since the test.

Adapted WHO case definition for PCC: fulfilling all criteria of the current WHO case definition, 4 except for the criterion of an alternative diagnosis that could explain the symptoms, as recent studies showed associations between COVID-19 and new-onset illnesses. 19–21

Symptom present: having ≥1 symptom present. 22

Differentiating symptom present: having ≥1 symptom present that is observed to be significantly different between SARS-CoV-2 positives and negatives. 1 23

Differentiating symptom present with at least a moderate severity: having ≥1 symptom present that is observed to be significantly different between SARS-CoV-2 positives and negatives with a severity score of ≥5/10. 1 23

Not recovered: Indicating not feeling (fully) recovered. 24

Of note, the following 44 prelisted symptoms (in alphabetical order) were considered: amnesia, brain fog, burning sensation in the trachea, chest tightness, cold, concentration difficulties, confusion, cough, coughing up mucus, diarrhoea, dizziness, dreariness or depression, earache, elevated body temperature, eye difficulties, fatigue, fear, fever, hair loss, headache, heat flushes, increased resting heart rate, irritability, joint pain, loss of appetite, loss or change of smell, loss or change of taste, muscle pain or weakness, nausea, nerve pain, pain between shoulder blades, pain or burning sensation in the lungs, palpitations, runny nose, shortness of breath, skin rashes or red spots on toes or feet, sleeping problems, sneezing, sore throat, stomach ache, sudden weight loss, tinnitus, voice difficulties and vomiting. 23

Baseline characteristics

Other factors include sex, age, education level and body weight and height to calculate body mass index (BMI). Furthermore, the date of the test to calculate the time since the test and information about hospital admission during acute infection (yes or no; in PCR-positives only) were surveyed. The number of (pre-existing) comorbidities present (before the test) was determined using a predefined list of comorbidities ( online supplemental eMethods 2 ) and a question on whether this specific comorbidity was present before the SARS-CoV-2 test. Perceived health the year prior to testing was assessed using EQVAS (retrospectively).

Statistical analysis

Descriptive statistics were presented for the three groups (ie, positives with PCC, positives without PCC and SARS-CoV-2 negatives). Categorical data were reported as numbers (frequencies) and ordinal data as medians (IQR). Continuous data were checked for normality using histograms and Q-Q plots and reported as mean (SD) or median (IQR) as appropriate.

Univariable and multivariable regression models were performed for the main continuous outcomes (ie, EQVAS, EQ5D index and ADL impairment) with ordinary least squares linear regression to assess the main determinant (ie, PCC, no PPC and SARS-CoV-2 negatives). The multivariable analyses were adjusted for a minimally sufficient set of confounders identified in the literature: age, sex, pre-existing comorbidities when tested, time since the test and health the year prior to the test. 2 22 23 The three groups were modelled as two dummy variables in this analysis, with the SARS-CoV-2 negatives as the reference group. The WHO case definition was used to classify SARS-CoV-2-positive individuals into the (no) PCC group. If multicollinearity was present (variance inflation factor >5), variables were identified and removed from the model. Interaction terms for sex, age, health the year prior to the test, pre-existing comorbidities when tested and time since the test on the one hand and the two dummy variables (ie, group variables) on the other hand were explored, as a potential effect of the group variables on health and ADL might depend on these confounders included in the model. 2 23 25 When found to be statistically significant, the interaction term was included in the final models.

Furthermore, analysis of covariance adjusting for time since the test and health the year prior to the test were performed for subgroups by sex (male or female), age (18–40, 41–60 and 60+ years) and presence of pre-existing comorbidities at the test (yes or no). This was done for positives with and without PCC according to the WHO case definition and the SARS-CoV-2-negative group and for the various commonly used definitions for PCC.

A priori, the level of significance was set at 0.01 (two-tailed) to account for multiple testing in this study and the high sample size in the multivariable regression analyses. Model assumptions were checked when performing the analyses. Tables and figures include 95% CIs. Statistical analyses were performed using SPSS V.27.0 (IBM, Armonk, NY, USA). Visualisations were made using GraphPad Prism V.9.3.1 (GraphPad Software, La Jolla, CA, USA).

Adults (n=61 655) were invited by email to participate. Of the 18 859 respondents, 12 453 were eligible as they provided minimal data and showed sufficient certainty to be the intended invitee. 23 Individuals were excluded if they were tested less than 3 months prior to participation (n=2656), did not complete the health section of the survey (n=1331), were PCR negative but reported to have SARS-CoV-2 antibodies before vaccination (n=56) or did not report sex (n=1). Consequently, 8409 individuals (6381 SARS-CoV-2 positive and 2028 negative) were included ( figure 1 ). Age and sex data of all invitees and invitees (not) included in the analysis are reported in online supplemental eTable 1 .

• Download figure
• Open in new tab
• Download powerpoint

Flowchart of invitees, respondents, participants eligible for inclusion and study population included in the analysis.

Description of the study population

In total, 39.4% of positive individuals (n=2513) had PCC, according to the WHO case definition. Demographical and clinical data of the SARS-CoV-2 positives with PCC, positives without PCC and negatives are shown in table 1 . In short, the positives with and without PCC had a similar sex distribution (62% and 57% females, respectively), mean age (51±15 years) and median time since infection of 10 (7–11) months. Positives without PCC had, on average, better perceived health the year prior to the test and fewer comorbidities present when tested than positives with PCC and negatives. Furthermore, the negative group was, on average, 7 years older and had a lower proportion of females (49%) than both positive groups.

• View inline

Characteristics of SARS-CoV-2 negatives and SARS-CoV-2 positives stratified for PCC according to the WHO case definition

Differences in health and ADL

SARS-CoV-2 positives with PCC reported a higher median ADL impairment due to health (30% vs 0% and 0%) and a worse average perceived health at this moment (EQVAS: 70.8 vs 81.5 and 79.2; EQ5D index: 0.800 vs 0.896 and 0.878) compared with positives without PCC and negatives, respectively. The greatest impairments in positives with PCC were observed for the domain’s usual activities and pain/discomfort ( table 1 ). A more detailed overview of the extent of impairment across the several health domains can be found in online supplemental eFigure 1 .

The results of the univariable and multivariable regression models can be found in table 2 . The adjusted EQ5D index was significantly lower in positives with PCC compared with negatives (−0.065 points, 95% CI −0.074 to −0.056; p<0.001). No significant difference between positives without PCC and negatives was observed (−0.003 points, 95% CI −0.011 to 0.005; p=0.492).

Unadjusted and adjusted regression models for health and ADL impairment adjusted for age, sex, comorbidities at test, health status year prior to test and time since test, with SARS-CoV-2 negatives as reference group and PCC according to the WHO case definition

Also, the perceived health status measured using the EQVAS was 8.50 points (95% CI −9.71 to −7.29; p<0.001) lower in positives with PCC than in negatives. While the EQVAS was not statistically significantly higher in positives without PCC than in negatives (0.26 points, 95% CI −0.68 to 1.19; p=0.593). Our model indicated that the presence of PCC and comorbidities together has a smaller effect on EQVAS than the sum of each. Specifically, the associated burden of PCC on EQVAS decreases by 1.49 points (95% CI 0.86 to 2.12; p<0.001) for each comorbidity present. The specific models for EQVAS for people without comorbidities, with one comorbidity and with ≥2 comorbidities are reported in online supplemental eTable 2 . The beta coefficients for PCC and no PCC (vs reference: negatives) in the abovementioned models are −9.00 points (95% CI −10.47 to −7.53; p<0.001) and −0.46 points (95% CI −1.72 to 0.80; p=0.476); −6.64 points (95% CI −8.54 to −4.74; p<0.001) and 0.99 points (95% CI −0.80 to 2.77; p=0.278) and −3.73 points (95% CI −5.91 to −1.55; p<0.001) and 1.47 points (95% CI −0.80 to 3.73; p=0.204), respectively.

Furthermore, ADL impairment due to health problems was 16.72% (95% CI 15.01 to 18.43; p<0.001) higher in positives with PCC compared with negatives. The ADL impairment observed in positives without PCC was not significantly higher than that observed in negatives (1.97%, 95% CI 0.40 to 3.54; p=0.014).

A sensitivity analysis of 8156 individuals was performed for the abovementioned multivariable models (253 individuals with a self-reported test result were excluded from the analyses). The results can be found in online supplemental eTable 3 . In short, a similar direction and extent of findings were observed as the analyses in 8409 individuals.

Differences in health and ADL stratified for sex, age and the presence of comorbidities

Results of analyses for EQVAS, EQ5D index and ADL impairment due to health after stratification on sex, age and the presence of pre-existing comorbidities and adjusted for health prior to the test and time since the test can be found in figure 2 . Briefly, significantly worse health and larger ADL impairment were observed in the PCC group (by the WHO case definition) compared with the positive group without PCC and the negative group; this was observed in nearly all strata.

Health and ADL impairment for SARS-CoV-2 negatives and positives with and without post-COVID-19 condition according to the WHO case definition, stratified for sex, age and pre-existing comorbidities and adjusted for time since test and health prior to the test. From left to right: ●: SARS-CoV-2 positives with post-COVID-19 condition, ●: SARS-CoV-2 positives without post-COVID-19 condition; ●: SARS-CoV-2 negatives. Whiskers are 95% CI. # p<0.01 and *p<0.001. ADL, activities of daily living.

Health and ADL in different PCC definitions

Overall, similar (direction of) findings were observed when comparing different case definitions with the SARS-CoV-2-negative control group.

In general, the participants meeting the criteria for the WHO case definition and the definition based on the presence of ≥1 symptom presented the best health and least ADL impairment of all definitions (ie, smallest differences with the SARS-CoV-2-negative group). Contrary, those meeting the criteria for the definition based on the presence of ≥1 differentiating symptom between SARS-CoV-2-positive and SARS-CoV-2-negative individuals with at least a moderate severity and the definition based on the feeling of not being recovered presented the worst health and highest ADL impairment of all definitions.

Results for EQVAS were comparable for all PCC definitions after stratification for age, sex and comorbidities, except for certain case definitions in the stratum of males aged 41–60 years without comorbidities ( figure 3 ). Furthermore, slight differences between certain case definitions were found for the outcomes of the EQ5D index and ADL impairment. These differences (ie, absence of overlap of 95% CIs) were mainly located in the strata with men and women without comorbidities and between the above-mentioned PCC definitions.

Health and ADL impairment for SARS-CoV-2 negatives and different PCC case definitions stratified for sex, age and pre-existing comorbidities. From left to right: ●: WHO case definition, ●: WHO case definition except for criterion of alternative diagnosis, ●: ≥1 symptom, ●: ≥1 symptom that differs between SARS-CoV-2 positives and negatives, ●: ≥1 symptom that differs between SARS-CoV-2 positives and negatives with a severity ≥5/10; ●: feeling not recovered; ●: SARS-CoV-2 negatives. Whiskers are 95% CI, # p<0.01 and *p<0.001 from SARS-CoV-2 negatives. Horizontal brackets indicate the absence of overlap in 95% CI. ADL, activities of daily living; PCC, post-COVID-19 condition.

In this population-based cohort study, the impact of PCC on health and impairment in ADL was studied in adults who tested positive for SARS-CoV-2 and controls. The associated burden of PCC on health and impairment in ADL is meaningful, irrespective of the case definition used, age, sex, presence of pre-existing comorbidities, time since the test or health status prior to the test.

Findings from the current study confirm previous reports that PCC affects health and ADL. 6 8 9 Nevertheless, findings are difficult to compare, as most studies used other measures or reported the findings of all SARS-CoV-2-positive individuals without presenting results for PCC separately. 6 25 26 Studies that specifically reported on PCC found values that were considerably worse in terms of health and ADL. 6 10 11 27 28 Vaes et al , for example, used the same outcome measures and found a mean EQVAS of 56 points and ADL impairment of 60% around 6 months after infection. 11 A conceivable explanation for this is that previous studies mainly reported on individuals recruited via online support groups or clinics, describing a subgroup of the population with a possible bias resulting in an overestimation of the burden. 10 11 28 29 Still, an associated burden of PCC is found when values are compared with Dutch population norms (EQVAS: 70.8 vs 81.4 and EQ5D index: 0.800 vs 0.869, respectively), while positives without PCC and negatives show similar values to the Dutch population norms. 17 30 After controlling for possible confounders, the PCC group had on average 8.50 and 0.065 points lower EQVAS and EQ5D index scores, respectively, and 16.7% higher ADL impairment than the negative group. In the literature, a difference of 7 points for EQVAS (patients with cancer), 0.063 points for the EQ5D index (EQ5D value set for England) and 20% for ADL impairment (patients with psoriasis) is established as meaningful. 31–33

To date, the WHO case definition for PCC is considered the golden standard in the absence of a laboratory test to diagnose PCC. 4 Though the different definitions for PCC found comparable results for health and ADL impairment, indicating that although there is heterogeneity in the case definitions, the same conclusion on the impact of PCC on health and ADL can be drawn. Slight differences observed between definitions can be due to misclassification, as evidence emerges that a SARS-CoV-2 infection and PCC are associated with new-onset illnesses in the case of the WHO case definition, for example, 34 or in the case of a definition using the presence of ≥1 symptom, as (generic) symptoms (eg, fatigue and pain) are also present in chronic diseases and the general population. 1 35 36

The associated burden on perceived health (ie, EQVAS) decreases with the presence of comorbidities, presumably because the room to deteriorate in terms of health is smaller when there are more comorbidities. In addition, no interactions between the presence of comorbidities and PCC were observed for the EQ5D index (and ADL impairment). This has to do with the nature of both health measures, as EQVAS assesses the perceived health of an individual, while the EQ5D index is a score based on limitations in five domains. Until today, it remains unknown whether PCC is a unique condition, an illness similar to myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) and other postinfectious illnesses or even a combination of conditions. To determine this, a comparison with ME/CFS and other postinfectious illnesses is needed.

The major strengths of this study are the large sample size and population-based study design, as the recruitment of individuals was done using the Dutch COVID-19 PCR test registry. By inviting all PCR-positive individuals and matched negative controls, we were able to compare results with a reference group and consider (non-)pandemic-related factors. Furthermore, various definitions for PCC were used and compared in the current study, as well as standardised and validated questionnaires.

This study has a number of limitations. Only individuals with a valid email address were invited for this online study. This may have resulted in a potential selection bias among digitally illiterate individuals, as they were not invited or did not complete the study. Likewise, this may have been the case for people with disabilities or severe diseases. In total, 14% of invitees are included in the current study, although this response rate would be higher if individuals were not excluded for several reasons. 15 Similar rates were reported in other population-based studies regarding PCC with the same recruitment strategy (eg, Whitaker et al , 26%–29%, Hastie et al , 16%) 2 25 and what is expected in email surveys in general. 37 Furthermore, an underrepresentation of invitees aged 18–40 years and an over-representation of invitees aged 50–80 years are observed in the current sample and were more pronounced in the negative invitees. In the current study, less than 2% of the population was hospitalised for COVID-19. It is possible that people went directly to the hospital instead of being tested at the PHS and are therefore underrepresented in our sample. Moreover, misclassification bias regarding the (confirmed) SARS-CoV-2 diagnosis is inevitable, as in the early phase of the pandemic, the availability and access to testing were limited. 38 As a consequence, individuals who had SARS-CoV-2 in the early phase of the pandemic (before 1 June 2020) or were asymptomatic could be included in the negative reference group. Nevertheless, the potential misclassification of SARS-CoV-2 positives in the negative group would not change the direction of the findings. Another potential limitation is that data gathering was done by self-report, leaving potentially relevant information missing (eg, comorbidities). To fulfil the WHO case definition, alternative diagnoses that could explain symptoms need to be excluded. Our study did not include clinician-reported information or information on alternative diagnoses based on medical records. The self-reported data regarding (pre-existing) comorbidities in our study may have a potential recall and misclassification bias. Furthermore, certain questions were prone to recall bias (eg, health the year prior to the test). The study includes test results until the last quarter of 2021 (ie, before the omicron wave). Therefore, results cannot be generalised to vaccinated individuals who developed PCC by a breakthrough infection, as the majority were unvaccinated when they were infected with SARS-CoV-2 or with PCC by the omicron variant. Hence, recent research suggests there is no difference in PCC sequela between SARS-CoV-2 variants. 39 40 Of note, this study was limited to adults; however, PCC is also present in children and adolescents. 34

Individuals with PCC have substantial and clinically meaningful worse health and more impairment in ADL than negative controls, irrespective of sex, age, pre-existing comorbidities, time since the test and health status prior to the test and regardless of the case definition used to define PCC.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

The PRIME post-COVID study involves human participants. The medical ethics committee of Maastricht University Medical Centre+ waived this study (Maastricht, the Netherlands; METC2021-2884), as the Medical Research Involving Human Subjects Act did not apply to this study. This study was registered at ClinicalTrials.gov Protocol Registration and Results System ( NCT05128695 ).Participants gave informed consent to participate in the study before taking part.

Acknowledgments

We gratefully acknowledge LCJ Steijvers, CPB Moonen, AW Vaes and N Bouwmeester-Vincken for their valuable contributions to the development of the survey, and CPB Moonen for her participation in the data collection.

• Ballering AV ,
• van Zon SKR ,
• Olde Hartman TC , et al
• Whitaker M ,
• Elliott J ,
• Chadeau-Hyam M , et al
• Callard F ,
• Soriano JB ,
• Marshall JC , et al
• Lopez-Leon S ,
• Wegman-Ostrosky T ,
• Perelman C , et al
• Pinto C , et al
• Franklin K , et al
• Delbressine JM ,
• Goërtz YMJ , et al
• Tarazona V ,
• Kirouchena D ,
• Clerc P , et al
• McCorkell L , et al
• Goërtz YMJ ,
• Van Herck M , et al
• Amin-Chowdhury Z ,
• Munblit D ,
• O’Hara ME ,
• Akrami A , et al
• Poudel AN ,
• Cooper N , et al
• Pagen DME ,
• van Bilsen CJA ,
• Brinkhues S , et al
• Herdman M ,
• Lloyd A , et al
• Versteegh MM ,
• Vermeulen KM ,
• Evers S , et al
• Reilly MC ,
• Zbrozek AS ,
• Bowe B , et al
• Yen-Ting Chen T ,
• Wang S-I , et al
• Van Herck M ,
• Delbressine JM , et al
• Hastie CE ,
• McAuley A , et al
• Bernabei R ,
• Landi F , et al
• Tabacof L ,
• Tosto-Mancuso J ,
• Wood J , et al
• Chaplin E , et al
• van den Borst B ,
• Peters JB ,
• Brink M , et al
• Janssen B ,
• Pickard AS ,
• McClure NS ,
• Xie F , et al
• Sun L , et al
• McCorkell L ,
• Vogel JM , et al
• Braamse AMJ ,
• Spruit MA , et al
• Petrie KJ ,
• Crichton F , et al
• Muraleedharan A
• den Hartog G ,
• Schepp RM , et al
• Mizrahi B ,
• Flaks-Manov N , et al
• Molteni E ,
• Deng J , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1

X @vanherck_m

Presented at European Respiratory Society’s International Congress 2023

Contributors MVH, DMEP, CJAvB, SB, KK, CDJdH, SM, HLGtW, CJPAH, MAS and NHTMDM designed the study. DMEP, CJAvB and SB actively participated in data collection. MVH performed the data analysis, wrote the first draft of the manuscript, and is the guarantor of this work. DMEP, CJAvB, SB, KK, CDJdH, SM, CB, DJAJ, MAS and NHTMDM supervised data analysis. All authors were involved in data interpretation, revised the manuscript critically for important intellectual content, approved the final version and agreed to be accountable for all aspects of the manuscript. MAS and NHTMDM are joint last authors.

Funding This work was supported by the research fund of the Dutch National Institute for Public Health and environment (RIVM) for local Public Health Services (Grant numbers: 3910090442/3910105642/3910121041).

Competing interests None declared.

Patient and public involvement statement Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

Outdoor air pollution as a risk factor for testing positive for SARS-CoV-2: A nationwide test-negative case-control study in the Netherlands

Affiliations.

• 1 Institute for Risk Assessment Sciences (IRAS), Faculty of Veterinary Medicine, Utrecht University, Utrecht, the Netherlands.
• 2 National Institute for Public Health and the Environment (RIVM), Centre for Infectious Disease Control (CIb), Bilthoven, the Netherlands.
• 3 Institute for Marine and Atmospheric Research Utrecht, Utrecht University, Utrecht, the Netherlands; National Institute for Public Health and the Environment (RIVM), Center for Environmental Quality (MIL), Bilthoven, the Netherlands.
• 4 National Institute for Public Health and the Environment (RIVM), Center for Sustainability, Environment and Health (DMG), Bilthoven, the Netherlands.
• 5 Municipal Health Services, Provinces of Overijssel and Gelderland, the Netherlands.
• 6 National Institute for Public Health and the Environment (RIVM), Center for Environmental Quality (MIL), Bilthoven, the Netherlands.
• 7 Institute for Risk Assessment Sciences (IRAS), Faculty of Veterinary Medicine, Utrecht University, Utrecht, the Netherlands; National Institute for Public Health and the Environment (RIVM), Centre for Infectious Disease Control (CIb), Bilthoven, the Netherlands. Electronic address: [email protected].
• PMID: 38652943
• DOI: 10.1016/j.ijheh.2024.114382

Air pollution is a known risk factor for several diseases, but the extent to which it influences COVID-19 compared to other respiratory diseases remains unclear. We performed a test-negative case-control study among people with COVID-19-compatible symptoms who were tested for SARS-CoV-2 infection, to assess whether their long- and short-term exposure to ambient air pollution (AAP) was associated with testing positive (vs. negative) for SARS-CoV-2. We used individual-level data for all adult residents in the Netherlands who were tested for SARS-CoV-2 between June and November 2020, when only symptomatic people were tested, and modeled ambient concentrations of PM10, PM2.5, NO 2 and O 3 at geocoded residential addresses. In long-term exposure analysis, we selected individuals who did not change residential address in 2017-2019 (1.7 million tests) and considered the average concentrations of PM10, PM2.5 and NO 2 in that period, and different sources of PM (industry, livestock, other agricultural activities, road traffic, other Dutch sources, foreign sources). In short-term exposure analysis, individuals not changing residential address in the two weeks before testing day (2.7 million tests) were included in the analyses, thus considering 1- and 2-week average concentrations of PM10, PM2.5, NO 2 and O 3 before testing day as exposure. Mixed-effects logistic regression analysis with adjustment for several confounders, including municipality and testing week to account for spatiotemporal variation in viral circulation, was used. Overall, there was no statistically significant effect of long-term exposure to the studied pollutants on the odds of testing positive vs. negative for SARS-CoV-2. However, significant positive associations of long-term exposure to PM10 and PM2.5 from specifically foreign and livestock sources, and to PM10 from other agricultural sources, were observed. Short-term exposure to PM10 (adjusting for NO 2 ) and PM2.5 were also positively associated with increased odds of testing positive for SARS-CoV-2. While these exposures seemed to increase COVID-19 risk relative to other respiratory diseases, the underlying biological mechanisms remain unclear. This study reinforces the need to continue to strive for better air quality to support public health.

Keywords: Air pollution; COVID-19; NO(2); PM10; PM2.5.

Copyright © 2024 The Authors. Published by Elsevier GmbH.. All rights reserved.

SARS-CoV-2 Viral Shedding and Rapid Antigen Test Performance — Respiratory Virus Transmission Network, November 2022–May 2023

Weekly / April 25, 2024 / 73(16);365–371

Sarah E. Smith-Jeffcoat, MPH 1 ; Alexandra M. Mellis, PhD 2 ; Carlos G. Grijalva, MD 3 ; H. Keipp Talbot, MD 3 ; Jonathan Schmitz, MD, PhD 3 ; Karen Lutrick, PhD 4 ; Katherine D. Ellingson, PhD 4 ; Melissa S. Stockwell, MD 5 ,6 ,7 ; Son H. McLaren, MD 8 ; Huong Q. Nguyen, PhD 9 ; Suchitra Rao, MBBS 10 ; Edwin J. Asturias, MD 10 ; Meredith E. Davis-Gardner, PhD 11 ; Mehul S. Suthar, PhD 11 ; Hannah L. Kirking, MD 1 ; RVTN-Sentinel Study Group ( View author affiliations )

What is already known about this topic?

During the COVID-19 pandemic, rapid antigen tests were found to detect potentially transmissible SARS-CoV-2 infection, but antigen tests were less sensitive than reverse transcription–polymerase chain reaction (RT-PCR) testing.

What is added by this report?

During November 2022–May 2023, among persons infected with SARS-CoV-2, sensitivity of rapid antigen tests was 47% compared with RT-PCR and 80% compared with viral culture. Antigen tests continue to detect potentially transmissible infection but miss many infections identified by positive RT-PCR test results.

What are the implications for public health practice?

Rapid antigen tests can aid in identifying infectiousness of persons infected with SARS-CoV-2 and providing access to diagnostic testing for persons with COVID-19 symptoms. Persons in the community eligible for antiviral treatment should seek more sensitive diagnostic tests from a health care provider. Clinicians should consider RT-PCR testing for persons for whom antiviral treatment is recommended.

• Article PDF
• Full Issue PDF

As population immunity to SARS-CoV-2 evolves and new variants emerge, the role and accuracy of antigen tests remain active questions. To describe recent test performance, the detection of SARS-CoV-2 by antigen testing was compared with that by reverse transcription–polymerase chain reaction (RT-PCR) and viral culture testing during November 2022–May 2023. Participants who were enrolled in a household transmission study completed daily symptom diaries and collected two nasal swabs (tested for SARS-CoV-2 via RT-PCR, culture, and antigen tests) each day for 10 days after enrollment. Among participants with SARS-CoV-2 infection, the percentages of positive antigen, RT-PCR, and culture results were calculated each day from the onset of symptoms or, in asymptomatic persons, from the date of the first positive test result. Antigen test sensitivity was calculated using RT-PCR and viral culture as references. The peak percentage of positive antigen (59.0%) and RT-PCR (83.0%) results occurred 3 days after onset, and the peak percentage of positive culture results (52%) occurred 2 days after onset. The sensitivity of antigen tests was 47% (95% CI = 44%–50%) and 80% (95% CI = 76%–85%) using RT-PCR and culture, respectively, as references. Clinicians should be aware of the lower sensitivity of antigen testing compared with RT-PCR, which might lead to false-negative results. This finding has implications for timely initiation of SARS-CoV-2 antiviral treatment, when early diagnosis is essential; clinicians should consider RT-PCR for persons for whom antiviral treatment is recommended. Persons in the community who are at high risk for severe COVID-19 illness and eligible for antiviral treatment should seek testing from health care providers with the goal of obtaining a more sensitive diagnostic test than antigen tests (i.e., an RT-PCR test).

Introduction

SARS-CoV-2 rapid antigen tests were developed and received Food and Drug Administration Emergency Use Authorization early during the COVID-19 pandemic.* These tests were initially rolled out broadly in the United States to diagnose cases and isolate persons who received positive test results to aid in preventing onward spread at a time when population SARS-CoV-2 immunity was low, and rates of severe COVID-19–associated outcomes were high. In addition, demands for testing exceeded supply, and long turnaround times for reverse transcription–polymerase chain reaction (RT-PCR) test results contributed to ongoing transmission. Wide access to antigen tests was made possible through U.S. government initiatives implemented to prevent transmission. † , § After the emergence of the Omicron variant in late 2021, at-home antigen test use began to increase sharply ( 1 , 2 ).

Studies conducted during circulation of SARS-CoV-2 pre-Delta and Delta variants illustrated that antigen tests have high specificity, but lower sensitivity when compared with RT-PCR tests, thereby missing a substantial number of infections but correlating more closely with viral culture results ( 3 – 6 ). Viral culture, although not frequently used for routine patient care, is able to detect actively replicating virus (thus identifying when a person is likely to be infectious), whereas RT-PCR cannot distinguish between replicating virus and viral fragments. Most of these studies included few participants with vaccine- or infection-induced immunity. SARS-CoV-2 variants and population immunity have evolved since many of the studies assessing antigen tests were performed; thus, the role that antigen tests should play in diagnosing SARS-CoV-2 infection remains an active question. The objective of this investigation was to reevaluate the performance characteristics of SARS-CoV-2 antigen tests with those of RT-PCR and viral culture tests during a period with greater population immunity and more recently circulating SARS-CoV-2 Omicron variants.

This evaluation included participants enrolled in an antigen test substudy within a case-ascertained household transmission study during November 2022–May 2023 ¶ ( 7 ). Index patients with confirmed SARS-CoV-2 infection and their household contacts were enrolled within 7 days of illness onset in the index patient. Participants completed baseline surveys including demographic characteristics, COVID-19 signs or symptoms (symptoms),** vaccination, †† and self-reported previous infection. Participants (index patients and contacts) also provided a blood specimen for SARS-CoV-2 anti-N antibody detection §§ ( 8 , 9 ). For 10 days after enrollment, all participants completed daily COVID-19 symptom diaries and collected two nasal swabs each day. One swab was self-collected in viral transport media, stored in refrigerator for up to 72 hours, then collected by a study team member and stored at −12°F (−80°C) until aliquoted for automated RT-PCR (Hologic Panther Fusion) ¶¶ and viral culture,*** and the other swab was used for at-home antigen testing. ††† Participants interpreted and reported their antigen test results in their daily symptom diary. For this analysis, SARS-CoV-2 infection was defined as at least one positive RT-PCR test result during the study period; onset was defined as the first day of symptoms or, if the participant remained asymptomatic, day of first positive test result.

Among participants who ever received a positive RT-PCR test result and had one or more paired RT-PCR and antigen results reported, the percentage of positive antigen, RT-PCR, and viral culture results was calculated for each day relative to onset. The percentage of positive antigen test results was stratified by symptom and fever status. Sensitivity of antigen testing among paired samples collected from 2 days before until 10 days after onset was computed using two references: 1) same-day positive RT-PCR result and 2) same-day positive culture result, stratified by overall symptom status and presence of fever alone or fever or cough. Wilson score intervals were used for calculating 95% CIs around percentage of positive test results. Cluster-robust bootstrapping was used to calculate 95% CIs around sensitivity to account for within-participant correlation. All analyses were performed in RStudio (version 4.2.3; RStudio). This study was reviewed and approved by the Vanderbilt University Institutional Review Board. §§§

Characteristics of Study Participants

Among 354 participants in 129 households, 236 (67%) received a positive SARS-CoV-2 RT-PCR test result and were included in this investigation ( Table ). Participants ranged in age from 2 months to 83 years (median = 36 years; IQR = 17–50 years), 133 (56%) were non-Hispanic White persons, and 140 (59%) were female. Ninety-two (40%) participants reported receipt of a COVID-19 vaccine ≤12 months before enrollment; 82 (35%) had received ≥2 doses, but the most recent dose was >12 months before enrollment; 57 (24%) were unvaccinated (including those who had only ever received 1 dose); and vaccination status was unknown for five participants. A total of 102 (43%) participants had self-reported or serologic evidence of previous SARS-CoV-2 infection. At least one COVID-19 symptom was reported by 219 (93%) participants, including 182 (77%) who reported cough and 156 (66%) who reported fever.

SARS-CoV-2 Test Results

Among the 236 SARS-CoV-2–infected participants (i.e., those who received a positive RT-PCR test result), 2,244 antigen results were reported and included in analyses. Overall, 143 (61%) participants received one or more positive culture result, and 164 (69%) received one or more positive antigen test result.

The highest percentage of positive antigen (59%; 95% CI = 51%–67%) and RT-PCR (83%; 95% CI = 76%–88%) test results occurred 3 days after onset ( Figure 1 ). The highest percentage of positive viral culture results (52%; 95% CI = 43%–61%) occurred 2 days after onset. Among the 219 symptomatic participants, the highest percentage of positive antigen test results was 65% (95% CI = 57%–73%) at 3 days after onset among those who experienced any COVID-19 symptom and 80% (95% CI = 68%–88%) at 2 days after onset among those who reported fever.

Sensitivity of Antigen Testing

Compared with same-day collected RT-PCR and culture results, the overall sensitivities of daily antigen test results were 47% (95% CI = 44%–50%) and 80% (95% CI = 76%–85%), respectively ( Figure 2 ) (Supplementary Table, https://stacks.cdc.gov/view/cdc/153544 ). When stratified by symptoms experienced on the day of specimen collection, antigen test sensitivity increased with occurrence of any COVID-19 symptoms (56% and 85% compared with RT-PCR and culture, respectively) and peaked on days that fever was reported (77% and 94% compared with RT-PCR and culture, respectively). Compared with RT-PCR and culture results, sensitivity of antigen testing was low on days when no symptoms were reported (18% and 45%, respectively).

Among participants enrolled in a household transmission study during a period of increased disease- and vaccine-induced immunity, and when circulating viruses differed antigenically from the ancestral SARS-CoV-2 strain, antigen and culture tests detected a similar proportion of SARS-CoV-2 infections, but detection by RT-PCR was higher than that by either antigen or culture. Similarly, paired antigen test sensitivity was low compared with RT-PCR (47%), but relatively high compared with culture (80%). The sensitivity of antigen testing was higher when symptoms were present on the test day and peaked on days when participants reported fever. Although viral culture is not an absolute marker of transmissibility, this pattern suggests that positive antigen test results could indicate transmissible virus; thus, antigen tests might aid persons with COVID-19 in determining when they are no longer infectious once symptoms begin to resolve.

The findings from this investigation remain similar to those reported in other studies throughout the COVID-19 pandemic ( 3 – 6 ). For example, considering the current study’s sensitivity results, an early 2021 study comparing antigen testing with RT-PCR and culture found similar antigen test sensitivity compared with culture (84%), but slightly higher sensitivity compared with RT-PCR (64%) ( 3 ). The sensitivity difference between these two studies could be attributed to many factors, including differences in participant immunity, infecting variants, the limit of detection of the reference RT-PCR, or sampling methods.

Minimizing false negative test results is important because additional modalities, including antiviral medications, are available to prevent severe outcomes. Antiviral treatments for SARS-CoV-2 infection should be started as soon as possible, and within 5–7 days of symptom onset. ¶¶¶ Therefore, persons who are at higher risk for severe illness and eligible for antiviral treatment would benefit from a more accurate diagnostic test. In most clinical scenarios in the United States, this approach means a SARS-CoV-2 RT-PCR test would be a better diagnostic test to minimize the risk for a false-negative result. Alternatively, if RT-PCR tests are not available or accessible, clinicians and patients should follow FDA’s serial antigen testing recommendations to help optimize diagnostic test performance.****

Limitations

The findings in this report are subject to at least three limitations. First, participants included in this analysis might not represent all U.S. persons infected with SARS-CoV-2 and represent those with mild to moderate illness. These findings might not apply to persons with more severe COVID-19 illness. Second, one commercially available antigen test was used in this study; results might not apply to all available antigen tests. Finally, because of the parent study design, onset for asymptomatic participants (i.e., the day of the first positive test result), could be biased if household members were not enrolled early enough to record the earliest positive test result.

Implications for Public Health Practice

As COVID-19 becomes endemic and public focus shifts from stopping transmission to preventing severe illness, †††† diagnostic testing should emphasize use of the best tests to identify infection in persons who would benefit from treatment. The low sensitivity of antigen testing among persons with asymptomatic infections illustrates that these tests should only be used once symptoms are present. Conversely, the higher sensitivity when symptoms are present (especially cough or fever) supports the need to stay at home when symptomatic, irrespective of test result. §§§§ The low sensitivity of antigen tests compared with RT-PCR tests has implications for timely initiation of anti–SARS-CoV-2 treatment when early and accurate diagnosis is important. With several treatment options available, clinicians should consider more sensitive RT-PCR tests for accurate diagnosis in persons at higher risk for severe illness to minimize delays in treatment initiation. Persons in the community who are at high risk for severe COVID-19 illness and eligible for antiviral treatment should seek testing from health care providers with the goal of obtaining a more sensitive diagnostic test than antigen tests (i.e., an RT-PCR test).

Acknowledgments

Supraja Malladi, CDC; Erica Anderson, Marcia Blair, Jorge Celedonio, Daniel Chandler, Brittany Creasman, Ryan Dalforno, Kimberly Hart, Andrea Stafford Hintz, Judy King, Christopher Lindsell, Zhouwen Liu, Samuel Massion, Rendie E. McHenry, John Meghreblian, Lauren Milner, Catalina Padilla-Azain, Bryan Peterson, Suryakala Sarilla, Brianna Schibley-Laird, Laura Short, Ruby Swaim, Afan Swan, His-nien Tan, Timothy Williams, Paige Yates, Vanderbilt University Medical Center; Hannah Berger, Brianna Breu, Gina Burbey, Leila Deering, DeeAnn Hertel, Garrett Heuer, Sarah Kopitzke, Carrie Marcis, Jennifer Meece, Vicki Moon, Jennifer Moran, Miriah Rotar, Carla Rottscheit, Elisha Stefanski, Sandy Strey, Melissa Strupp, Murdoch Children’s Research Institute; Lisa Saiman, Celibell Y Vargas, Anny L. Diaz Perez, Ana Valdez de Romero, Raul A. Silverio Francisco, Columbia University.

RVTN-Sentinel Study Group

Melissa A. Rolfes, National Center for Immunization and Respiratory Diseases, CDC; Jessica E. Biddle, National Center for Immunization and Respiratory Diseases, CDC; Yuwei Zhu, Vanderbilt University Medical Center, Nashville, Tennessee; Karla Ledezma, University of Arizona, Tucson, Arizona; Kathleen Pryor, University of Arizona, Tucson, Arizona; Ellen Sano, Columbia University Irvin Medical Center, New York, New York; Joshua G. Petrie, Marshfield Clinic Research Institute, Marshfield, Wisconsin.

Corresponding author: Sarah E. Smith-Jeffcoat, [email protected] .

1 Coronavirus and Other Respiratory Viruses Division, National Center for Immunization and Respiratory Diseases, CDC; 2 Influenza Division, National Center for Immunization and Respiratory Diseases, CDC; 3 Vanderbilt University Medical Center, Nashville, Tennessee; 4 University of Arizona Colleges of Medicine and Public Health, Tucson, Arizona; 5 Division of Child and Adolescent Health, Department of Pediatrics, Vagelos College of Physicians and Surgeons, Columbia University, New York, New York; 6 Department of Population and Family Health, Mailman School of Public Health, New York, New York; 7 New York-Presbyterian Hospital, New York, New York; 8 Department of Emergency Medicine, Vagelos College of Physicians and Surgeons, Columbia University, New York, New York; 9 Marshfield Clinic Research Institute, Marshfield, Wisconsin; 10 Children’s Hospital Colorado, Aurora, Colorado; 11 Department of Pediatrics-Infectious Diseases, Emory Vaccine Center, Emory Primate Research Center, Emory University School of Medicine, Atlanta, Georgia.

All authors have completed and submitted the International Committee of Medical Journal Editors form for disclosure of potential conflicts of interest. Edwin J. Asturias reports grant support from Pfizer, consulting fees from Hillevax and Moderna, and payment from Merck for a lecture delivered at the Latin American Vaccine Summit. Carlos G. Grijalva reports support from the Food and Drug Administration and grants from the National Institutes of Health (NIH) and Syneos Health. Son H. McLaren reports institutional support from the Respiratory Virus Transmission Network, receipt of the Ken Graff Young Investigator Award from the American Academy of Pediatrics, Section on Emergency Medicine, institutional support from the National Center for Advancing Translational Science, the National Heart, Lung, and Blood Institute, and the Doris Duke Charitable Foundation COVID-19 Fund to Retain Clinician-Scientists. Suchitra Rao reports grant support from Biofire. Melissa S. Stockwell reports institutional support from the University of Washington, Boston Children’s Hospital, Westat, and New York University, and service as the Associate Director of the American Academy of Pediatrics Pediatric Research in Office Settings Research Network (payment to the trustees of Columbia University). Huong Q. Nguyen reports research support from CSL Seqirus, GSK, and ModernaTX, and an honorarium from ModernaTX for participating in a consultancy group, outside the submitted work. No other potential conflicts of interest were disclosed.

* https://www.fda.gov/news-events/press-announcements/coronavirus-covid-19-update-fda-authorizes-first-antigen-test-help-rapid-detection-virus-causes

† https://www.covid.gov/tools-and-resources/resources/tests

§ https://www.whitehouse.gov/wp-content/uploads/2021/01/National-Strategy-for-the-COVID-19-Response-and-Pandemic-Preparedness.pdf

¶ The Respiratory Virus Transmission Network sites that participated in the antigen substudy were located in Arizona, Colorado, New York, Tennessee, and Wisconsin. Persons who received test results positive for SARS-CoV-2 were recruited from participating medical centers, community testing sites, actively surveilled cohorts, and public health registries at five sites.

** Elicited COVID-19 symptoms included fever (including feeling feverish and chills), cough, sore throat, runny nose, nasal congestion, fatigue (including feeling run-down), wheezing, trouble breathing (including shortness of breath), chest tightness (including chest pain), loss of smell or loss of taste, headache, abdominal pain, diarrhea, vomiting, and muscle or body aches.

†† Vaccination history was self-reported and then verified by study team using state vaccination registries, electronic medical records, and pharmacy records.

§§ Detection of antinucleocapsid antibodies from a dried blood spot collected at baseline was considered serological evidence of previous SARS-CoV-2 infection. Simultaneous detection and differentiation of total binding antibody (immunoglobulin [Ig]M, IgG, and IgA) to SARS-CoV-2 2019-nCoV WHU02 strain nucleocapsid protein, Wuhan-Hu-1 strain spike protein receptor binding domain, and Wuhan-Hu-1 strain spike protein trimer in capillary (finger stick) dried blood was performed using the ProcartaPlex Immunoassay multiplex custom panel (Invitrogen) deployed on the MAGPIX System (Luminex).

¶¶ RT-PCR results were interpreted as categorically positive or negative according to the FDA-authorized parameters of the Hologic Panther Fusion SARS-CoV-2 assay, as utilized for in vitro diagnostic purposes. https://www.fda.gov/media/136156/download?attachment

*** Viral culture was performed on Vero E6 cells expressing both ACE2 and TMPRSS2. Cells were infected with serial dilutions of virus in Dulbecco’s Modified Eagle Medium (DMEM) containing ciprofloxacin, and cytopathic effect (CPE) was visually observed during a period of 5 days. Observation of CPE was considered positive for viral culture.

††† Quidel QuickVue At-Home COVID-19 Test (available as over-the-counter). https://www.fda.gov/media/146312/download

§§§ 45 C.F.R. part 46.114; 21 C.F.R. part 56.114.

¶¶¶ https://www.cdc.gov/coronavirus/2019-ncov/your-health/treatments-for-severe-illness.html

**** https://www.fda.gov/medical-devices/safety-communications/home-covid-19-antigen-tests-take-steps-reduce-your-risk-false-negative-results-fda-safety

†††† https://www.cdc.gov/respiratory-viruses/whats-new/changing-threat-covid-19.html

§§§§ https://www.cdc.gov/respiratory-viruses/prevention/precautions-when-sick.html

• Qasmieh SA, Robertson MM, Rane MS, et al. The importance of incorporating at-home testing into SARS-CoV-2 point prevalence estimates: findings from a US national cohort, February 2022. JMIR Public Health Surveill 2022;8:e38196. https://doi.org/10.2196/38196 PMID:36240020
• Rader B, Gertz A, Iuliano AD, et al. Use of at-home COVID-19 tests—United States, August 23, 2021–March 12, 2022. MMWR Morb Mortal Wkly Rep 2022;71:489–94. https://doi.org/10.15585/mmwr.mm7113e1 PMID:35358168
• Chu VT, Schwartz NG, Donnelly MAP, et al.; COVID-19 Household Transmission Team. Comparison of home antigen testing with RT-PCR and viral culture during the course of SARS-CoV-2 infection. JAMA Intern Med 2022;182:701–9. https://doi.org/10.1001/jamainternmed.2022.1827 PMID:35486394
• Tu Y-P, Green C, Hao L, et al. COVID-19 antigen results correlate with the quantity of replication-competent SARS-CoV-2 in a cross-sectional study of ambulatory adults during the Delta wave. Microbiol Spectr 2023;11:e0006423. https://doi.org/10.1128/spectrum.00064-23 PMID:37097146
• Almendares O, Prince-Guerra JL, Nolen LD, et al.; CDC COVID-19 Surge Diagnostic Testing Laboratory. Performance characteristics of the Abbott BinaxNOW SARS-CoV-2 antigen test in comparison to real-time reverse transcriptase PCR and viral culture in community testing sites during November 2020. J Clin Microbiol 2022;60:e0174221. https://doi.org/10.1128/JCM.01742-21 PMID:34705535
• Currie DW, Shah MM, Salvatore PP, et al.; CDC COVID-19 Response Epidemiology Field Studies Team. Relationship of SARS-CoV-2 antigen and reverse transcription PCR positivity for viral cultures. Emerg Infect Dis 2022;28:717–20. https://doi.org/10.3201/eid2803.211747 PMID:35202532
• Rolfes MA, Talbot HK, Morrissey KG, et al. Reduced risk of SARS-CoV-2 infection among household contacts with recent vaccination and previous COVID-19 infection: results from two multi-site case-ascertained household transmission studies. medRxiv [Preprint posted online October 21, 2023]. https://doi.org/10.1101/2023.10.20.23297317
• Chen L, Liu W, Zhang Q, et al. RNA based mNGS approach identifies a novel human coronavirus from two individual pneumonia cases in 2019 Wuhan outbreak. Emerg Microbes Infect 2020;9:313–9. https://doi.org/10.1080/22221751.2020.1725399 PMID:32020836
• Wu F, Zhao S, Yu B, et al. A new coronavirus associated with human respiratory disease in China. Nature 2020;579:265–9. https://doi.org/10.1038/s41586-020-2008-3 PMID:32015508

Abbreviations: RT-PCR = reverse transcription–polymerase chain reaction; SVI = Social Vulnerability Index. * SARS-CoV-2 infection defined as having received at least one positive RT-PCR result during study testing. † Persons of Hispanic or Latino (Hispanic) origin might be of any race but are categorized as Hispanic; all racial groups are non-Hispanic. § SVI was determined using the 2020 U.S. Census Bureau decennial tract location of the home. SVI uses 16 census variables to indicate the relative vulnerability of every census tract to a hazardous event with values closer to 1 representing highly vulnerable areas and values closer to 0 representing least vulnerable areas. ¶ Vaccination history was self-reported and then verified by study team. Participants were considered vaccinated within 12 months before enrollment if they had received ≥2 doses and the most recent dose was received between 14 days and 12 months before enrollment; vaccinated >12 months before enrollment if they had received ≥2 doses and the most recent dose was received >12 months before enrollment; and unvaccinated if they received <2 doses before enrollment. ** By self-report or serologic evidence. Previous SARS-CoV-2 infection was defined as self-report of a previous infection ≥1 month before enrollment or by detection of antinucleocapsid antibodies from a dried blood spot collected at baseline. †† Elicited COVID-19 signs and symptoms included fever (including feeling feverish or chills), cough, sore throat, runny nose, nasal congestion, fatigue (including feeling run-down), wheezing, trouble breathing (including shortness of breath), chest tightness (including chest pain), loss of smell or loss of taste, headache, abdominal pain, diarrhea, vomiting, and muscle or body aches.

FIGURE 1 . Percentage* of rapid antigen, reverse transcription–polymerase chain reaction, and viral culture test results that were positive for SARS-CoV-2 (A) and percentage of antigen test results that were positive, by symptom status† (B) and presence of fever (C) each day since onset § among participants infected with SARS-CoV-2 ¶ — Respiratory Virus Transmission Network, November 2022–May 2023

Abbreviation: RT-PCR = reverse transcription–polymerase chain reaction.

* With 95% CIs indicated by shaded areas.

† Elicited COVID-19 signs and symptoms included fever (including feeling feverish or chills), cough, sore throat, runny nose, nasal congestion, fatigue (including feeling run-down), wheezing, trouble breathing (including shortness of breath), chest tightness (including chest pain), loss of smell or loss of taste, headache, abdominal pain, diarrhea, vomiting, and muscle or body aches.

§ Date of symptom onset or, for asymptomatic persons, date of first positive test result.

¶ SARS-CoV-2 infection defined as having received at least one positive RT-PCR test result during study testing.

FIGURE 2 . Sensitivity* of rapid antigen tests results for diagnosing SARS-CoV-2 infection compared with reverse transcription–polymerase chain reaction (A) and viral culture (B), overall and by presence of symptoms † — Respiratory Virus Transmission Network, November 2022–May 2023

* With 95% CIs indicated by error bars.

Suggested citation for this article: Smith-Jeffcoat SE, Mellis AM, Grijalva CG, et al. SARS-CoV-2 Viral Shedding and Rapid Antigen Test Performance — Respiratory Virus Transmission Network, November 2022–May 2023. MMWR Morb Mortal Wkly Rep 2024;73:365–371. DOI: http://dx.doi.org/10.15585/mmwr.mm7316a2 .

MMWR and Morbidity and Mortality Weekly Report are service marks of the U.S. Department of Health and Human Services. Use of trade names and commercial sources is for identification only and does not imply endorsement by the U.S. Department of Health and Human Services. References to non-CDC sites on the Internet are provided as a service to MMWR readers and do not constitute or imply endorsement of these organizations or their programs by CDC or the U.S. Department of Health and Human Services. CDC is not responsible for the content of pages found at these sites. URL addresses listed in MMWR were current as of the date of publication.

All HTML versions of MMWR articles are generated from final proofs through an automated process. This conversion might result in character translation or format errors in the HTML version. Users are referred to the electronic PDF version ( https://www.cdc.gov/mmwr ) and/or the original MMWR paper copy for printable versions of official text, figures, and tables.

Exit Notification / Disclaimer Policy

• The Centers for Disease Control and Prevention (CDC) cannot attest to the accuracy of a non-federal website.
• Linking to a non-federal website does not constitute an endorsement by CDC or any of its employees of the sponsors or the information and products presented on the website.
• You will be subject to the destination website's privacy policy when you follow the link.
• CDC is not responsible for Section 508 compliance (accessibility) on other federal or private website.

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

The study on setting priorities of zoonotic agents for medical preparedness and allocation of research resources

Roles Data curation, Formal analysis, Writing – original draft

Affiliation Graduate Institute of Microbiology and Public Health, National Chung Hsing University, Taichung, Taiwan, R.O.C

Roles Supervision, Writing – review & editing

Affiliation Department of Applied Economics, National Chung Hsing University, Taichung, Taiwan, R.O.C

Roles Supervision

Affiliation Children’s Hospital, China Medical University, Taichung, Taiwan, R.O.C

Roles Conceptualization, Formal analysis, Methodology, Supervision, Writing – review & editing

* E-mail: [email protected]

• Kung-Ching Wang, 
• Chia-Lin Chang, 
• Sung-Hsi Wei, 
• Chao-Chin Chang

• Published: April 30, 2024
• https://doi.org/10.1371/journal.pone.0299527
• Peer Review
• Reader Comments

The aim of this study is to develop a scoring platform to be used as a reference for both medical preparedness and research resource allocation in the prioritization of zoonoses. Using a case-control design, a comprehensive analysis of 46 zoonoses was conducted to identify factors influencing disease prioritization. This analysis provides a basis for constructing models and calculating prioritization scores for different diseases. The case group (n = 23) includes diseases that require immediate notification to health authorities within 24 hours of diagnosis. The control group (n = 23) includes diseases that do not require such immediate notification. Two different models were developed for primary disease prioritization: one model incorporated the four most commonly used prioritization criteria identified through an extensive literature review. The second model used the results of multiple logistic regression analysis to identify significant factors (with p-value less than 0.1) associated with 24-hour reporting, allowing for objective determination of disease prioritization criteria. These different modeling approaches may result in different weights and positive or negative effects of relevant factors within each model. Our study results highlight the variability of zoonotic disease information across time and geographic regions. It provides an objective platform to rank zoonoses and highlights the critical need for regular updates in the prioritization process to ensure timely preparedness. This study successfully established an objective framework for assessing the importance of zoonotic diseases. From a government perspective, it advocates applying principles that consider disease characteristics and medical resource preparedness in prioritization. The results of this study also emphasize the need for dynamic prioritization to effectively improve preparedness to prevent and control disease.

Citation: Wang K-C, Chang C-L, Wei S-H, Chang C-C (2024) The study on setting priorities of zoonotic agents for medical preparedness and allocation of research resources. PLoS ONE 19(4): e0299527. https://doi.org/10.1371/journal.pone.0299527

Editor: Balbir B. Singh, Guru Angad Dev Veterinary and Animal Sciences University, INDIA

Received: October 11, 2023; Accepted: February 13, 2024; Published: April 30, 2024

Copyright: © 2024 Wang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All data relevant to the analysis in this study were collected from the websites or references described in the Materials and Methods. For transformed data, all information was summarized in Tables 2 , 4 , 5 , 6 , and 9 .

Funding: The study was partially supported by the National Science and Technology Council, Executive Yuan, Taiwan in the form of a grant to C-CC [NSTC-111-2313-B-005-045-MY2].

Competing interests: The authors have declared that no competing interests exist.

1. Introduction

Zoonoses refers to diseases that that are naturally transmitted between vertebrate animals and man,” as defined in 1951 by the World Health Organization (WHO) Expert Committee on Zoonoses [ 1 ]. Approximately 60% of known human infectious diseases and 75% of emerging infectious diseases are caused by zoonotic agents [ 2 ]. With the expansion of urbanization and agriculture, humans are coming into more frequent contact with wildlife. Additionally, climate change and global trade also contribute to the spread of zoonses. International travel and population movement facilitate the global spread of infectious diseases. Once a zoonosis has been introduced into a country, factors such as urbanization and an aging population increase the likelihood of disease transmission and cause high case-fatality rates in humans. Moreover, several factors related to the characteristics of the pathogen, including its mode of transmission, influence the magnitude of an epidemic of a zoonosis. The availability of therapeutic agents or vaccines to prevent viral or bacterial diseases also determines the epidemic scale of a zoonosis once occurred. As a result, preventing and controlling zoonoses have become critical public health issues worldwide [ 3 ].

In recent years, the discovery of more emerging and re-emerging zoonoses has prompted countries to develop effective response measures and prepare relevant medical resources. However, regarding the purpose for development strategies for disease prevention and diagnosis, it is also necessary to establish a priority ranking system for communicable diseases to allocate research resources. Up to date, the methodology of One Health Zoonotic Disease Prioritization (OHZDP) has been recommended by CDC in the US as a tool for zoonoses prioritization, and various methods used for this purpose in different countries include the Hirsch index (h-index), Delphi technique, multi-criteria decision analysis (MCDA), and questionnaires; each method is with its advantages and disadvantages [ 4 ]. Consequently, achieving a consensus on the methods for prioritizing diseases is challenging [ 5 ].

Therefore, the purpose of this study is to respectively establish medical preparedness priority ranking and research priority ranking systems for zoonotic infectious diseases. Using a case-control study for comparing relative importance of zoonoses, statistical analyses on the corresponding epidemiological data were conducted to identify the most critical factors for setting priorities. Subsequently, we use the constructed statistical model to calculate the ranking score for each zoonoses, which can be applied for disease prioritization.

2. Materials and methods

Developing a disease priority ranking method with binary logistic regression model.

This study involved two main models: Models A and B. The factors used in Model A were chosen through literature review approach, and the frequently used four factors for disease prioritization were included in the model after literature summarization (please refer to description given below). For the construction of Model B, the influencing factors used in this study for prioritizing diseases were based on results of statistical analysis using a case-control study design approach; categorical variables were analyzed using Chi-square test or Fisher’s exact test, and continuous variables were analyzed using independent t-test. Based on the results of the univariate analysis, a binary logistic regression model was used for constructing the multiple logistic regression model to determine the weight of each factor for disease prioritization. While statistical significance of a factor was determined with p < 0.05, factors with p < 0.1 in the univariate analysis were still included in the multiple logistic model to adjust potential confounding effect.

The dependent variable in the binary logistic regression model was based on the concept of a case-control study, with diseases requiring reporting within 24 hours considered as the case group and diseases not requiring such reporting as the control group. After completing the construction of the model, the weight for each factor was determined based on the obtained odds ratio (OR) value: OR ≥ 4 or ≤ 0.25 received a weight of 4; OR values between 3–4 or 0.25–0.33 received a weight of 3; OR values between 2–3 or 0.33–0.5 received a weight of 2, and OR values between 1–2 or 0.5–1 received a weight of 1. In Models A and B, further sub-models (Models A.1, A.2, and B.1, B.2) were constructed based on the different prioritization needs, namely “the characteristics of the disease itself and the ability to prepare medical resources” or “the need for stricter border controls and enhanced research on vaccine development or therapeutic drugs”, as these two purposes may lead to totally different prioritization ranking system while assigning a positive or negative value of the weight.

Finally, after the weight of a factor has been determined, the total score for each disease was calculated for prioritization using the formulated overall equation. Data analysis was performed using IBM SPSS Statistics for Windows, Version 20.0. Armonk, NY: IBM Corp.

Criteria of literature search and review for the study

Using “priorit* and zoono*/disease*” as keywords, a literature search was conducted in PubMed for articles published between 2010 and 2020, resulting in a total of 713 articles. After further screening the titles to remove misclassified articles, 55 relevant articles remained. These 55 articles were further filtered based on the following criteria: published in English, containing methods for prioritizing diseases, listing prioritization criteria, and with substantial results. This resulted in a final selection of 25 relevant articles for literature summarization in this study ( Table 1 ) [ 6 – 30 ]. Through careful review of the 25 relevant articles, frequency of the 17 criteria were then summarized, and the top 3 criteria were identified and further used for zoonosis prioritization in this study.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0299527.t001

Disease selection for the study.

Notifiable diseases refer to communicable diseases classified by Taiwan’s Centers for Disease Control and Prevention (CDC) based on the level of risk, such as mortality rate, incidence rate, and transmission speed; cases related to all notifiable diseases must be reported to the CDC. As the main focus of this study is related to prioritization of “zoonoses”, the diseases were further selected from the CDC’s website. Furthermore, a case-control study was applied for comparison to determine the most influential factors associated with disease prioritization through statistical analysis. The cases included a total of 23 diseases that require reporting within 24 hours, while the controls included also 23 diseases that do not require such reporting ( Table 2 ).

https://doi.org/10.1371/journal.pone.0299527.t002

Data collection for factors relevant to disease prioritization

The case-fatality rate in humans, availability of treatment and vaccines for humans, pathogenicity, and transmission modes may vary due to new study information and different data resources. This study therefore collected relevant data from international references such as “Mandell, Douglas, and Bennett’s Principles and Practice of Infectious Diseases” [ 31 ], as well as the websites of the Centers for Disease Control and Prevention (CDC) of the United States [ 32 ], and the World Health Organization (WHO) [ 33 ], to gather these information. Regarding the disease incidence, we used the number of confirmed disease cases from 2018 to 2020 obtained from the website of the Taiwan Centers for Disease Control and Prevention (CDC) infectious disease statistics query system [ 34 ], and the mid-year population of Taiwan from 2018 to 2020 obtained from the website of the Ministry of the Interior [ 35 ] to calculate yearly incidence for notifiable diseases.

For non-notifiable diseases, incidence data was collected through literature review from PubMed using the disease name and "Taiwan" as keywords. To identify whether the outbreak has been ever occurred in Taiwan and any arthropod vector responsible for the transmission, we used the disease name and "Taiwan" as keywords to search for relevant literature in PubMed and the epidemic report from Taiwan CDC. Regarding the case-fatality rate and availability of vaccines in animals, as well as the animal species that could be infected, the information was collected from the official websites of the Iowa State University Food Safety and Public Health Center [ 36 ] (The Center for Food Security and Public Health, Iowa State University, USA 2022), the World Organization for Animal Health (WOAH) [ 37 ], and the MSD Veterinary Manual website [ 38 ]. If no data was available from these sources, relevant literature was searched using the disease’s English name in PubMed. The statistics annual reports published by the Bureau of Animal and Plant Health Inspection and Quarantine (BAPHIQ) [ 39 ], Taiwan, from 2018 to 2020 were reviewed to determine whether the disease has ever occurred in animals. Whether the pathogen can be used as bioterrorism agent was checked in the website of the Centers for Disease Control and Prevention in USA [ 40 ]. Relevant information collected from World Health Organization (WHO) [ 41 , 42 ] were collected to identify whether the disease occurs in humans or animals needs to be reported to WHO or WOAH, respectively.

Definition of the risk score of a country associated with Taiwan

To determine the risk score of a country, the following steps were taken and analyzed. Disease occurrence data from 2010 to 2020 were collected from the websites of National Health Commission in People’s Republic of China [ 43 ], National Institute of Infectious Diseases in Japan [ 44 ], Disease Management Headquarters in South Korea [ 45 ], Epidemiology Bureau, Department of Health in the Philippines [ 46 ], Ministry of Health Portal in Vietnam [ 47 ], Department of Disease Control in Thailand [ 48 ], Ministry of Health in Indonesia [ 49 ], Ministry of Health in Malaysia [ 50 ], Ministry of Health in Singapore [ 51 ], Centre for Health Protection in Hong Kong [ 52 ], and the Centers for Disease Control and Prevention in USA [ 53 ]. If a country had no relevant data on a particular disease, PubMed was used to search for related literature using the disease name and the country’s name. Furthermore, the risk assessment of countries closely related to Taiwan was based on data collected from the Tourism Bureau’s tourism statistics database of inbound travelers to Taiwan from 2010 to 2020 [ 54 ]. The data on the number of residents in each country and the number of Taiwanese outbound travelers to each country were combined. The top 11 countries with the highest total number of inbound travelers, outbound travelers, and migrant workers were selected for further evaluation; a risk score (from 1 to 4) was assigned to each country based on the quartile distribution of the total number of people and whether the disease has ever occurred in the country from 2010 to 2020 [ 55 ].

The definition of a score to present the importance of diversity of transmission routes of a disease.

Different zoonoses may have various transmission routes. The more diverse these routes are, the more challenging on disease control becomes. Therefore, the weights of transmission routes were determined by assessing five variables: contact transmission, airborne transmission, foodborne transmission, vector-borne transmission, and person-to-person transmission. These variables were incorporated into the multiple logistic regression model after a case-control study analysis as defined above, and corresponding scores were assigned based on the calculated odds ratio (OR) values. Based on the OR values after such statistical analysis, a score of 4 was assigned for foodborne or person-to-person transmission, 3 for contact or vector-borne transmission, and 2 for airborne transmission. As a disease may have various transmission routes, the total score for each disease based on their demonstrated transmission routes was calculated. The scores of all diseases evaluated in this study were then subjected to distribution analysis and score quartiles were determined. Using the quartiles, a weighted value of 4 of a disease was assigned if the total score was ≥9, ≥4 and <9 received a weighted value of 3, a score of 3 received a weighted value of 2, and the score less than 3 received a weighted value of 1. The weighted value was finally used to present diversity of transmission routes of a disease.

Identification of factors for model construction and settings of their weights

In Model A, 25 relevant literatures were reviewed. The results showed that the three most commonly used conditions, were “severity of the disease in humans”, “occurrence of the disease in the country”, and “curability in humans” ( Table 3 ).

https://doi.org/10.1371/journal.pone.0299527.t003

The factor "availability of prevention measures in humans" and the factor "economic losses" were tied in the 4th rank for common usage. However, the economic loss factor was not included in our final model, since its definition was relevant to whether the infection in animals needed to be reported to the World Organization for Animal Health (WOAH), which was based on disease and pathogen characteristics. Therefore, it could potentially lead to collinearity issues with the other factors in the model. Consequently, the four most commonly used conditions corresponded to the factors “human case fatality rate exceeding 5%,” “occurrence in Taiwan,” “curability in humans,” and “availability of prevention measures in humans” were finally included in Model A.

In Model B, regarding the impact based on the risk score of a country associated with Taiwan, the total impact score and weight for each disease were calculated ( Table 4 ). Diseases that received a weight score of 4 included dengue fever, Chikungunya fever, Zika viral infection, melioidosis, leptospirosis, severe COVID-19 infection, Japanese encephalitis, listeriosis, scrub typhus, toxoplasmosis, Q fever, salmonellosis, cryptosporidiosis, and Streptococcus suis type 2 infection. These diseases with easily transmissible nature received high scores primarily due to their wide-ranging impact on multiple countries.

https://doi.org/10.1371/journal.pone.0299527.t004

Regarding the impact of multiple transmission routes for each disease, the results showed that diseases receiving a weight score of 4 for the transmission route included plague, anthrax, melioidosis, novel influenza A viral infection, MERS-CoV2 viral infection, Lassa fever, brucellosis, Q fever, tularemia, bovine tuberculosis, salmonellosis, and Nipah viral infection (please refer to Table 5 ).

https://doi.org/10.1371/journal.pone.0299527.t005

The overall prioritization results after the analysis of Model A.

As the above results, Model A utilizes four factors: “case-fatality rate in humans > 5%,” “the disease ever occurred in Taiwan,” “the disease with therapeutic drugs”, and “availability of prevention measures in humans” for multiple logistic regression analysis. After the regression analysis and based on the OR value derived from each factor, the weight was determined. As mentioning in the materials and methods, model construction needs to consider different purposes for disease prioritization. Therefore, further sub-models (Models A.1 and A.2) were constructed based on different prioritization needs, namely “the characteristics of the disease itself and the ability to prepare medical resources” or “the need for stricter border controls and enhanced research on vaccine development or therapeutic drugs”. According to these two different main purposes, in Models A.1 and A.2, further consideration is subjectively to assign the positive or negative value for the weight of each factor for calculation of prioritization scores ( Table 6 ).

https://doi.org/10.1371/journal.pone.0299527.t006

Model A.1 is constructed based on the consideration of “disease characteristics and the availability of medical resources”. The formula for calculating each disease priority ranking score is as follows: Total score = 3 * (case-fatality rate in humans > 5%) + 1 * (the disease ever occurred in Taiwan) + 4 * (the disease with therapeutic drugs in humans) + 4 * (with available preventive measures in humans). The results show that the diseases with the highest ranking are leptospirosis, bovine tuberculosis, and Hantavirus syndrome, all scoring 12 points. The diseases with the second-highest ranking are plague, anthrax, novel influenza A viral infection, Ebola viral infection, and tularemia, all scoring 11 points. Severe COVID-19 infection and Q fever ranked third, both scoring 9 points (see Table 7 ).

https://doi.org/10.1371/journal.pone.0299527.t007

Model A.2 is constructed based on the consideration of “the need for stricter border controls and enhanced research on vaccines or therapeutic drugs for the disease”. In comparison to Model A.1, the weights for “the disease with therapeutic drugs in humans” and “with available preventive measures in humans” are unchanged but subjectively assigned to negative values. The formula for calculating the disease priority ranking score is as follows: Total score = 3 * (case-fatality rate > 5% in humans) + 1 * (the disease ever occurred in Taiwan) - 4 * (the disease with therapeutic drugs in humans) - 4 * (with available preventive measures in humans). The results show that the diseases with the highest ranking are Zika viral infection and SFTS, both scoring 4 points. The diseases with the second-highest ranking are SARS, West Nile fever, Rift Valley fever, MERS-CoV2 viral infection, Marburg viral infection, Hendra viral infection, new variant Creutzfeldt-Jakob disease, and Nipah viral infection, all scoring 3 points. Chikungunya fever ranks third, scoring 1 point ( Table 7 ).

The overall prioritization results after the analysis of Model B.

Based on the case-control study design and the univariate analysis results ( Table 8 ), a total of nine factors were found to meet the criteria for inclusion in the multiple logistic regression model with factors that showed p value less than 0.1. The factors include “human case-fatality rate of the disease >5%,” “human case ever occurred in Taiwan, 2018 to 2020,” “diverse transmission modes of the disease,” “the disease with therapeutic drugs”, “preventive measures available in humans,” “pathogen type,” “the disease can infect economic animals,” “the disease needs to be reported to WHO,” and “the risk score of a country associated with Taiwan”. However, upon further consideration, the factor “disease needs to be reported to WHO” was excluded from the model, because it is a result based on WHO’s consideration of disease characteristics, and it may exhibit collinearity with other related factors in the model (e.g., “human case fatality rate >5%”). Moreover, as “pathogen type” is highly correlated with “human case fatality rate >5%” (e.g., viral infections), it was also excluded from the model to avoid potential collinearity. The factor “human case ever occurred in Taiwan, 2018 to 2020” was not included in the model due to difficulties in obtaining accurate information for the disease, especially for the controls that include most of the diseases not reported within 24 hours and might underestimate its importance.

https://doi.org/10.1371/journal.pone.0299527.t008

According to the analysis by multiple logistic regression model, the four factors “case-fatality rate > 5% in humans”, “the disease with therapeutic drugs”, “available vaccine for prevention in humans”, and “with the ability to infect economic animals” were assigned weights based on their odds ratio values (please refer to Table 9 ). The weights for the “the risk score of a country associated with Taiwan” and “multiple transmission routes” for each disease were listed in Table 4 and 5 , respectively. Once the weights for each factor were determined, the final score for disease prioritization was calculated using the constructed model.

https://doi.org/10.1371/journal.pone.0299527.t009

Model B.1 is constructed based on the consideration of “disease characteristics and the availability of medical resources” ( Table 9 ). The formula for calculating each disease priority ranking score is as follows: Total score = 4 * (case-fatality rate > 5% in humans) + 4 * (the disease with therapeutic drugs) + 4 * (available vaccine for prevention in humans) + 4 * (with the ability to infect economic animals) + the weight score for countries with close exchanges + the weight score for transmission routes. The results show that the diseases with the highest ranking are anthrax, leptospirosis, novel influenza A viral infection, and bovine tuberculosis, all scoring 22 points. The diseases with the second-highest ranking are melioidosis, Q fever, and salmonellosis, all scoring 20 points. Listeriosis and Streptococcus suis type 2 infection rank third, both scoring 19 points. The scores and rankings of other diseases are listed in Table 10 .

https://doi.org/10.1371/journal.pone.0299527.t010

Model B.2 is constructed based on the consideration of “the need for stricter border controls and enhanced research on vaccines or therapeutic drugs for the disease.” In comparison to Model B.1, the weights for “the disease with therapeutic drugs” and “available vaccine for prevention in humans” remain the same but subjectively assigned to negative values ( Table 9 ). The formula for calculating the disease priority ranking score is as follows: Total score = 4 * (case-fatality rate > 5% in humans) - 4 * (the disease with therapeutic drugs) - 4 * (available vaccine for prevention in humans) + 4 * (with the ability to infect economic animals) + weight score for countries with close exchanges + weight score for transmission routes. The results show that the disease with the highest ranking is Zika viral infection, scoring 15 points. The diseases with the second-highest ranking are melioidosis, salmonellosis, Nipah viral infection, West Nile fever and SFTS, all scoring 12 points. Rift Valley fever, Streptococcus suis type 2 infection, new variant Creutzfeldt-Jakob disease and listeriosis rank third, all scoring 11 points. The scores and rankings of other diseases are listed in Table 10 .

4. Discussion and conclusions

The construction of the model for disease prioritization requires a scientific and objective evaluation to include factors, as well as consideration of the relevant weight for the factor. This study showed several new directions for prioritization on scientific basis. Firstly, to address these research focuses, Model A utilizes a literature-based approach to establish a prioritization scoring model using commonly cited factors from the past studies. On the other hand, Model B adopts a more objective approach based on whether a disease needs to be reported within 24 hours to the government, aiming to identify important influencing factors and establish a prioritization scoring model. Secondly, regarding the assignment of weights for each factor, an objective method by logistic regression is employed and the results based on the range of odds ratio (OR) value was used as the reference to assign the weight. In this study, we also developed a scientific platform to determine the risk score of an associated country and the score regarding a disease with multiple transmission routes. Finally, we raised a new idea for two different purposes of disease prioritization, including “the characteristics of the disease and the availability of medical resources” or “the need for stricter border controls and enhanced research on vaccines or therapeutic drugs”. A negative or positive value needs to further consider different prioritization goals and subjectively assigned to the weight of each factor in order to calculate the prioritization score for a disease. Therefore, this study not only provides zoonoses prioritization results for medical references, but also presents innovative research methods for studying disease prioritization.

The results of this study, compared with similar studies conducted in other countries, show differences regarding prioritization results of disease rankings. These differences may be attributed not only to the different approaches used for model construction but also to the uniqueness of each country’s situation. These different also highlight the importance of this research, indicating that each country needs to consider not only the characteristics of the diseases but also the overall national context to study disease prioritization. It is not appropriate to directly adopt the ranking results developed in other countries without careful consideration.

Although there is no complete alignment between the disease prioritization orders of Taiwan and other countries, some diseases remain high priority in both Taiwan and other countries. For instance, the novel influenza A viral infection is considered a high-priority disease in the Netherlands, Vietnam, India, Uganda, and Colombia [ 9 , 13 , 21 , 28 , 56 ]. Anthrax is also regarded as a high-priority disease in India, Kenya, Uganda, and Australia [ 19 , 27 , 28 , 57 ]. Q fever is a high-priority disease in Italy and Australia [ 29 , 57 ]. West Nile fever is a high-priority disease in Burkina Faso and Canada [ 14 , 22 ]. These cases demonstrate that these zoonoses, due to their high case- fatality rates in humans, receive attention in multiple countries.

One of the most significant diseases that recently garnered international attention is monkeypox. On July 23, 2022, the World Health Organization (WHO) declared monkeypox a global public health emergency, as it started to spread in Europe and North America since May 2022. Monkeypox is classified into the West African strain and the Congo strain. The current epidemic is caused by a virus strain similar to the West African strain, with a fatality rate of 3.6% [ 58 ]. The disease mainly spreads through contact and inhalation, caused by a virus with no known infection in economic animals. Although there is no specific antiviral treatment for monkeypox, tecovirimat used to treat smallpox can be employed for treatment. The live attenuated vaccine can also confer protection two weeks after two doses [ 59 ]. If monkeypox is further assessed using the Model A.1 and A.2 frameworks, its scoring results are 9 and -7, respectively. Under Model B.1 and B.2, the scores are 14 and -2, respectively. Comparing these results with the ranking outcomes for zoonotic diseases in Tables 4 and 7 , monkeypox ranks the third tied with Q fever and severe COVID-19 infection in A.1, and the eighth tied with toxoplasmosis and cryptosporidiosis in B.1. In Models A.2 and B.2, monkeypox ranks last tied with Q fever and severe COVID-19 infection in A.2, and the last one without any disease ties in B.2. Therefore, when considering the importance of monkeypox in the future, clear objectives should be set based on either “medical resource preparedness” or “stricter border controls and enhanced research on vaccines or therapeutic drugs” to ensure a scientifically and objectively founded consideration. This outcome also demonstrates the predictive and applicative nature of the models used in this research. Considering the overall needs of national epidemic prevention agencies, it is recommended to prioritize consideration based on the ranking results under the category of “characteristics of the disease and the availability of medical resources”.

The priority order of diseases may change over time or with the emergence of new infectious diseases. Therefore, it is necessary to regularly re-assess the priority of diseases. However, no specific research has determined how often disease prioritization should be evaluated. The WHO’s R&D Blueprint is a global strategic and preparedness plan designed to rapidly initiate research and development during major disease outbreaks. Its first disease prioritization was conducted in 2015 and subsequent reevaluations were done in 2017 and 2018 [ 60 ]. The European CDC recommends periodic reevaluation when disease drivers change or when new diseases emerge that could affect rankings [ 5 ]. Among countries that have previously conducted disease prioritization, Germany did so in 2008 and 2011 [ 7 , 10 ]. In our study, seven criteria were considered: case-fatality rate > 5% in humans, effective treatment available, existence of preventive measures for humans, historical occurrence in Taiwan, ability to infect economic animals, level of risk in closely interacting countries, and modes of disease transmission. While criteria related to infecting economic animals and disease transmission routes are less likely to change due to pathogen characteristics, advancements in disease monitoring and detection methods may provide new scientific evidence. For criteria related to the case-fatality rate, effective treatment availability, existence of preventive measures for humans, historical occurrence in a country, and level of risk in closely interacting countries, continuous research could also be changed according to new findings. Therefore, periodic reevaluation is necessary when changes in time and new scientific data emerge, to re-assess the results regarding disease prioritization.

In comparison to other methods for evaluating disease priority, particularly OHZDP, our models hold the advantage of assessing disease priority with less manpower and in a more time-efficient manner. The OHZDP process brings together representatives from human, animal, and environmental health sectors, along with other relevant partners, to prioritize the most concerning zoonotic diseases for multisectoral One Health collaboration in a country, region, or an area. While the workshop demands both time and financial resources, our model, although requiring data collection, excels in evaluating diseases with reduced manpower and greater time efficiency.

This study had some limitations. The case group and control group consisted of a total of 46 diseases, so adding more independent variables to the model could lead to instability of the constructed model. Some diseases have wild animals as their hosts, resulting in limited data that could be obtained relevant to animal occurrence (only three diseases in the case group and five in the control group) and animal fatality rate (only 12 diseases with available information in the case group and 14 in the control group). Due to these limitations in animal data, the study primarily focused on the impact of diseases on humans and could not thoroughly evaluate the economic impact of certain diseases. Future research should address this limitation and conduct more in-depth studies to modify the model. Regarding closely interacting countries, it is also needs to concern countries that may have limited accessibility of monitoring data for diseases. Diseases not included in monitoring data were collected from PubMed, and diseases published in literature typically have special or severe characteristics. Additionally, this study used known zoonotic diseases to construct the model, but the global prevalence of emerging pathogens is constantly evolving. Therefore, future research should continually incorporate newly discovered pathogens for renew disease prioritization results.

• 1. Chomel BB. Zoonoses. In: Schaechter M (eds). Encyclopedia of Microbiology (Third edition). San Diego: SDSU; 2009. pp.820–829.
• View Article
• PubMed/NCBI
• Google Scholar
• 3. Lipkin WI. Zoonoses. In: Mandell GL, Bennett JE, Dolin R, eds. Mandell, Douglas, and Bennett’s Principles and Practice of Infectious Diseases. Philadelphia: PA:Elsevier/Saunders; 2019. pp.3813–7.
• 5. European Centre for Disease Prevention and Control. Literature review: best practices in ranking emerging infectious disease threats. Stockholm 2015. Available from: https://doi.org/10.2900/653753 .
• 31. Mandell GL, Bennett JE, Dolin R. Mandell, Douglas, and Bennett’s principles and practice of infectious diseases. Philadelphia: PA:Elsevier/Saunders; 2019.
• 33. World Health Organization. [Cited 2022 November 7]. Available from: https://www.who.int/ .
• 35. Department of Household Registration, Ministry of the Interior, Republic of China (Taiwan). [Cited 2022 November 7]. Available from: https://www.ris.gov.tw/app/portal .
• 36. Center for Food Security and Public Health, Iowa State University, USA. Homepage. [Cited 2022 August 1]. Available from: https://www.cfsph.iastate.edu/ .
• 37. World Organization for Animal Health. [Cited 2022 August 1]. Available from: https://www.woah.org/en/home/ .
• 38. MSD veterinary manual. Merck & Co., Inc., Rahway, NJ, USA. Homepage. [Cited 2022 August 1]. Available from: https://www.msdvetmanual.com/ .
• 39. Animal and Plant Health Inspection Agency, Ministry of Agriculture, Republic of China (Taiwan). [Cited 2022 November 7]. Available from: https://www.aphia.gov.tw/ .
• 40. Centers for Disease Control and Prevention, United States. Bioterrorism agents/diseases. [Cited 2022 November 7]. Available from: https://emergency.cdc.gov/agent/agentlist-category.asp .
• 41. World Health Organization. International health regulations (IHR). [Cited 2022 November 7]. Available from: https://www.who.int/health-topics/international-health-regulations#tab=tab_1 .
• 42. World Organization for Animal Health. Animal Diseases. [Cited 2021 December 15]. Available from: https://www.oie.int/en/what-we-do/animal-health-and-welfare/animal-diseases/ .
• 47. Ministry of Health. Health statistics yearbook, Vietnam. [Cited 2022 December 15]. Available from: https://moh.gov.vn/thong-ke-y-te .
• 48. Institute of Research Knowledge Management and Standards for Disease Control. Thailand. Prevention research program control disease and health hazards, 2019–2021. [Cited 2022 December 15]. Available from: http://irem.ddc.moph.go.th/book/detail/92 .
• 50. Ministry of Health, Malaysia. Annual report. Available from: https://www.moh.gov.my/moh/resources/Penerbitan/Penerbitan%20Utama/ANNUAL%20REPORT/LAPORAN_TAHUNAN_KKM_2019.pdf . (last accessed on 15/12/2021).
• 55. Ministry of Labor, Republic of China (Taiwan). [Cited 2022 April 30]. Available from: https://statfy.mol.gov.tw/index12.aspx .
• 59. Centers for Disease Control and Prevention, United States. Monkeypox. [Cited 2022 July 1] Available from: https://www.cdc.gov/poxvirus/monkeypox/index.html .