online education during covid 19 research paper

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Wiley - PMC COVID-19 Collection

Students’ experience of online learning during the COVID‐19 pandemic: A province‐wide survey study

Lixiang yan.

1 Centre for Learning Analytics at Monash, Faculty of Information Technology, Monash University, Clayton VIC, Australia

Alexander Whitelock‐Wainwright

2 Portfolio of the Deputy Vice‐Chancellor (Education), Monash University, Melbourne VIC, Australia

Quanlong Guan

3 Department of Computer Science, Jinan University, Guangzhou China

Gangxin Wen

4 College of Cyber Security, Jinan University, Guangzhou China

Dragan Gašević

Guanliang chen, associated data.

The data is not openly available as it is restricted by the Chinese government.

Online learning is currently adopted by educational institutions worldwide to provide students with ongoing education during the COVID‐19 pandemic. Even though online learning research has been advancing in uncovering student experiences in various settings (i.e., tertiary, adult, and professional education), very little progress has been achieved in understanding the experience of the K‐12 student population, especially when narrowed down to different school‐year segments (i.e., primary and secondary school students). This study explores how students at different stages of their K‐12 education reacted to the mandatory full‐time online learning during the COVID‐19 pandemic. For this purpose, we conducted a province‐wide survey study in which the online learning experience of 1,170,769 Chinese students was collected from the Guangdong Province of China. We performed cross‐tabulation and Chi‐square analysis to compare students’ online learning conditions, experiences, and expectations. Results from this survey study provide evidence that students’ online learning experiences are significantly different across school years. Foremost, policy implications were made to advise government authorises and schools on improving the delivery of online learning, and potential directions were identified for future research into K‐12 online learning.

Practitioner notes

What is already known about this topic

• Online learning has been widely adopted during the COVID‐19 pandemic to ensure the continuation of K‐12 education.
• Student success in K‐12 online education is substantially lower than in conventional schools.
• Students experienced various difficulties related to the delivery of online learning.

What this paper adds

• Provide empirical evidence for the online learning experience of students in different school years.
• Identify the different needs of students in primary, middle, and high school.
• Identify the challenges of delivering online learning to students of different age.

Implications for practice and/or policy

• Authority and schools need to provide sufficient technical support to students in online learning.
• The delivery of online learning needs to be customised for students in different school years.

INTRODUCTION

The ongoing COVID‐19 pandemic poses significant challenges to the global education system. By July 2020, the UN Educational, Scientific and Cultural Organization (2020) reported nationwide school closure in 111 countries, affecting over 1.07 billion students, which is around 61% of the global student population. Traditional brick‐and‐mortar schools are forced to transform into full‐time virtual schools to provide students with ongoing education (Van Lancker & Parolin,  2020 ). Consequently, students must adapt to the transition from face‐to‐face learning to fully remote online learning, where synchronous video conferences, social media, and asynchronous discussion forums become their primary venues for knowledge construction and peer communication.

For K‐12 students, this sudden transition is problematic as they often lack prior online learning experience (Barbour & Reeves,  2009 ). Barbour and LaBonte ( 2017 ) estimated that even in countries where online learning is growing rapidly, such as USA and Canada, less than 10% of the K‐12 student population had prior experience with this format. Maladaptation to online learning could expose inexperienced students to various vulnerabilities, including decrements in academic performance (Molnar et al.,  2019 ), feeling of isolation (Song et al.,  2004 ), and lack of learning motivation (Muilenburg & Berge,  2005 ). Unfortunately, with confirmed cases continuing to rise each day, and new outbreaks occur on a global scale, full‐time online learning for most students could last longer than anticipated (World Health Organization,  2020 ). Even after the pandemic, the current mass adoption of online learning could have lasting impacts on the global education system, and potentially accelerate and expand the rapid growth of virtual schools on a global scale (Molnar et al.,  2019 ). Thus, understanding students' learning conditions and their experiences of online learning during the COVID pandemic becomes imperative.

Emerging evidence on students’ online learning experience during the COVID‐19 pandemic has identified several major concerns, including issues with internet connection (Agung et al.,  2020 ; Basuony et al.,  2020 ), problems with IT equipment (Bączek et al.,  2021 ; Niemi & Kousa,  2020 ), limited collaborative learning opportunities (Bączek et al.,  2021 ; Yates et al.,  2020 ), reduced learning motivation (Basuony et al.,  2020 ; Niemi & Kousa,  2020 ; Yates et al.,  2020 ), and increased learning burdens (Niemi & Kousa,  2020 ). Although these findings provided valuable insights about the issues students experienced during online learning, information about their learning conditions and future expectations were less mentioned. Such information could assist educational authorises and institutions to better comprehend students’ difficulties and potentially improve their online learning experience. Additionally, most of these recent studies were limited to higher education, except for Yates et al. ( 2020 ) and Niemi and Kousa’s ( 2020 ) studies on senior high school students. Empirical research targeting the full spectrum of K‐12students remain scarce. Therefore, to address these gaps, the current paper reports the findings of a large‐scale study that sought to explore K‐12 students’ online learning experience during the COVID‐19 pandemic in a provincial sample of over one million Chinese students. The findings of this study provide policy recommendations to educational institutions and authorities regarding the delivery of K‐12 online education.

LITERATURE REVIEW

Learning conditions and technologies.

Having stable access to the internet is critical to students’ learning experience during online learning. Berge ( 2005 ) expressed the concern of the divide in digital‐readiness, and the pedagogical approach between different countries could influence students’ online learning experience. Digital‐readiness is the availability and adoption of information technologies and infrastructures in a country. Western countries like America (3rd) scored significantly higher in digital‐readiness compared to Asian countries like China (54th; Cisco,  2019 ). Students from low digital‐readiness countries could experience additional technology‐related problems. Supporting evidence is emerging in recent studies conducted during the COVID‐19 pandemic. In Egypt's capital city, Basuony et al. ( 2020 ) found that only around 13.9%of the students experienced issues with their internet connection. Whereas more than two‐thirds of the students in rural Indonesia reported issues of unstable internet, insufficient internet data, and incompatible learning device (Agung et al.,  2020 ).

Another influential factor for K‐12 students to adequately adapt to online learning is the accessibility of appropriate technological devices, especially having access to a desktop or a laptop (Barbour et al., 2018 ). However, it is unlikely for most of the students to satisfy this requirement. Even in higher education, around 76% of students reported having incompatible devices for online learning and only 15% of students used laptop for online learning, whereas around 85% of them used smartphone (Agung et al.,  2020 ). It is very likely that K‐12 students also suffer from this availability issue as they depend on their parents to provide access to relevant learning devices.

Technical issues surrounding technological devices could also influence students’ experience in online learning. (Barbour & Reeves,  2009 ) argues that students need to have a high level of digital literacy to find and use relevant information and communicate with others through technological devices. Students lacking this ability could experience difficulties in online learning. Bączek et al. ( 2021 ) found that around 54% of the medical students experienced technical problems with IT equipment and this issue was more prevalent in students with lower years of tertiary education. Likewise, Niemi and Kousa ( 2020 ) also find that students in a Finish high school experienced increased amounts of technical problems during the examination period, which involved additional technical applications. These findings are concerning as young children and adolescent in primary and lower secondary school could be more vulnerable to these technical problems as they are less experienced with the technologies in online learning (Barbour & LaBonte,  2017 ). Therefore, it is essential to investigate the learning conditions and the related difficulties experienced by students in K‐12 education as the extend of effects on them remain underexplored.

Learning experience and interactions

Apart from the aforementioned issues, the extent of interaction and collaborative learning opportunities available in online learning could also influence students’ experience. The literature on online learning has long emphasised the role of effective interaction for the success of student learning. According to Muirhead and Juwah ( 2004 ), interaction is an event that can take the shape of any type of communication between two or subjects and objects. Specifically, the literature acknowledges the three typical forms of interactions (Moore,  1989 ): (i) student‐content, (ii) student‐student, and (iii) student‐teacher. Anderson ( 2003 ) posits, in the well‐known interaction equivalency theorem, learning experiences will not deteriorate if only one of the three interaction is of high quality, and the other two can be reduced or even eliminated. Quality interaction can be accomplished by across two dimensions: (i) structure—pedagogical means that guide student interaction with contents or other students and (ii) dialogue—communication that happens between students and teachers and among students. To be able to scale online learning and prevent the growth of teaching costs, the emphasise is typically on structure (i.e., pedagogy) that can promote effective student‐content and student‐student interaction. The role of technology and media is typically recognised as a way to amplify the effect of pedagogy (Lou et al.,  2006 ). Novel technological innovations—for example learning analytics‐based personalised feedback at scale (Pardo et al.,  2019 ) —can also empower teachers to promote their interaction with students.

Online education can lead to a sense of isolation, which can be detrimental to student success (McInnerney & Roberts,  2004 ). Therefore, integration of social interaction into pedagogy for online learning is essential, especially at the times when students do not actually know each other or have communication and collaboration skills underdeveloped (Garrison et al.,  2010 ; Gašević et al.,  2015 ). Unfortunately, existing evidence suggested that online learning delivery during the COVID‐19 pandemic often lacks interactivity and collaborative experiences (Bączek et al.,  2021 ; Yates et al.,  2020 ). Bączek et al., ( 2021 ) found that around half of the medical students reported reduced interaction with teachers, and only 4% of students think online learning classes are interactive. Likewise, Yates et al. ( 2020 )’s study in high school students also revealed that over half of the students preferred in‐class collaboration over online collaboration as they value the immediate support and the proximity to teachers and peers from in‐class interaction.

Learning expectations and age differentiation

Although these studies have provided valuable insights and stressed the need for more interactivity in online learning, K‐12 students in different school years could exhibit different expectations for the desired activities in online learning. Piaget's Cognitive Developmental Theory illustrated children's difficulties in understanding abstract and hypothetical concepts (Thomas,  2000 ). Primary school students will encounter many abstract concepts in their STEM education (Uttal & Cohen,  2012 ). In face‐to‐face learning, teachers provide constant guidance on students’ learning progress and can help them to understand difficult concepts. Unfortunately, the level of guidance significantly drops in online learning, and, in most cases, children have to face learning obstacles by themselves (Barbour,  2013 ). Additionally, lower primary school students may lack the metacognitive skills to use various online learning functions, maintain engagement in synchronous online learning, develop and execute self‐regulated learning plans, and engage in meaningful peer interactions during online learning (Barbour,  2013 ; Broadbent & Poon,  2015 ; Huffaker & Calvert, 2003; Wang et al.,  2013 ). Thus, understanding these younger students’ expectations is imperative as delivering online learning to them in the same way as a virtual high school could hinder their learning experiences. For students with more matured metacognition, their expectations of online learning could be substantially different from younger students. Niemi et al.’s study ( 2020 ) with students in a Finish high school have found that students often reported heavy workload and fatigue during online learning. These issues could cause anxiety and reduce students’ learning motivation, which would have negative consequences on their emotional well‐being and academic performance (Niemi & Kousa,  2020 ; Yates et al.,  2020 ), especially for senior students who are under the pressure of examinations. Consequently, their expectations of online learning could be orientated toward having additional learning support functions and materials. Likewise, they could also prefer having more opportunities for peer interactions as these interactions are beneficial to their emotional well‐being and learning performance (Gašević et al., 2013 ; Montague & Rinaldi, 2001 ). Therefore, it is imperative to investigate the differences between online learning expectations in students of different school years to suit their needs better.

Research questions

By building upon the aforementioned relevant works, this study aimed to contribute to the online learning literature with a comprehensive understanding of the online learning experience that K‐12 students had during the COVID‐19 pandemic period in China. Additionally, this study also aimed to provide a thorough discussion of what potential actions can be undertaken to improve online learning delivery. Formally, this study was guided by three research questions (RQs):

RQ1 . What learning conditions were experienced by students across 12 years of education during their online learning process in the pandemic period? RQ2 . What benefits and obstacles were perceived by students across 12 years of education when performing online learning? RQ3 . What expectations do students, across 12 years of education, have for future online learning practices ?

Participants

The total number of K‐12 students in the Guangdong Province of China is around 15 million. In China, students of Year 1–6, Year 7–9, and Year 10–12 are referred to as students of primary school, middle school, and high school, respectively. Typically, students in China start their study in primary school at the age of around six. At the end of their high‐school study, students have to take the National College Entrance Examination (NCEE; also known as Gaokao) to apply for tertiary education. The survey was administrated across the whole Guangdong Province, that is the survey was exposed to all of the 15 million K‐12 students, though it was not mandatory for those students to accomplish the survey. A total of 1,170,769 students completed the survey, which accounts for a response rate of 7.80%. After removing responses with missing values and responses submitted from the same IP address (duplicates), we had 1,048,575 valid responses, which accounts to about 7% of the total K‐12 students in the Guangdong Province. The number of students in different school years is shown in Figure  1 . Overall, students were evenly distributed across different school years, except for a smaller sample in students of Year 10–12.

The number of students in each school year

Survey design

The survey was designed collaboratively by multiple relevant parties. Firstly, three educational researchers working in colleges and universities and three educational practitioners working in the Department of Education in Guangdong Province were recruited to co‐design the survey. Then, the initial draft of the survey was sent to 30 teachers from different primary and secondary schools, whose feedback and suggestions were considered to improve the survey. The final survey consisted of a total of 20 questions, which, broadly, can be classified into four categories: demographic, behaviours, experiences, and expectations. Details are available in Appendix.

All K‐12 students in the Guangdong Province were made to have full‐time online learning from March 1, 2020 after the outbreak of COVID‐19 in January in China. A province‐level online learning platform was provided to all schools by the government. In addition to the learning platform, these schools can also use additional third‐party platforms to facilitate the teaching activities, for example WeChat and Dingding, which provide services similar to WhatsApp and Zoom. The main change for most teachers was that they had to shift the classroom‐based lectures to online lectures with the aid of web‐conferencing tools. Similarly, these teachers also needed to perform homework marking and have consultation sessions in an online manner.

The Department of Education in the Guangdong Province of China distributed the survey to all K‐12 schools in the province on March 21, 2020 and collected responses on March 26, 2020. Students could access and answer the survey anonymously by either scan the Quick Response code along with the survey or click the survey address link on their mobile device. The survey was administrated in a completely voluntary manner and no incentives were given to the participants. Ethical approval was granted by the Department of Education in the Guangdong Province. Parental approval was not required since the survey was entirely anonymous and facilitated by the regulating authority, which satisfies China's ethical process.

The original survey was in Chinese, which was later translated by two bilingual researchers and verified by an external translator who is certified by the Australian National Accreditation Authority of Translators and Interpreters. The original and translated survey questionnaires are available in Supporting Information. Given the limited space we have here and the fact that not every survey item is relevant to the RQs, the following items were chosen to answer the RQs: item Q3 (learning media) and Q11 (learning approaches) for RQ1, item Q13 (perceived obstacle) and Q19 (perceived benefits) for RQ2, and item Q19 (expected learning activities) for RQ3. Cross‐tabulation based approaches were used to analyse the collected data. To scrutinise whether the differences displayed by students of different school years were statistically significant, we performed Chi‐square tests and calculated the Cramer's V to assess the strengths of the association after chi‐square had determined significance.

For the analyses, students were segmented into four categories based on their school years, that is Year 1–3, Year 4–6, Year 7–9, and Year 10–12, to provide a clear understanding of the different experiences and needs that different students had for online learning. This segmentation was based on the educational structure of Chinese schools: elementary school (Year 1–6), middle school (Year 7–9), and high school (Year 10–12). Children in elementary school can further be segmented into junior (Year 1–3) or senior (Year 4–6) students because senior elementary students in China are facing more workloads compared to junior students due to the provincial Middle School Entry Examination at the end of Year 6.

Learning conditions—RQ1

Learning media.

The Chi‐square test showed significant association between school years and students’ reported usage of learning media, χ 2 (55, N  = 1,853,952) = 46,675.38, p  < 0.001. The Cramer's V is 0.07 ( df ∗ = 5), which indicates a small‐to‐medium effect according to Cohen’s ( 1988 ) guidelines. Based on Figure  2 , we observed that an average of up to 87.39% students used smartphones to perform online learning, while only 25.43% students used computer, which suggests that smartphones, with widespread availability in China (2020), have been adopted by students for online learning. As for the prevalence of the two media, we noticed that both smartphones ( χ 2 (3, N  = 1,048,575) = 9,395.05, p < 0.001, Cramer's V  = 0.10 ( df ∗ = 1)) and computers ( χ 2 (3, N  = 1,048,575) = 11,025.58, p <.001, Cramer's V  = 0.10 ( df ∗ = 1)) were more adopted by high‐school‐year (Year 7–12) than early‐school‐year students (Year 1–6), both with a small effect size. Besides, apparent discrepancies can be observed between the usages of TV and paper‐based materials across different school years, that is early‐school‐year students reported more TV usage ( χ 2 (3, N  = 1,048,575) = 19,505.08, p <.001), with a small‐to‐medium effect size, Cramer's V  = 0.14( df ∗ = 1). High‐school‐year students (especially Year 10–12) reported more usage of paper‐based materials ( χ 2 (3, N  = 1,048,575) = 23,401.64, p < 0.001), with a small‐to‐medium effect size, Cramer's V  = 0.15( df ∗ = 1).

Learning media used by students in online learning

Learning approaches

School years is also significantly associated with the different learning approaches students used to tackle difficult concepts during online learning, χ 2 (55, N  = 2,383,751) = 58,030.74, p < 0.001. The strength of this association is weak to moderate as shown by the Cramer's V (0.07, df ∗ = 5; Cohen,  1988 ). When encountering problems related to difficult concepts, students typically chose to “solve independently by searching online” or “rewatch recorded lectures” instead of consulting to their teachers or peers (Figure  3 ). This is probably because, compared to classroom‐based education, it is relatively less convenient and more challenging for students to seek help from others when performing online learning. Besides, compared to high‐school‐year students, early‐school‐year students (Year 1–6), reported much less use of “solve independently by searching online” ( χ 2 (3, N  = 1,048,575) = 48,100.15, p <.001), with a small‐to‐medium effect size, Cramer's V  = 0.21 ( df ∗ = 1). Also, among those approaches of seeking help from others, significantly more high‐school‐year students preferred “communicating with other students” than early‐school‐year students ( χ 2 (3, N  = 1,048,575) = 81,723.37, p < 0.001), with a medium effect size, Cramer's V  = 0.28 ( df ∗ = 1).

Learning approaches used by students in online learning

Perceived benefits and obstacles—RQ2

Perceived benefits.

The association between school years and perceived benefits in online learning is statistically significant, χ 2 (66, N  = 2,716,127) = 29,534.23, p  < 0.001, and the Cramer's V (0.04, df ∗ = 6) indicates a small effect (Cohen,  1988 ). Unsurprisingly, benefits brought by the convenience of online learning are widely recognised by students across all school years (Figure  4 ), that is up to 75% of students reported that it is “more convenient to review course content” and 54% said that they “can learn anytime and anywhere” . Besides, we noticed that about 50% of early‐school‐year students appreciated the “access to courses delivered by famous teachers” and 40%–47% of high‐school‐year students indicated that online learning is “helpful to develop self‐regulation and autonomy” .

Perceived benefits of online learning reported by students

Perceived obstacles

The Chi‐square test shows a significant association between school years and students’ perceived obstacles in online learning, χ 2 (77, N  = 2,699,003) = 31,987.56, p < 0.001. This association is relatively weak as shown by the Cramer's V (0.04, df ∗ = 7; Cohen,  1988 ). As shown in Figure  5 , the biggest obstacles encountered by up to 73% of students were the “eyestrain caused by long staring at screens” . Disengagement caused by nearby disturbance was reported by around 40% of students, especially those of Year 1–3 and 10–12. Technological‐wise, about 50% of students experienced poor Internet connection during their learning process, and around 20% of students reported the “confusion in setting up the platforms” across of school years.

Perceived obstacles of online learning reported by students

Expectations for future practices of online learning – RQ3

Online learning activities.

The association between school years and students’ expected online learning activities is significant, χ 2 (66, N  = 2,416,093) = 38,784.81, p < 0.001. The Cramer's V is 0.05 ( df ∗ = 6) which suggests a small effect (Cohen,  1988 ). As shown in Figure  6 , the most expected activity for future online learning is “real‐time interaction with teachers” (55%), followed by “online group discussion and collaboration” (38%). We also observed that more early‐school‐year students expect reflective activities, such as “regular online practice examinations” ( χ 2 (3, N  = 1,048,575) = 11,644.98, p < 0.001), with a small effect size, Cramer's V  = 0.11 ( df ∗ = 1). In contrast, more high‐school‐year students expect “intelligent recommendation system …” ( χ 2 (3, N  = 1,048,575) = 15,327.00, p < 0.001), with a small effect size, Cramer's V  = 0.12 ( df ∗ = 1).

Students’ expected online learning activities

Regarding students’ learning conditions, substantial differences were observed in learning media, family dependency, and learning approaches adopted in online learning between students in different school years. The finding of more computer and smartphone usage in high‐school‐year than early‐school‐year students can probably be explained by that, with the growing abilities in utilising these media as well as the educational systems and tools which run on these media, high‐school‐year students tend to make better use of these media for online learning practices. Whereas, the differences in paper‐based materials may imply that high‐school‐year students in China have to accomplish a substantial amount of exercise, assignments, and exam papers to prepare for the National College Entrance Examination (NCEE), whose delivery was not entirely digitised due to the sudden transition to online learning. Meanwhile, high‐school‐year students may also have preferred using paper‐based materials for exam practice, as eventually, they would take their NCEE in the paper format. Therefore, these substantial differences in students’ usage of learning media should be addressed by customising the delivery method of online learning for different school years.

Other than these between‐age differences in learning media, the prevalence of smartphone in online learning resonates with Agung et al.’s ( 2020 ) finding on the issues surrounding the availability of compatible learning device. The prevalence of smartphone in K‐12 students is potentially problematic as the majority of the online learning platform and content is designed for computer‐based learning (Berge,  2005 ; Molnar et al.,  2019 ). Whereas learning with smartphones has its own unique challenges. For example, Gikas and Grant ( 2013 ) discovered that students who learn with smartphone experienced frustration with the small screen‐size, especially when trying to type with the tiny keypad. Another challenge relates to the distraction of various social media applications. Although similar distractions exist in computer and web‐based social media, the level of popularity, especially in the young generation, are much higher in mobile‐based social media (Montag et al.,  2018 ). In particular, the message notification function in smartphones could disengage students from learning activities and allure them to social media applications (Gikas & Grant,  2013 ). Given these challenges of learning with smartphones, more research efforts should be devoted to analysing students’ online learning behaviour in the setting of mobile learning to accommodate their needs better.

The differences in learning approaches, once again, illustrated that early‐school‐year students have different needs compared to high‐school‐year students. In particular, the low usage of the independent learning methods in early‐school‐year students may reflect their inability to engage in independent learning. Besides, the differences in help seeking behaviours demonstrated the distinctive needs for communication and interaction between different students, that is early‐school‐year students have a strong reliance on teachers and high‐school‐year students, who are equipped with stronger communication ability, are more inclined to interact with their peers. This finding implies that the design of online learning platforms should take students’ different needs into account. Thus, customisation is urgently needed for the delivery of online learning to different school years.

In terms of the perceived benefits and challenges of online learning, our results resonate with several previous findings. In particular, the benefits of convenience are in line with the flexibility advantages of online learning, which were mentioned in prior works (Appana,  2008 ; Bączek et al.,  2021 ; Barbour,  2013 ; Basuony et al.,  2020 ; Harvey et al.,  2014 ). Early‐school‐year students’ higher appreciation in having “access to courses delivered by famous teachers” and lower appreciation in the independent learning skills developed through online learning are also in line with previous literature (Barbour,  2013 ; Harvey et al.,  2014 ; Oliver et al.,  2009 ). Again, these similar findings may indicate the strong reliance that early‐school‐year students place on teachers, while high‐school‐year students are more capable of adapting to online learning by developing independent learning skills.

Technology‐wise, students’ experience of poor internet connection and confusion in setting up online learning platforms are particularly concerning. The problem of poor internet connection corroborated the findings reported in prior studies (Agung et al.,  2020 ; Barbour,  2013 ; Basuony et al.,  2020 ; Berge,  2005 ; Rice,  2006 ), that is the access issue surrounded the digital divide as one of the main challenges of online learning. In the era of 4G and 5G networks, educational authorities and institutions that deliver online education could fall into the misconception of most students have a stable internet connection at home. The internet issue we observed is particularly vital to students’ online learning experience as most students prefer real‐time communications (Figure  6 ), which rely heavily on stable internet connection. Likewise, the finding of students’ confusion in technology is also consistent with prior studies (Bączek et al.,  2021 ; Muilenburg & Berge,  2005 ; Niemi & Kousa,  2020 ; Song et al.,  2004 ). Students who were unsuccessfully in setting up the online learning platforms could potentially experience declines in confidence and enthusiasm for online learning, which would cause a subsequent unpleasant learning experience. Therefore, both the readiness of internet infrastructure and student technical skills remain as the significant challenges for the mass‐adoption of online learning.

On the other hand, students’ experience of eyestrain from extended screen time provided empirical evidence to support Spitzer’s ( 2001 ) speculation about the potential ergonomic impact of online learning. This negative effect is potentially related to the prevalence of smartphone device and the limited screen size of these devices. This finding not only demonstrates the potential ergonomic issues that would be caused by smartphone‐based online learning but also resonates with the aforementioned necessity of different platforms and content designs for different students.

A less‐mentioned problem in previous studies on online learning experiences is the disengagement caused by nearby disturbance, especially in Year 1–3 and 10–12. It is likely that early‐school‐year students suffered from this problem because of their underdeveloped metacognitive skills to concentrate on online learning without teachers’ guidance. As for high‐school‐year students, the reasons behind their disengagement require further investigation in the future. Especially it would be worthwhile to scrutinise whether this type of disengagement is caused by the substantial amount of coursework they have to undertake and the subsequent a higher level of pressure and a lower level of concentration while learning.

Across age‐level differences are also apparent in terms of students’ expectations of online learning. Although, our results demonstrated students’ needs of gaining social interaction with others during online learning, findings (Bączek et al.,  2021 ; Harvey et al.,  2014 ; Kuo et al.,  2014 ; Liu & Cavanaugh,  2012 ; Yates et al.,  2020 ). This need manifested differently across school years, with early‐school‐year students preferring more teacher interactions and learning regulation support. Once again, this finding may imply that early‐school‐year students are inadequate in engaging with online learning without proper guidance from their teachers. Whereas, high‐school‐year students prefer more peer interactions and recommendation to learning resources. This expectation can probably be explained by the large amount of coursework exposed to them. Thus, high‐school‐year students need further guidance to help them better direct their learning efforts. These differences in students’ expectations for future practices could guide the customisation of online learning delivery.

Implications

As shown in our results, improving the delivery of online learning not only requires the efforts of policymakers but also depend on the actions of teachers and parents. The following sub‐sections will provide recommendations for relevant stakeholders and discuss their essential roles in supporting online education.

Technical support

The majority of the students has experienced technical problems during online learning, including the internet lagging and confusion in setting up the learning platforms. These problems with technology could impair students’ learning experience (Kauffman,  2015 ; Muilenburg & Berge,  2005 ). Educational authorities and schools should always provide a thorough guide and assistance for students who are experiencing technical problems with online learning platforms or other related tools. Early screening and detection could also assist schools and teachers to direct their efforts more effectively in helping students with low technology skills (Wilkinson et al.,  2010 ). A potential identification method involves distributing age‐specific surveys that assess students’ Information and Communication Technology (ICT) skills at the beginning of online learning. For example, there are empirical validated ICT surveys available for both primary (Aesaert et al.,  2014 ) and high school (Claro et al.,  2012 ) students.

For students who had problems with internet lagging, the delivery of online learning should provide options that require fewer data and bandwidth. Lecture recording is the existing option but fails to address students’ need for real‐time interaction (Clark et al.,  2015 ; Malik & Fatima,  2017 ). A potential alternative involves providing students with the option to learn with digital or physical textbooks and audio‐conferencing, instead of screen sharing and video‐conferencing. This approach significantly reduces the amount of data usage and lowers the requirement of bandwidth for students to engage in smooth online interactions (Cisco,  2018 ). It also requires little additional efforts from teachers as official textbooks are often available for each school year, and thus, they only need to guide students through the materials during audio‐conferencing. Educational authority can further support this approach by making digital textbooks available for teachers and students, especially those in financial hardship. However, the lack of visual and instructor presence could potentially reduce students’ attention, recall of information, and satisfaction in online learning (Wang & Antonenko,  2017 ). Therefore, further research is required to understand whether the combination of digital or physical textbooks and audio‐conferencing is appropriate for students with internet problems. Alternatively, suppose the local technological infrastructure is well developed. In that case, governments and schools can also collaborate with internet providers to issue data and bandwidth vouchers for students who are experiencing internet problems due to financial hardship.

For future adoption of online learning, policymakers should consider the readiness of the local internet infrastructure. This recommendation is particularly important for developing countries, like Bangladesh, where the majority of the students reported the lack of internet infrastructure (Ramij & Sultana,  2020 ). In such environments, online education may become infeasible, and alternative delivery method could be more appropriate, for example, the Telesecundaria program provides TV education for rural areas of Mexico (Calderoni,  1998 ).

Other than technical problems, choosing a suitable online learning platform is also vital for providing students with a better learning experience. Governments and schools should choose an online learning platform that is customised for smartphone‐based learning, as the majority of students could be using smartphones for online learning. This recommendation is highly relevant for situations where students are forced or involuntarily engaged in online learning, like during the COVID‐19 pandemic, as they might not have access to a personal computer (Molnar et al.,  2019 ).

Customisation of delivery methods

Customising the delivery of online learning for students in different school years is the theme that appeared consistently across our findings. This customisation process is vital for making online learning an opportunity for students to develop independent learning skills, which could help prepare them for tertiary education and lifelong learning. However, the pedagogical design of K‐12 online learning programs should be differentiated from adult‐orientated programs as these programs are designed for independent learners, which is rarely the case for students in K‐12 education (Barbour & Reeves,  2009 ).

For early‐school‐year students, especially Year 1–3 students, providing them with sufficient guidance from both teachers and parents should be the priority as these students often lack the ability to monitor and reflect on learning progress. In particular, these students would prefer more real‐time interaction with teachers, tutoring from parents, and regular online practice examinations. These forms of guidance could help early‐school‐year students to cope with involuntary online learning, and potentially enhance their experience in future online learning. It should be noted that, early‐school‐year students demonstrated interest in intelligent monitoring and feedback systems for learning. Additional research is required to understand whether these young children are capable of understanding and using learning analytics that relay information on their learning progress. Similarly, future research should also investigate whether young children can communicate effectively through digital tools as potential inability could hinder student learning in online group activities. Therefore, the design of online learning for early‐school‐year students should focus less on independent learning but ensuring that students are learning effective under the guidance of teachers and parents.

In contrast, group learning and peer interaction are essential for older children and adolescents. The delivery of online learning for these students should focus on providing them with more opportunities to communicate with each other and engage in collaborative learning. Potential methods to achieve this goal involve assigning or encouraging students to form study groups (Lee et al.,  2011 ), directing students to use social media for peer communication (Dabbagh & Kitsantas,  2012 ), and providing students with online group assignments (Bickle & Rucker,  2018 ).

Special attention should be paid to students enrolled in high schools. For high‐school‐year students, in particular, students in Year 10–12, we also recommend to provide them with sufficient access to paper‐based learning materials, such as revision booklet and practice exam papers, so they remain familiar with paper‐based examinations. This recommendation applies to any students who engage in online learning but has to take their final examination in paper format. It is also imperative to assist high‐school‐year students who are facing examinations to direct their learning efforts better. Teachers can fulfil this need by sharing useful learning resources on the learning management system, if it is available, or through social media groups. Alternatively, students are interested in intelligent recommendation systems for learning resources, which are emerging in the literature (Corbi & Solans,  2014 ; Shishehchi et al.,  2010 ). These systems could provide personalised recommendations based on a series of evaluation on learners’ knowledge. Although it is infeasible for situations where the transformation to online learning happened rapidly (i.e., during the COVID‐19 pandemic), policymakers can consider embedding such systems in future online education.

Limitations

The current findings are limited to primary and secondary Chinese students who were involuntarily engaged in online learning during the COVID‐19 pandemic. Despite the large sample size, the population may not be representative as participants are all from a single province. Also, information about the quality of online learning platforms, teaching contents, and pedagogy approaches were missing because of the large scale of our study. It is likely that the infrastructures of online learning in China, such as learning platforms, instructional designs, and teachers’ knowledge about online pedagogy, were underprepared for the sudden transition. Thus, our findings may not represent the experience of students who voluntarily participated in well‐prepared online learning programs, in particular, the virtual school programs in America and Canada (Barbour & LaBonte,  2017 ; Molnar et al.,  2019 ). Lastly, the survey was only evaluated and validated by teachers but not students. Therefore, students with the lowest reading comprehension levels might have a different understanding of the items’ meaning, especially terminologies that involve abstract contracts like self‐regulation and autonomy in item Q17.

In conclusion, we identified across‐year differences between primary and secondary school students’ online learning experience during the COVID‐19 pandemic. Several recommendations were made for the future practice and research of online learning in the K‐12 student population. First, educational authorities and schools should provide sufficient technical support to help students to overcome potential internet and technical problems, as well as choosing online learning platforms that have been customised for smartphones. Second, customising the online pedagogy design for students in different school years, in particular, focusing on providing sufficient guidance for young children, more online collaborative opportunity for older children and adolescent, and additional learning resource for senior students who are facing final examinations.

CONFLICT OF INTEREST

There is no potential conflict of interest in this study.

ETHICS STATEMENT

The data are collected by the Department of Education of the Guangdong Province who also has the authority to approve research studies in K12 education in the province.

Supporting information

Supplementary Material

ACKNOWLEDGEMENTS

This work is supported by the National Natural Science Foundation of China (62077028, 61877029), the Science and Technology Planning Project of Guangdong (2020B0909030005, 2020B1212030003, 2020ZDZX3013, 2019B1515120010, 2018KTSCX016, 2019A050510024), the Science and Technology Planning Project of Guangzhou (201902010041), and the Fundamental Research Funds for the Central Universities (21617408, 21619404).

SURVEY ITEMS

Yan, L , Whitelock‐Wainwright, A , Guan, Q , Wen, G , Gašević, D , & Chen, G . Students’ experience of online learning during the COVID‐19 pandemic: A province‐wide survey study . Br J Educ Technol . 2021; 52 :2038–2057. 10.1111/bjet.13102 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]

DATA AVAILABILITY STATEMENT

• Aesaert, K. , Van Nijlen, D. , Vanderlinde, R. , & van Braak, J. (2014). Direct measures of digital information processing and communication skills in primary education: Using item response theory for the development and validation of an ICT competence scale . Computers & Education , 76 , 168–181. 10.1016/j.compedu.2014.03.013 [ CrossRef ] [ Google Scholar ]
• Agung, A. S. N. , Surtikanti, M. W. , & Quinones, C. A. (2020). Students’ perception of online learning during COVID‐19 pandemic: A case study on the English students of STKIP Pamane Talino . SOSHUM: Jurnal Sosial Dan Humaniora , 10 ( 2 ), 225–235. 10.31940/soshum.v10i2.1316 [ CrossRef ] [ Google Scholar ]
• Anderson, T. (2003). Getting the mix right again: An updated and theoretical rationale for interaction . The International Review of Research in Open and Distributed Learning , 4 ( 2 ). 10.19173/irrodl.v4i2.149 [ CrossRef ] [ Google Scholar ]
• Appana, S. (2008). A review of benefits and limitations of online learning in the context of the student, the instructor and the tenured faculty . International Journal on E‐learning , 7 ( 1 ), 5–22. [ Google Scholar ]
• Bączek, M. , Zagańczyk‐Bączek, M. , Szpringer, M. , Jaroszyński, A. , & Wożakowska‐Kapłon, B. (2021). Students’ perception of online learning during the COVID‐19 pandemic: A survey study of Polish medical students . Medicine , 100 ( 7 ), e24821. 10.1097/MD.0000000000024821 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Barbour, M. K. (2013). The landscape of k‐12 online learning: Examining what is known . Handbook of Distance Education , 3 , 574–593. [ Google Scholar ]
• Barbour, M. , Huerta, L. , & Miron, G. (2018). Virtual schools in the US: Case studies of policy, performance and research evidence. In Society for information technology & teacher education international conference (pp. 672–677). Association for the Advancement of Computing in Education (AACE). [ Google Scholar ]
• Barbour, M. K. , & LaBonte, R. (2017). State of the nation: K‐12 e‐learning in Canada, 2017 edition . http://k12sotn.ca/wp‐content/uploads/2018/02/StateNation17.pdf [ Google Scholar ]
• Barbour, M. K. , & Reeves, T. C. (2009). The reality of virtual schools: A review of the literature . Computers & Education , 52 ( 2 ), 402–416. [ Google Scholar ]
• Basuony, M. A. K. , EmadEldeen, R. , Farghaly, M. , El‐Bassiouny, N. , & Mohamed, E. K. A. (2020). The factors affecting student satisfaction with online education during the COVID‐19 pandemic: An empirical study of an emerging Muslim country . Journal of Islamic Marketing . 10.1108/JIMA-09-2020-0301 [ CrossRef ] [ Google Scholar ]
• Berge, Z. L. (2005). Virtual schools: Planning for success . Teachers College Press, Columbia University. [ Google Scholar ]
• Bickle, M. C. , & Rucker, R. (2018). Student‐to‐student interaction: Humanizing the online classroom using technology and group assignments . Quarterly Review of Distance Education , 19 ( 1 ), 1–56. [ Google Scholar ]
• Broadbent, J. , & Poon, W. L. (2015). Self‐regulated learning strategies & academic achievement in online higher education learning environments: A systematic review . The Internet and Higher Education , 27 , 1–13. [ Google Scholar ]
• Calderoni, J. (1998). Telesecundaria: Using TV to bring education to rural Mexico (Tech. Rep.). The World Bank. [ Google Scholar ]
• Cisco . (2018). Bandwidth requirements for meetings with cisco Webex and collaboration meeting rooms white paper . http://dwz.date/dpbc [ Google Scholar ]
• Cisco . (2019). Cisco digital readiness 2019 . https://www.cisco.com/c/m/en_us/about/corporate‐social‐responsibility/research‐resources/digital‐readiness‐index.html#/ (Library Catalog: www.cisco.com). [ Google Scholar ]
• Clark, C. , Strudler, N. , & Grove, K. (2015). Comparing asynchronous and synchronous video vs. text based discussions in an online teacher education course . Online Learning , 19 ( 3 ), 48–69. [ Google Scholar ]
• Claro, M. , Preiss, D. D. , San Martín, E. , Jara, I. , Hinostroza, J. E. , Valenzuela, S. , Cortes, F. , & Nussbaum, M. (2012). Assessment of 21st century ICT skills in Chile: Test design and results from high school level students . Computers & Education , 59 ( 3 ), 1042–1053. 10.1016/j.compedu.2012.04.004 [ CrossRef ] [ Google Scholar ]
• Cohen, J. (1988). Statistical power analysis for the behavioral sciences . Routledge Academic. [ Google Scholar ]
• Corbi, A. , & Solans, D. B. (2014). Review of current student‐monitoring techniques used in elearning‐focused recommender systems and learning analytics: The experience API & LIME model case study . IJIMAI , 2 ( 7 ), 44–52. [ Google Scholar ]
• Dabbagh, N. , & Kitsantas, A. (2012). Personal learning environments, social media, and self‐regulated learning: A natural formula for connecting formal and informal learning . The Internet and Higher Education , 15 ( 1 ), 3–8. 10.1016/j.iheduc.2011.06.002 [ CrossRef ] [ Google Scholar ]
• Garrison, D. R. , Cleveland‐Innes, M. , & Fung, T. S. (2010). Exploring causal relationships among teaching, cognitive and social presence: Student perceptions of the community of inquiry framework . The Internet and Higher Education , 13 ( 1–2 ), 31–36. 10.1016/j.iheduc.2009.10.002 [ CrossRef ] [ Google Scholar ]
• Gašević, D. , Adesope, O. , Joksimović, S. , & Kovanović, V. (2015). Externally‐facilitated regulation scaffolding and role assignment to develop cognitive presence in asynchronous online discussions . The Internet and Higher Education , 24 , 53–65. 10.1016/j.iheduc.2014.09.006 [ CrossRef ] [ Google Scholar ]
• Gašević, D. , Zouaq, A. , & Janzen, R. (2013). “Choose your classmates, your GPA is at stake!” The association of cross‐class social ties and academic performance . American Behavioral Scientist , 57 ( 10 ), 1460–1479. [ Google Scholar ]
• Gikas, J. , & Grant, M. M. (2013). Mobile computing devices in higher education: Student perspectives on learning with cellphones, smartphones & social media . The Internet and Higher Education , 19 , 18–26. [ Google Scholar ]
• Harvey, D. , Greer, D. , Basham, J. , & Hu, B. (2014). From the student perspective: Experiences of middle and high school students in online learning . American Journal of Distance Education , 28 ( 1 ), 14–26. 10.1080/08923647.2014.868739 [ CrossRef ] [ Google Scholar ]
• Kauffman, H. (2015). A review of predictive factors of student success in and satisfaction with online learning . Research in Learning Technology , 23 . 10.3402/rlt.v23.26507 [ CrossRef ] [ Google Scholar ]
• Kuo, Y.‐C. , Walker, A. E. , Belland, B. R. , Schroder, K. E. , & Kuo, Y.‐T. (2014). A case study of integrating interwise: Interaction, internet self‐efficacy, and satisfaction in synchronous online learning environments . International Review of Research in Open and Distributed Learning , 15 ( 1 ), 161–181. 10.19173/irrodl.v15i1.1664 [ CrossRef ] [ Google Scholar ]
• Lee, S. J. , Srinivasan, S. , Trail, T. , Lewis, D. , & Lopez, S. (2011). Examining the relationship among student perception of support, course satisfaction, and learning outcomes in online learning . The Internet and Higher Education , 14 ( 3 ), 158–163. 10.1016/j.iheduc.2011.04.001 [ CrossRef ] [ Google Scholar ]
• Liu, F. , & Cavanaugh, C. (2012). Factors influencing student academic performance in online high school algebra . Open Learning: The Journal of Open, Distance and e‐Learning , 27 ( 2 ), 149–167. 10.1080/02680513.2012.678613 [ CrossRef ] [ Google Scholar ]
• Lou, Y. , Bernard, R. M. , & Abrami, P. C. (2006). Media and pedagogy in undergraduate distance education: A theory‐based meta‐analysis of empirical literature . Educational Technology Research and Development , 54 ( 2 ), 141–176. 10.1007/s11423-006-8252-x [ CrossRef ] [ Google Scholar ]
• Malik, M. , & Fatima, G. (2017). E‐learning: Students’ perspectives about asynchronous and synchronous resources at higher education level . Bulletin of Education and Research , 39 ( 2 ), 183–195. [ Google Scholar ]
• McInnerney, J. M. , & Roberts, T. S. (2004). Online learning: Social interaction and the creation of a sense of community . Journal of Educational Technology & Society , 7 ( 3 ), 73–81. [ Google Scholar ]
• Molnar, A. , Miron, G. , Elgeberi, N. , Barbour, M. K. , Huerta, L. , Shafer, S. R. , & Rice, J. K. (2019). Virtual schools in the US 2019 . National Education Policy Center. [ Google Scholar ]
• Montague, M. , & Rinaldi, C. (2001). Classroom dynamics and children at risk: A followup . Learning Disability Quarterly , 24 ( 2 ), 75–83. [ Google Scholar ]
• Montag, C. , Becker, B. , & Gan, C. (2018). The multipurpose application Wechat: A review on recent research . Frontiers in Psychology , 9 , 2247. 10.3389/fpsyg.2018.02247 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Moore, M. G. (1989). Editorial: Three types of interaction . American Journal of Distance Education , 3 ( 2 ), 1–7. 10.1080/08923648909526659 [ CrossRef ] [ Google Scholar ]
• Muilenburg, L. Y. , & Berge, Z. L. (2005). Student barriers to online learning: A factor analytic study . Distance Education , 26 ( 1 ), 29–48. 10.1080/01587910500081269 [ CrossRef ] [ Google Scholar ]
• Muirhead, B. , & Juwah, C. (2004). Interactivity in computer‐mediated college and university education: A recent review of the literature . Journal of Educational Technology & Society , 7 ( 1 ), 12–20. [ Google Scholar ]
• Niemi, H. M. , & Kousa, P. (2020). A case study of students’ and teachers’ perceptions in a finnish high school during the COVID pandemic . International Journal of Technology in Education and Science , 4 ( 4 ), 352–369. 10.46328/ijtes.v4i4.167 [ CrossRef ] [ Google Scholar ]
• Oliver, K. , Osborne, J. , & Brady, K. (2009). What are secondary students’ expectations for teachers in virtual school environments? Distance Education , 30 ( 1 ), 23–45. 10.1080/01587910902845923 [ CrossRef ] [ Google Scholar ]
• Pardo, A. , Jovanovic, J. , Dawson, S. , Gašević, D. , & Mirriahi, N. (2019). Using learning analytics to scale the provision of personalised feedback . British Journal of Educational Technology , 50 ( 1 ), 128–138. 10.1111/bjet.12592 [ CrossRef ] [ Google Scholar ]
• Ramij, M. , & Sultana, A. (2020). Preparedness of online classes in developing countries amid covid‐19 outbreak: A perspective from Bangladesh. Afrin, Preparedness of Online Classes in Developing Countries amid COVID‐19 Outbreak: A Perspective from Bangladesh (June 29, 2020) .
• Rice, K. L. (2006). A comprehensive look at distance education in the k–12 context . Journal of Research on Technology in Education , 38 ( 4 ), 425–448. 10.1080/15391523.2006.10782468 [ CrossRef ] [ Google Scholar ]
• Shishehchi, S. , Banihashem, S. Y. , & Zin, N. A. M. (2010). A proposed semantic recommendation system for elearning: A rule and ontology based e‐learning recommendation system. In 2010 international symposium on information technology (Vol. 1, pp. 1–5).
• Song, L. , Singleton, E. S. , Hill, J. R. , & Koh, M. H. (2004). Improving online learning: Student perceptions of useful and challenging characteristics . The Internet and Higher Education , 7 ( 1 ), 59–70. 10.1016/j.iheduc.2003.11.003 [ CrossRef ] [ Google Scholar ]
• Spitzer, D. R. (2001). Don’t forget the high‐touch with the high‐tech in distance learning . Educational Technology , 41 ( 2 ), 51–55. [ Google Scholar ]
• Thomas, R. M. (2000). Comparing theories of child development. Wadsworth/Thomson Learning. United Nations Educational, Scientific and Cultural Organization. (2020, March). Education: From disruption to recovery . https://en.unesco.org/covid19/educationresponse (Library Catalog: en.unesco.org)
• Uttal, D. H. , & Cohen, C. A. (2012). Spatial thinking and stem education: When, why, and how? In Psychology of learning and motivation (Vol. 57 , pp. 147–181). Elsevier. [ Google Scholar ]
• Van Lancker, W. , & Parolin, Z. (2020). Covid‐19, school closures, and child poverty: A social crisis in the making . The Lancet Public Health , 5 ( 5 ), e243–e244. 10.1016/S2468-2667(20)30084-0 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wang, C.‐H. , Shannon, D. M. , & Ross, M. E. (2013). Students’ characteristics, self‐regulated learning, technology self‐efficacy, and course outcomes in online learning . Distance Education , 34 ( 3 ), 302–323. 10.1080/01587919.2013.835779 [ CrossRef ] [ Google Scholar ]
• Wang, J. , & Antonenko, P. D. (2017). Instructor presence in instructional video: Effects on visual attention, recall, and perceived learning . Computers in Human Behavior , 71 , 79–89. 10.1016/j.chb.2017.01.049 [ CrossRef ] [ Google Scholar ]
• Wilkinson, A. , Roberts, J. , & While, A. E. (2010). Construction of an instrument to measure student information and communication technology skills, experience and attitudes to e‐learning . Computers in Human Behavior , 26 ( 6 ), 1369–1376. 10.1016/j.chb.2010.04.010 [ CrossRef ] [ Google Scholar ]
• World Health Organization . (2020, July). Coronavirus disease 2019 (COVID‐19): Situation Report‐164 (Situation Report No. 164). https://www.who.int/docs/default‐source/coronaviruse/situation‐reports/20200702‐covid‐19‐sitrep‐164.pdf?sfvrsn$=$ac074f58$_$2
• Yates, A. , Starkey, L. , Egerton, B. , & Flueggen, F. (2020). High school students’ experience of online learning during Covid‐19: The influence of technology and pedagogy . Technology, Pedagogy and Education , 9 , 1–15. 10.1080/1475939X.2020.1854337 [ CrossRef ] [ Google Scholar ]

• © 2024

Online Education During COVID-19 and Beyond

Opportunities, Challenges and Outlook

• Silvia Puiu 0 ,
• Samuel O. Idowu 1

Faculty of Economics and Business Administration, University of Craiova, Craiova, Romania

You can also search for this editor in PubMed   Google Scholar

Guildhall School of Business and Law, London Metropolitan University, London, UK

• Gives an outlook on possible future scenarios in the field of education on a global level
• Discusses sustainable solutions for coping with online education
• Explores the challenges of managing online courses from different perspectives

Part of the book series: CSR, Sustainability, Ethics & Governance (CSEG)

• Table of contents

About this book

Editors and affiliations, about the editors, bibliographic information.

• Publish with us

Buying options

• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Other ways to access

This is a preview of subscription content, log in via an institution to check for access.

Table of contents (22 chapters)

Front matter, online education during covid-19 and beyond: an introduction.

• Silvia Puiu, Samuel O. Idowu

Online Education During COVID-19 and Beyond in Europe

Education in intergenerational family businesses at the university of ruse “angel kanchev” before, during and after the covid-19 restrictions.

• Daniel Pavlov

When a Phenomenon-Based University Course Went Online: Students’ Experiences and Reflections After Sauna Bathing

• Lili-Ann Wolff, Emma Heikkilä, Kirsi Wallinheimo, Wilhelm Blomberg

Online Education During COVID-19 and Beyond: Opportunities, Challenges, and Its Future—The Greek Perspective

• Panagiotopoulos Φ. Ioannis, Arapitsa V. Evdokia

COVID-19 Pandemic, New Technologies and Relational Goods Challenges in Higher Education: Are We Closer (to Sustainability) or Further Away? Insights from Italy

• Maria-Gabriella Baldarelli, Mara Del Baldo

Online Education in Poland During the Covid-19 Pandemic

• Anna Cierniak-Emerych

Soft Skills Formed Through Online Education—Are They Sufficiently Developed for Economics Graduates’ Integration into the Labour Market?

• Suzana Demyen, Adrian-Costinel Tănase, Florența-Diana Tănase

Online Economic Higher Education in a Multidisciplinary University in Romania—Challenges During the COVID-19 Pandemic and the Consequences of Returning to Traditional Face-To-Face Interaction

• Mariana Juganaru, Ion-Danut Juganaru, Andreea-Daniela Moraru

Stem Students’ Progress in Two Romanian and United States of America Universities Before, During, and Beyond the COVID-19 Pandemic

• Mihaela Tinca Udristioiu, Dragos Amarie

Shifting from Physical to Virtual Classroom in Accounting Education: A Study of Students’ Experiences During the COVID-19 Pandemic in Serbia

• Aleksandra Fedajev, Dejan Jovanović, Milica Veličković

From Distance to Distance: On How a Master’s Programme in Organisation and Leadership Went Through Pandemic Change

• Magnus Frostenson, Mats Persson, Sol Skinnarland

Online Education in Higher Education Institutions During the Covid-19 Pandemic: The Case of Türkiye

• Gizem Aras Beger

Online Education During COVID-19 and Beyond in Africa

Online higher education in ghana through covid-19 and post-covid-19: experiences and perceptions of students and academic staff.

• Gabriel Botchwey

Online Lectures in Higher Education Institutions in Nigeria During the COVID-19 Pandemic: Challenges and Prospects

• Gloria O. Okafor, Amaka E. Agbata, Jerry C. Orajekwe, Chinedu U. Asogwa

Online Higher Education in South Africa During COVID-19 and Beyond: Opportunities, Challenges, and Its Future

• Ndangwa Noyoo, Minenhle Matela, Mziwandile Sobantu, Chance Chagunda

Lived Experiences of Female Social Work and Community Develpment Students at the University of Zambia (UNZA) About Online Education During Covid-19 Pandemic

• Isaac Kabelenga, Mathias Alubafi Fubah

Online Education During COVID-19 and Beyond in Asia

This book aims to provide sustainable solutions for better understanding and management of online education in different parts of the world. In this context, it explores the attitudes and perceptions of stakeholders, such as students, faculty, and other actors on issues related to online education. In particular, it examines the challenges they have faced over the years when online courses were introduced due to the COVID-19 pandemic.

A model is proposed that includes five variables: specific communication issues in online education, the ability of professors to offer online courses, the quality of online education, students' perceived stress during online education, and the technical requirements of online education.

The book will be of interest to anyone concerned with the new and future ways of teaching and learning.

Chapter “When a Phenomenon-Based University Course Went Online: Students’ Experiences and Reflections After Sauna Bathing” is available open access under a Creative Commons Attribution 4.0 International License via link.springer.com.

• Internet for online education
• Non-verbal communication
• Online classes
• Online education
• Face-to-face education
• Digital transition
• Online education management
• Higher education management

Silvia Puiu

Samuel O. Idowu

Silvia Puiu is an Associate Professor in the Department of Management, Marketing and Business Administration of the Faculty of Economics and Business Administration at the University of Craiova, Romania. She earned a Ph.D. in management in 2012 and teaches management, ethics management in business, marketing, and creative writing in marketing. During her postdoctoral studies, Silvia Puiu conducted research into ethics management in the public sector of Romania and spent two months in Italy at the University of Milano-Bicocca, conducting comparative research between Romania and Italy regarding the Corruption Perception Index, ethics management in the public sector, and regulations on whistleblowing. Her research interests include topics such as corporate social responsibility, strategic management, ethics management, and marketing and management.

Samuel O. Idowu   is a senior lecturer at London Metropolitan University (UK). He is a fellow member of the Chartered Governance Institute, a fellow of the Royal Society of Arts, a Liveryman of the Worshipful Company of Chartered Secretaries & Administrators, and a named freeman of the City of London. Samuel has been in academia for more than 30 years. He is editor-in-chief of the International Journal of CSR and editor-in-chief of the American Journal of Economics and Business Administration and also a series editor of the book series CSR, Sustainability, Ethics & Governance.

Book Title : Online Education During COVID-19 and Beyond

Book Subtitle : Opportunities, Challenges and Outlook

Editors : Silvia Puiu, Samuel O. Idowu

Series Title : CSR, Sustainability, Ethics & Governance

DOI : https://doi.org/10.1007/978-3-031-49353-9

Publisher : Springer Cham

eBook Packages : Business and Management , Business and Management (R0)

Copyright Information : The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2024

Hardcover ISBN : 978-3-031-49352-2 Published: 11 April 2024

Softcover ISBN : 978-3-031-49355-3 Due: 12 May 2024

eBook ISBN : 978-3-031-49353-9 Published: 10 April 2024

Series ISSN : 2196-7075

Series E-ISSN : 2196-7083

Edition Number : 1

Number of Pages : XXIII, 436

Number of Illustrations : 22 b/w illustrations, 27 illustrations in colour

Topics : Business and Management, general , Education, general , Media and Communication

Policies and ethics

• Find a journal
• Track your research

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 30 January 2023

A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic

• Bastian A. Betthäuser   ORCID: orcid.org/0000-0002-4544-4073 1 , 2 , 3 ,
• Anders M. Bach-Mortensen   ORCID: orcid.org/0000-0001-7804-7958 2 &
• Per Engzell   ORCID: orcid.org/0000-0002-2404-6308 3 , 4 , 5  

Nature Human Behaviour volume  7 ,  pages 375–385 ( 2023 ) Cite this article

69k Accesses

103 Citations

1964 Altmetric

Metrics details

• Social policy

To what extent has the learning progress of school-aged children slowed down during the COVID-19 pandemic? A growing number of studies address this question, but findings vary depending on context. Here we conduct a pre-registered systematic review, quality appraisal and meta-analysis of 42 studies across 15 countries to assess the magnitude of learning deficits during the pandemic. We find a substantial overall learning deficit (Cohen’s d  = −0.14, 95% confidence interval −0.17 to −0.10), which arose early in the pandemic and persists over time. Learning deficits are particularly large among children from low socio-economic backgrounds. They are also larger in maths than in reading and in middle-income countries relative to high-income countries. There is a lack of evidence on learning progress during the pandemic in low-income countries. Future research should address this evidence gap and avoid the common risks of bias that we identify.

Similar content being viewed by others

Elementary school teachers’ perspectives about learning during the COVID-19 pandemic

Aymee Alvarez-Rivero, Candice Odgers & Daniel Ansari

A methodological perspective on learning in the developing brain

Anna C. K. van Duijvenvoorde, Lucy B. Whitmore, … Kathryn L. Mills

Measuring and forecasting progress in education: what about early childhood?

Linda M. Richter, Jere R. Behrman, … Hirokazu Yoshikawa

The coronavirus disease 2019 (COVID-19) pandemic has led to one of the largest disruptions to learning in history. To a large extent, this is due to school closures, which are estimated to have affected 95% of the world’s student population 1 . But even when face-to-face teaching resumed, instruction has often been compromised by hybrid teaching, and by children or teachers having to quarantine and miss classes. The effect of limited face-to-face instruction is compounded by the pandemic’s consequences for children’s out-of-school learning environment, as well as their mental and physical health. Lockdowns have restricted children’s movement and their ability to play, meet other children and engage in extra-curricular activities. Children’s wellbeing and family relationships have also suffered due to economic uncertainties and conflicting demands of work, care and learning. These negative consequences can be expected to be most pronounced for children from low socio-economic family backgrounds, exacerbating pre-existing educational inequalities.

It is critical to understand the extent to which learning progress has changed since the onset of the COVID-19 pandemic. We use the term ‘learning deficit’ to encompass both a delay in expected learning progress, as well as a loss of skills and knowledge already gained. The COVID-19 learning deficit is likely to affect children’s life chances through their education and labour market prospects. At the societal level, it can have important implications for growth, prosperity and social cohesion. As policy-makers across the world are seeking to limit further learning deficits and to devise policies to recover learning deficits that have already been incurred, assessing the current state of learning is crucial. A careful assessment of the COVID-19 learning deficit is also necessary to weigh the true costs and benefits of school closures.

A number of narrative reviews have sought to summarize the emerging research on COVID-19 and learning, mostly focusing on learning progress relatively early in the pandemic 2 , 3 , 4 , 5 , 6 . Moreover, two reviews harmonized and synthesized existing estimates of learning deficits during the pandemic 7 , 8 . In line with the narrative reviews, these two reviews find a substantial reduction in learning progress during the pandemic. However, this finding is based on a relatively small number of studies (18 and 10 studies, respectively). The limited evidence that was available at the time these reviews were conducted also precluded them from meta-analysing variation in the magnitude of learning deficits over time and across subjects, different groups of students or country contexts.

In this Article, we conduct a systematic review and meta-analysis of the evidence on COVID-19 learning deficits 2.5 years into the pandemic. Our primary pre-registered research question was ‘What is the effect of the COVID-19 pandemic on learning progress amongst school-age children?’, and we address this question using evidence from studies examining changes in learning outcomes during the pandemic. Our second pre-registered research aim was ‘To examine whether the effect of the COVID-19 pandemic on learning differs across different social background groups, age groups, boys and girls, learning areas or subjects, national contexts’.

We contribute to the existing research in two ways. First, we describe and appraise the up-to-date body of evidence, including its geographic reach and quality. More specifically, we ask the following questions: (1) what is the state of the evidence, in terms of the available peer-reviewed research and grey literature, on learning progress of school-aged children during the COVID-19 pandemic?, (2) which countries are represented in the available evidence? and (3) what is the quality of the existing evidence?

Our second contribution is to harmonize, synthesize and meta-analyse the existing evidence, with special attention to variation across different subpopulations and country contexts. On the basis of the identified studies, we ask (4) to what extent has the learning progress of school-aged children changed since the onset of the pandemic?, (5) how has the magnitude of the learning deficit (if any) evolved since the beginning of the pandemic?, (6) to what extent has the pandemic reinforced inequalities between children from different socio-economic backgrounds?, (7) are there differences in the magnitude of learning deficits between subject domains (maths and reading) and between age groups (primary and secondary students)? and (8) to what extent does the magnitude of learning deficits vary across national contexts?

Below, we report our answers to each of these questions in turn. The questions correspond to the analysis plan set out in our pre-registered protocol ( https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021249944 ), but we have adjusted the order and wording to aid readability. We had planned to examine gender differences in learning progress during the pandemic, but found there to be insufficient evidence to conduct this subgroup analysis, as the large majority of the identified studies do not provide evidence on learning deficits separately by gender. We also planned to examine how the magnitude of learning deficits differs across groups of students with varying exposures to school closures. This was not possible as the available data on school closures lack sufficient depth with respect to variation of school closures within countries, across grade levels and with respect to different modes of instruction, to meaningfully examine this association.

The state of the evidence

Our systematic review identified 42 studies on learning progress during the COVID-19 pandemic that met our inclusion criteria. To be included in our systematic review and meta-analysis, studies had to use a measure of learning that can be standardized (using Cohen’s d ) and base their estimates on empirical data collected since the onset of the COVID-19 pandemic (rather than making projections based on pre-COVID-19 data). As shown in Fig. 1 , the initial literature search resulted in 5,153 hits after removal of duplicates. All studies were double screened by the first two authors. The formal database search process identified 15 eligible studies. We also hand searched relevant preprint repositories and policy databases. Further, to ensure that our study selection was as up to date as possible, we conducted two full forward and backward citation searches of all included studies on 15 February 2022, and on 8 August 2022. The citation and preprint hand searches allowed us to identify 27 additional eligible studies, resulting in a total of 42 studies. Most of these studies were published after the initial database search, which illustrates that the body of evidence continues to expand. Most studies provide multiple estimates of COVID-19 learning deficits, separately for maths and reading and for different school grades. The number of estimates ( n  = 291) is therefore larger than the number of included studies ( n  = 42).

Flow diagram of the study identification and selection process, following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

The geographic reach of evidence is limited

Table 1 presents all included studies and estimates of COVID-19 learning deficits (in brackets), grouped by the 15 countries represented: Australia, Belgium, Brazil, Colombia, Denmark, Germany, Italy, Mexico, the Netherlands, South Africa, Spain, Sweden, Switzerland, the UK and the United States. About half of the estimates ( n  = 149) are from the United States, 58 are from the UK, a further 70 are from other European countries and the remaining 14 estimates are from Australia, Brazil, Colombia, Mexico and South Africa. As this list shows, there is a strong over-representation of studies from high-income countries, a dearth of studies from middle-income countries and no studies from low-income countries. This skewed representation should be kept in mind when interpreting our synthesis of the existing evidence on COVID-19 learning deficits.

The quality of evidence is mixed

We assessed the quality of the evidence using an adapted version of the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool 9 . More specifically, we analysed the risk of bias of each estimate from confounding, sample selection, classification of treatments, missing data, the measurement of outcomes and the selection of reported results. A.M.B.-M. and B.A.B. performed the risk-of-bias assessments, which were independently checked by the respective other author. We then assigned each study an overall risk-of-bias rating (low, moderate, serious or critical) based on the estimate and domain with the highest risk of bias.

Figure 2a shows the distribution of all studies of COVID-19 learning deficits according to their risk-of-bias rating separately for each domain (top six rows), as well as the distribution of studies according to their overall risk of bias rating (bottom row). The overall risk of bias was considered ‘low’ for 15% of studies, ‘moderate’ for 30% of studies, ‘serious’ for 25% of studies and ‘critical’ for 30% of studies.

a , Domain-specific and overall distribution of studies of COVID-19 learning deficits by risk of bias rating using ROBINS-I, including studies rated to be at critical risk of bias ( n  = 19 out of a total of n  = 61 studies shown in this figure). In line with ROBINS-I guidance, studies rated to be at critical risk of bias were excluded from all analyses and other figures in this article and in the Supplementary Information (including b ). b , z curve: distribution of the z scores of all estimates included in the meta-analysis ( n  = 291) to test for publication bias. The dotted line indicates z  = 1.96 ( P  = 0.050), the conventional threshold for statistical significance. The overlaid curve shows a normal distribution. The absence of a spike in the distribution of the z scores just above the threshold for statistical significance and the absence of a slump just below it indicate the absence of evidence for publication bias.

In line with ROBINS-I guidance, we excluded studies rated to be at critical risk of bias ( n  = 19) from all of our analyses and figures, except for Fig. 2a , which visualizes the distribution of studies according to their risk of bias 9 . These are thus not part of the 42 studies included in our meta-analysis. Supplementary Table 2 provides an overview of these studies as well as the main potential sources of risk of bias. Moreover, in Supplementary Figs. 3 – 6 , we replicate all our results excluding studies deemed to be at serious risk of bias.

As shown in Fig. 2a , common sources of potential bias were confounding, sample selection and missing data. Studies rated at risk of confounding typically compared only two timepoints, without accounting for longer time trends in learning progress. The main causes of selection bias were the use of convenience samples and insufficient consideration of self-selection by schools or students. Several studies found evidence of selection bias, often with students from a low socio-economic background or schools in deprived areas being under-represented after (as compared with before) the pandemic, but this was not always adjusted for. Some studies also reported a higher amount of missing data post-pandemic, again generally without adjustment, and several studies did not report any information on missing data. For an overview of the risk-of-bias ratings for each domain of each study, see Supplementary Fig. 1 and Supplementary Tables 1 and 2 .

No evidence of publication bias

Publication bias can occur if authors self-censor to conform to theoretical expectations, or if journals favour statistically significant results. To mitigate this concern, we include not only published papers, but also preprints, working papers and policy reports.

Moreover, Fig. 2b tests for publication bias by showing the distribution of z -statistics for the effect size estimates of all identified studies. The dotted line indicates z  = 1.96 ( P  = 0.050), the conventional threshold for statistical significance. The overlaid curve shows a normal distribution. If there was publication bias, we would expect a spike just above the threshold, and a slump just below it. There is no indication of this. Moreover, we do not find a left-skewed distribution of P values (see P curve in Supplementary Fig. 2a ), or an association between estimates of learning deficits and their standard errors (see funnel plot in Supplementary Fig. 2b ) that would suggest publication bias. Publication bias thus does not appear to be a major concern.

Having assessed the quality of the existing evidence, we now present the substantive results of our meta-analysis, focusing on the magnitude of COVID-19 learning deficits and on the variation in learning deficits over time, across different groups of students, and across country contexts.

Learning progress slowed substantially during the pandemic

Figure 3 shows the effect sizes that we extracted from each study (averaged across grades and learning subject) as well as the pooled effect size (red diamond). Effects are expressed in standard deviations, using Cohen’s d . Estimates are pooled using inverse variance weights. The pooled effect size across all studies is d  = −0.14, t (41) = −7.30, two-tailed P  = 0.000, 95% confidence interval (CI) −0.17 to −0.10. Under normal circumstances, students generally improve their performance by around 0.4 standard deviations per school year 10 , 11 , 12 . Thus, the overall effect of d  = −0.14 suggests that students lost out on 0.14/0.4, or about 35%, of a school year’s worth of learning. On average, the learning progress of school-aged children has slowed substantially during the pandemic.

Effect sizes are expressed in standard deviations, using Cohen’s d , with 95% CI, and are sorted by magnitude.

Learning deficits arose early in the pandemic and persist

One may expect that children were able to recover learning that was lost early in the pandemic, after teachers and families had time to adjust to the new learning conditions and after structures for online learning and for recovering early learning deficits were set up. However, existing research on teacher strikes in Belgium 13 and Argentina 14 , shortened school years in Germany 15 and disruptions to education during World War II 16 suggests that learning deficits are difficult to compensate and tend to persist in the long run.

Figure 4 plots the magnitude of estimated learning deficits (on the vertical axis) by the date of measurement (on the horizontal axis). The colour of the circles reflects the relevant country, the size of the circles indicates the sample size for a given estimate and the line displays a linear trend. The figure suggests that learning deficits opened up early in the pandemic and have neither closed nor substantially widened since then. We find no evidence that the slope coefficient is different from zero ( β months  = −0.00, t (41) = −7.30, two-tailed P  = 0.097, 95% CI −0.01 to 0.00). This implies that efforts by children, parents, teachers and policy-makers to adjust to the changed circumstance have been successful in preventing further learning deficits but so far have been unable to reverse them. As shown in Supplementary Fig. 8 , the pattern of persistent learning deficits also emerges within each of the three countries for which we have a relatively large number of estimates at different timepoints: the United States, the UK and the Netherlands. However, it is important to note that estimates of learning deficits are based on distinct samples of students. Future research should continue to follow the learning progress of cohorts of students in different countries to reveal how learning deficits of these cohorts have developed and continue to develop since the onset of the pandemic.

The horizontal axis displays the date on which learning progress was measured. The vertical axis displays estimated learning deficits, expressed in standard deviation (s.d.) using Cohen’s d . The colour of the circles reflects the respective country, the size of the circles indicates the sample size for a given estimate and the line displays a linear trend with a 95% CI. The trend line is estimated as a linear regression using ordinary least squares, with standard errors clustered at the study level ( n  = 42 clusters). β months  = −0.00, t (41) = −7.30, two-tailed P  = 0.097, 95% CI −0.01 to 0.00.

Socio-economic inequality in education increased

Existing research on the development of learning gaps during summer vacations 17 , 18 , disruptions to schooling during the Ebola outbreak in Sierra Leone and Guinea 19 , and the 2005 earthquake in Pakistan 20 shows that the suspension of face-to-face teaching can increase educational inequality between children from different socio-economic backgrounds. Learning deficits during the COVID-19 pandemic are likely to have been particularly pronounced for children from low socio-economic backgrounds. These children have been more affected by school closures than children from more advantaged backgrounds 21 . Moreover, they are likely to be disadvantaged with respect to their access and ability to use digital learning technology, the quality of their home learning environment, the learning support they receive from teachers and parents, and their ability to study autonomously 22 , 23 , 24 .

Most studies we identify examine changes in socio-economic inequality during the pandemic, attesting to the importance of the issue. As studies use different measures of socio-economic background (for example, parental income, parental education, free school meal eligibility or neighbourhood disadvantage), pooling the estimates is not possible. Instead, we code all estimates according to whether they indicate a reduction, no change or an increase in learning inequality during the pandemic. Figure 5 displays this information. Estimates that indicate an increase in inequality are shown on the right, those that indicate a decrease on the left and those that suggest no change in the middle. Squares represent estimates of changes in inequality during the pandemic in reading performance, and circles represent estimates of changes in inequality in maths performance. The shading represents when in the pandemic educational inequality was measured, differentiating between the first, second and third year of the pandemic. Estimates are also arranged horizontally by grade level. A large majority of estimates indicate an increase in educational inequality between children from different socio-economic backgrounds. This holds for both maths and reading, across primary and secondary education, at each stage of the pandemic, and independently of how socio-economic background is measured.

Each circle/square refers to one estimate of over-time change in inequality in maths/reading performance ( n  = 211). Estimates that find a decrease/no change/increase in inequality are grouped on the left/middle/right. Within these categories, estimates are ordered horizontally by school grade. The shading indicates when in the pandemic a given measure was taken.

Learning deficits are larger in maths than in reading

Available research on summer learning deficits 17 , 25 , student absenteeism 26 , 27 and extreme weather events 28 suggests that learning progress in mathematics is more dependent on formal instruction than in reading. This might be due to parents being better equipped to help their children with reading, and children advancing their reading skills (but not their maths skills) when reading for enjoyment outside of school. Figure 6a shows that, similarly to earlier disruptions to learning, the estimated learning deficits during the COVID-19 pandemic are larger for maths than for reading (mean difference δ  = −0.07, t (41) = −4.02, two-tailed P  = 0.000, 95% CI −0.11 to −0.04). This difference is statistically significant and robust to dropping estimates from individual countries (Supplementary Fig. 9 ).

Each plot shows the distribution of COVID-19 learning deficit estimates for the respective subgroup, with the box marking the interquartile range and the white circle denoting the median. Whiskers mark upper and lower adjacent values: the furthest observation within 1.5 interquartile range of either side of the box. a , Learning subject (reading versus maths). Median: reading −0.09, maths −0.18. Interquartile range: reading −0.15 to −0.02, maths −0.23 to −0.09. b , Level of education (primary versus secondary). Median: primary −0.12, secondary −0.12. Interquartile range: primary −0.19 to −0.05, secondary −0.21 to −0.06. c , Country income level (high versus middle). Median: high −0.12, middle −0.37. Interquartile range: high −0.20 to −0.05, middle −0.65 to −0.30.

No evidence of variation across grade levels

One may expect learning deficits to be smaller for older than for younger children, as older children may be more autonomous in their learning and better able to cope with a sudden change in their learning environment. However, older students were subject to longer school closures in some countries, such as Denmark 29 , based partly on the assumption that they would be better able to learn from home. This may have offset any advantage that older children would otherwise have had in learning remotely.

Figure 6b shows the distribution of estimates of learning deficits for students at the primary and secondary level, respectively. Our analysis yields no evidence of variation in learning deficits across grade levels (mean difference δ  = −0.01, t (41) = −0.59, two-tailed P  = 0.556, 95% CI −0.06 to 0.03). Due to the limited number of available estimates of learning deficits, we cannot be certain about whether learning deficits differ between primary and secondary students or not.

Learning deficits are larger in poorer countries

Low- and middle-income countries were already struggling with a learning crisis before the pandemic. Despite large expansions of the proportion of children in school, children in low- and middle-income countries still perform poorly by international standards, and inequality in learning remains high 30 , 31 , 32 . The pandemic is likely to deepen this learning crisis and to undo past progress. Schools in low- and middle-income countries have not only been closed for longer, but have also had fewer resources to facilitate remote learning 33 , 34 . Moreover, the economic resources, availability of digital learning equipment and ability of children, parents, teachers and governments to support learning from home are likely to be lower in low- and middle-income countries 35 .

As discussed above, most evidence on COVID-19 learning deficits comes from high-income countries. We found no studies on low-income countries that met our inclusion criteria, and evidence from middle-income countries is limited to Brazil, Colombia, Mexico and South Africa. Figure 6c groups the estimates of COVID-19 learning deficits in these four middle-income countries together (on the right) and compares them with estimates from high-income countries (on the left). The learning deficit is appreciably larger in middle-income countries than in high-income countries (mean difference δ  = −0.29, t (41) = −2.78, two-tailed P  = 0.008, 95% CI −0.50 to −0.08). In fact, the three largest estimates of learning deficits in our sample are from middle-income countries (Fig. 3 ) 36 , 37 , 38 .

Two years since the COVID-19 pandemic, there is a growing number of studies examining the learning progress of school-aged children during the pandemic. This paper first systematically reviews the existing literature on learning progress of school-aged children during the pandemic and appraises its geographic reach and quality. Second, it harmonizes, synthesizes and meta-analyses the existing evidence to examine the extent to which learning progress has changed since the onset of the pandemic, and how it varies across different groups of students and across country contexts.

Our meta-analysis suggests that learning progress has slowed substantially during the COVID-19 pandemic. The pooled effect size of d  = −0.14, implies that students lost out on about 35% of a normal school year’s worth of learning. This confirms initial concerns that substantial learning deficits would arise during the pandemic 10 , 39 , 40 . But our results also suggest that fears of an accumulation of learning deficits as the pandemic continues have not materialized 41 , 42 . On average, learning deficits emerged early in the pandemic and have neither closed nor widened substantially. Future research should continue to follow the learning progress of cohorts of students in different countries to reveal how learning deficits of these cohorts have developed and continue to develop since the onset of the pandemic.

Most studies that we identify find that learning deficits have been largest for children from disadvantaged socio-economic backgrounds. This holds across different timepoints during the pandemic, countries, grade levels and learning subjects, and independently of how socio-economic background is measured. It suggests that the pandemic has exacerbated educational inequalities between children from different socio-economic backgrounds, which were already large before the pandemic 43 , 44 . Policy initiatives to compensate learning deficits need to prioritize support for children from low socio-economic backgrounds in order to allow them to recover the learning they lost during the pandemic.

There is a need for future research to assess how the COVID-19 pandemic has affected gender inequality in education. So far, there is very little evidence on this issue. The large majority of the studies that we identify do not examine learning deficits separately by gender.

Comparing estimates of learning deficits across subjects, we find that learning deficits tend to be larger in maths than in reading. As noted above, this may be due to the fact that parents and children have been in a better position to compensate school-based learning in reading by reading at home. Accordingly, there are grounds for policy initiatives to prioritize the compensation of learning deficits in maths and other science subjects.

A limitation of this study and the existing body of evidence on learning progress during the COVID-19 pandemic is that the existing studies primarily focus on high-income countries, while there is a dearth of evidence from low- and middle-income countries. This is particularly concerning because the small number of existing studies from middle-income countries suggest that learning deficits have been particularly severe in these countries. Learning deficits are likely to be even larger in low-income countries, considering that these countries already faced a learning crisis before the pandemic, generally implemented longer school closures, and were under-resourced and ill-equipped to facilitate remote learning 32 , 33 , 34 , 35 , 45 . It is critical that this evidence gap on low- and middle-income countries is addressed swiftly, and that the infrastructure to collect and share data on educational performance in middle- and low-income countries is strengthened. Collecting and making available these data is a key prerequisite for fully understanding how learning progress and related outcomes have changed since the onset of the pandemic 46 .

A further limitation is that about half of the studies that we identify are rated as having a serious or critical risk of bias. We seek to limit the risk of bias in our results by excluding all studies rated to be at critical risk of bias from all of our analyses. Moreover, in Supplementary Figs. 3 – 6 , we show that our results are robust to further excluding studies deemed to be at serious risk of bias. Future studies should minimize risk of bias in estimating learning deficits by employing research designs that appropriately account for common sources of bias. These include a lack of accounting for secular time trends, non-representative samples and imbalances between treatment and comparison groups.

The persistence of learning deficits two and a half years into the pandemic highlights the need for well-designed, well-resourced and decisive policy initiatives to recover learning deficits. Policy-makers, schools and families will need to identify and realize opportunities to complement and expand on regular school-based learning. Experimental evidence from low- and middle-income countries suggests that even relatively low-tech and low-cost learning interventions can have substantial, positive effects on students’ learning progress in the context of remote learning. For example, sending SMS messages with numeracy problems accompanied by short phone calls was found to lead to substantial learning gains in numeracy in Botswana 47 . Sending motivational text messages successfully limited learning losses in maths and Portuguese in Brazil 48 .

More evidence is needed to assess the effectiveness of other interventions for limiting or recovering learning deficits. Potential avenues include the use of the often extensive summer holidays to offer summer schools and learning camps, extending school days and school weeks, and organizing and scaling up tutoring programmes. Further potential lies in developing, advertising and providing access to learning apps, online learning platforms or educational TV programmes that are free at the point of use. Many countries have already begun investing substantial resources to capitalize on some of these opportunities. If these interventions prove effective, and if the momentum of existing policy efforts is maintained and expanded, the disruptions to learning during the pandemic may be a window of opportunity to improve the education afforded to children.

Eligibility criteria

We consider all types of primary research, including peer-reviewed publications, preprints, working papers and reports, for inclusion. To be eligible for inclusion, studies have to measure learning progress using test scores that can be standardized across studies using Cohen’s d . Moreover, studies have to be in English, Danish, Dutch, French, German, Norwegian, Spanish or Swedish.

Search strategy and study identification

We identified relevant studies using the following steps. First, we developed a Boolean search string defining the population (school-aged children), exposure (the COVID-19 pandemic) and outcomes of interest (learning progress). The full search string can be found in Section 1.1 of Supplementary Information . Second, we used this string to search the following academic databases: Coronavirus Research Database, the Education Resources Information Centre, International Bibliography of the Social Sciences, Politics Collection (PAIS index, policy file index, political science database and worldwide political science abstracts), Social Science Database, Sociology Collection (applied social science index and abstracts, sociological abstracts and sociology database), Cumulative Index to Nursing and Allied Health Literature, and Web of Science. Second, we hand-searched multiple preprint and working paper repositories (Social Science Research Network, Munich Personal RePEc Archive, IZA, National Bureau of Economic Research, OSF Preprints, PsyArXiv, SocArXiv and EdArXiv) and relevant policy websites, including the websites of the Organization for Economic Co-operation and Development, the United Nations, the World Bank and the Education Endowment Foundation. Third, we periodically posted our protocol via Twitter in order to crowdsource additional relevant studies not identified through the search. All titles and abstracts identified in our search were double-screened using the Rayyan online application 49 . Our initial search was conducted on 27 April 2021, and we conducted two forward and backward citation searches of all eligible studies identified in the above steps, on 14 February 2022, and on 8 August 2022, to ensure that our analysis includes recent relevant research.

Data extraction

From the studies that meet our inclusion criteria we extracted all estimates of learning deficits during the pandemic, separately for maths and reading and for different school grades. We also extracted the corresponding sample size, standard error, date(s) of measurement, author name(s) and country. Last, we recorded whether studies differentiate between children’s socio-economic background, which measure is used to this end and whether studies find an increase, decrease or no change in learning inequality. We contacted study authors if any of the above information was missing in the study. Data extraction was performed by B.A.B. and validated independently by A.M.B.-M., with discrepancies resolved through discussion and by conferring with P.E.

Measurement and standardizationr

We standardize all estimates of learning deficits during the pandemic using Cohen’s d , which expresses effect sizes in terms of standard deviations. Cohen’s d is calculated as the difference in the mean learning gain in a given subject (maths or reading) over two comparable periods before and after the onset of the pandemic, divided by the pooled standard deviation of learning progress in this subject:

Effect sizes expressed as β coefficients are converted to Cohen’s d :

We use a binary indicator for whether the study outcome is maths or reading. One study does not differentiate the outcome but includes a composite of maths and reading scores 50 .

Level of education

We distinguish between primary and secondary education. We first consulted the original studies for this information. Where this was not stated in a given study, students’ age was used in conjunction with information about education systems from external sources to determine the level of education 51 .

Country income level

We follow the World Bank’s classification of countries into four income groups: low, lower-middle, upper-middle and high income. Four countries in our sample are in the upper-middle-income group: Brazil, Colombia, Mexico and South Africa. All other countries are in the high-income group.

Data synthesis

We synthesize our data using three synthesis techniques. First, we generate a forest plot, based on all available estimates of learning progress during the pandemic. We pool estimates using a random-effects restricted maximum likelihood model and inverse variance weights to calculate an overall effect size (Fig. 3 ) 52 . Second, we code all estimates of changes in educational inequality between children from different socio-economic backgrounds during the pandemic, according to whether they indicate an increase, a decrease or no change in educational inequality. We visualize the resulting distribution using a harvest plot (Fig. 5 ) 53 . Third, given that the limited amount of available evidence precludes multivariate or causal analyses, we examine the bivariate association between COVID-19 learning deficits and the months in which learning was measured using a scatter plot (Fig. 4 ), and the bivariate association between COVID-19 learning deficits and subject, grade level and countries’ income level, using a series of violin plots (Fig. 6 ). The reported estimates, CIs and statistical significance tests of these bivariate associations are based on common-effects models with standard errors clustered by study, and two-sided tests. With respect to statistical tests reported, the data distribution was assumed to be normal, but this was not formally tested. The distribution of estimates of learning deficits is shown separately for the different moderator categories in Fig. 6 .

Pre-registration

We prospectively registered a protocol of our systematic review and meta-analysis in the International Prospective Register of Systematic Reviews (CRD42021249944) on 19 April 2021 ( https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021249944 ).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Data availability

The data used in the analyses for this manuscript were compiled by the authors based on the studies identified in the systematic review. The data are available on the Open Science Framework repository ( https://doi.org/10.17605/osf.io/u8gaz ). For our systematic review, we searched the following databases: Coronavirus Research Database ( https://proquest.libguides.com/covid19 ), Education Resources Information Centre database ( https://eric.ed.gov ), International Bibliography of the Social Sciences ( https://about.proquest.com/en/products-services/ibss-set-c/ ), Politics Collection ( https://about.proquest.com/en/products-services/ProQuest-Politics-Collection/ ), Social Science Database ( https://about.proquest.com/en/products-services/pq_social_science/ ), Sociology Collection ( https://about.proquest.com/en/products-services/ProQuest-Sociology-Collection/ ), Cumulative Index to Nursing and Allied Health Literature ( https://www.ebsco.com/products/research-databases/cinahl-database ) and Web of Science ( https://clarivate.com/webofsciencegroup/solutions/web-of-science/ ). We also searched the following preprint and working paper repositories: Social Science Research Network ( https://papers.ssrn.com/sol3/DisplayJournalBrowse.cfm ), Munich Personal RePEc Archive ( https://mpra.ub.uni-muenchen.de ), IZA ( https://www.iza.org/content/publications ), National Bureau of Economic Research ( https://www.nber.org/papers?page=1&perPage=50&sortBy=public_date ), OSF Preprints ( https://osf.io/preprints/ ), PsyArXiv ( https://psyarxiv.com ), SocArXiv ( https://osf.io/preprints/socarxiv ) and EdArXiv ( https://edarxiv.org ).

Code availability

All code needed to replicate our findings is available on the Open Science Framework repository ( https://doi.org/10.17605/osf.io/u8gaz ).

The Impact of COVID-19 on Children. UN Policy Briefs (United Nations, 2020).

Donnelly, R. & Patrinos, H. A. Learning loss during Covid-19: An early systematic review. Prospects (Paris) 51 , 601–609 (2022).

Hammerstein, S., König, C., Dreisörner, T. & Frey, A. Effects of COVID-19-related school closures on student achievement: a systematic review. Front. Psychol. https://doi.org/10.3389/fpsyg.2021.746289 (2021).

Panagouli, E. et al. School performance among children and adolescents during COVID-19 pandemic: a systematic review. Children 8 , 1134 (2021).

Article   PubMed   PubMed Central   Google Scholar  

Patrinos, H. A., Vegas, E. & Carter-Rau, R. An Analysis of COVID-19 Student Learning Loss (World Bank, 2022).

Zierer, K. Effects of pandemic-related school closures on pupils’ performance and learning in selected countries: a rapid review. Educ. Sci. 11 , 252 (2021).

Article   Google Scholar  

König, C. & Frey, A. The impact of COVID-19-related school closures on student achievement: a meta-analysis. Educ. Meas. Issues Pract. 41 , 16–22 (2022).

Storey, N. & Zhang, Q. A meta-analysis of COVID learning loss. Preprint at EdArXiv (2021).

Sterne, J. A. et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ https://doi.org/10.1136/bmj.i4919 (2016).

Azevedo, J. P., Hasan, A., Goldemberg, D., Iqbal, S. A. & Geven, K. Simulating the Potential Impacts of COVID-19 School Closures on Schooling and Learning Outcomes: A Set of Global Estimates (World Bank, 2020).

Bloom, H. S., Hill, C. J., Black, A. R. & Lipsey, M. W. Performance trajectories and performance gaps as achievement effect-size benchmarks for educational interventions. J. Res. Educ. Effectiveness 1 , 289–328 (2008).

Hill, C. J., Bloom, H. S., Black, A. R. & Lipsey, M. W. Empirical benchmarks for interpreting effect sizes in research. Child Dev. Perspect. 2 , 172–177 (2008).

Belot, M. & Webbink, D. Do teacher strikes harm educational attainment of students? Labour 24 , 391–406 (2010).

Jaume, D. & Willén, A. The long-run effects of teacher strikes: evidence from Argentina. J. Labor Econ. 37 , 1097–1139 (2019).

Cygan-Rehm, K. Are there no wage returns to compulsory schooling in Germany? A reassessment. J. Appl. Econ. 37 , 218–223 (2022).

Ichino, A. & Winter-Ebmer, R. The long-run educational cost of World War II. J. Labor Econ. 22 , 57–87 (2004).

Cooper, H., Nye, B., Charlton, K., Lindsay, J. & Greathouse, S. The effects of summer vacation on achievement test scores: a narrative and meta-analytic review. Rev. Educ. Res. 66 , 227–268 (1996).

Allington, R. L. et al. Addressing summer reading setback among economically disadvantaged elementary students. Read. Psychol. 31 , 411–427 (2010).

Smith, W. C. Consequences of school closure on access to education: lessons from the 2013–2016 Ebola pandemic. Int. Rev. Educ. 67 , 53–78 (2021).

Andrabi, T., Daniels, B. & Das, J. Human capital accumulation and disasters: evidence from the Pakistan earthquake of 2005. J. Hum. Resour . https://doi.org/10.35489/BSG-RISE-WP_2020/039 (2021).

Parolin, Z. & Lee, E. K. Large socio-economic, geographic and demographic disparities exist in exposure to school closures. Nat. Hum. Behav. 5 , 522–528 (2021).

Goudeau, S., Sanrey, C., Stanczak, A., Manstead, A. & Darnon, C. Why lockdown and distance learning during the COVID-19 pandemic are likely to increase the social class achievement gap. Nat. Hum. Behav. 5 , 1273–1281 (2021).

Article   PubMed   Google Scholar  

Bailey, D. H., Duncan, G. J., Murnane, R. J. & Au Yeung, N. Achievement gaps in the wake of COVID-19. Educ. Researcher 50 , 266–275 (2021).

van de Werfhorst, H. G. Inequality in learning is a major concern after school closures. Proc. Natl Acad. Sci. USA https://doi.org/10.1073/pnas.2105243118 (2021).

Alexander, K. L., Entwisle, D. R. & Olson, L. S. Schools, achievement, and inequality: a seasonal perspective. Educ. Eval. Policy Anal. 23 , 171–191 (2001).

Aucejo, E. M. & Romano, T. F. Assessing the effect of school days and absences on test score performance. Econ. Educ. Rev. 55 , 70–87 (2016).

Gottfried, M. A. The detrimental effects of missing school: evidence from urban siblings. Am. J. Educ. 117 , 147–182 (2011).

Goodman, J. Flaking Out: Student Absences and Snow Days as Disruptions of Instructional Time (National Bureau of Economic Research, 2014).

Birkelund, J. F. & Karlson, K. B. No evidence of a major learning slide 14 months into the COVID-19 pandemic in Denmark. European Societies https://doi.org/10.1080/14616696.2022.2129085 (2022).

Angrist, N., Djankov, S., Goldberg, P. K. & Patrinos, H. A. Measuring human capital using global learning data. Nature 592 , 403–408 (2021).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Torche, F. in Social Mobility in Developing Countries: Concepts, Methods, and Determinants (eds Iversen, V., Krishna, A. & Sen, K.) 139–171 (Oxford Univ. Press, 2021).

World Development Report 2018: Learning to Realize Education’s Promise (World Bank, 2018).

Policy Brief: Education during COVID-19 and Beyond (United Nations, 2020).

One Year into COVID-19 Education Disruption: Where Do We Stand? (UNESCO, 2021).

Azevedo, J. P., Hasan, A., Goldemberg, D., Geven, K. & Iqbal, S. A. Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes: a set of global estimates. World Bank Res. Observer 36 , 1–40 (2021).

Google Scholar  

Ardington, C., Wills, G. & Kotze, J. COVID-19 learning losses: early grade reading in South Africa. Int. J. Educ. Dev. 86 , 102480 (2021).

Hevia, F. J., Vergara-Lope, S., Velásquez-Durán, A. & Calderón, D. Estimation of the fundamental learning loss and learning poverty related to COVID-19 pandemic in Mexico. Int. J. Educ. Dev. 88 , 102515 (2022).

Lichand, G., Doria, C. A., Leal-Neto, O. & Fernandes, J. P. C. The impacts of remote learning in secondary education during the pandemic in Brazil. Nat. Hum. Behav. 6 , 1079–1086 (2022).

Major, L. E., Eyles, A., Machin, S. et al. Learning Loss since Lockdown: Variation across the Home Nations (Centre for Economic Performance, London School of Economics and Political Science, 2021).

Di Pietro, G., Biagi, F., Costa, P., Karpinski, Z. & Mazza, J. The Likely Impact of COVID-19 on Education: Reflections Based on the Existing Literature and Recent International Datasets (Publications Office of the European Union, 2020).

Fuchs-Schündeln, N., Krueger, D., Ludwig, A. & Popova, I. The Long-Term Distributional and Welfare Effects of COVID-19 School Closures (National Bureau of Economic Research, 2020).

Kaffenberger, M. Modelling the long-run learning impact of the COVID-19 learning shock: actions to (more than) mitigate loss. Int. J. Educ. Dev. 81 , 102326 (2021).

Attewell, P. & Newman, K. S. Growing Gaps: Educational Inequality around the World (Oxford Univ. Press, 2010).

Betthäuser, B. A., Kaiser, C. & Trinh, N. A. Regional variation in inequality of educational opportunity across europe. Socius https://doi.org/10.1177/23780231211019890 (2021).

Angrist, N. et al. Building back better to avert a learning catastrophe: estimating learning loss from covid-19 school shutdowns in africa and facilitating short-term and long-term learning recovery. Int. J. Educ. Dev. 84 , 102397 (2021).

Conley, D. & Johnson, T. Opinion: Past is future for the era of COVID-19 research in the social sciences. Proc. Natl Acad. Sci. USA https://doi.org/10.1073/pnas.2104155118 (2021).

Angrist, N., Bergman, P. & Matsheng, M. Experimental evidence on learning using low-tech when school is out. Nat. Hum. Behav. 6 , 941–950 (2022).

Lichand, G., Christen, J. & van Egeraat, E. Do Behavioral Nudges Work under Remote Learning? Evidence from Brazil during the Pandemic (Univ. Zurich, 2022).

Ouzzani, M., Hammady, H., Fedorowicz, Z. & Elmagarmid, A. Rayyan—a web and mobile app for systematic reviews. Syst. Rev. 5 , 1–10 (2016).

Tomasik, M. J., Helbling, L. A. & Moser, U. Educational gains of in-person vs. distance learning in primary and secondary schools: a natural experiment during the COVID-19 pandemic school closures in Switzerland. Int. J. Psychol. 56 , 566–576 (2021).

Eurybase: The Information Database on Education Systems in Europe (Eurydice, 2021).

Borenstein, M., Hedges, L. V., Higgins, J. P. & Rothstein, H. R. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res. Synth. Methods 1 , 97–111 (2010).

Ogilvie, D. et al. The harvest plot: a method for synthesising evidence about the differential effects of interventions. BMC Med. Res. Methodol. 8 , 1–7 (2008).

Gore, J., Fray, L., Miller, A., Harris, J. & Taggart, W. The impact of COVID-19 on student learning in New South Wales primary schools: an empirical study. Aust. Educ. Res. 48 , 605–637 (2021).

Gambi, L. & De Witte, K. The Resiliency of School Outcomes after the COVID-19 Pandemic: Standardised Test Scores and Inequality One Year after Long Term School Closures (FEB Research Report Department of Economics, 2021).

Maldonado, J. E. & De Witte, K. The effect of school closures on standardised student test outcomes. Br. Educ. Res. J. 48 , 49–94 (2021).

Vegas, E. COVID-19’s Impact on Learning Losses and Learning Inequality in Colombia (Center for Universal Education at Brookings, 2022).

Depping, D., Lücken, M., Musekamp, F. & Thonke, F. in Schule während der Corona-Pandemie. Neue Ergebnisse und Überblick über ein dynamisches Forschungsfeld (eds Fickermann, D. & Edelstein, B.) 51–79 (Münster & New York: Waxmann, 2021).

Ludewig, U. et al. Die COVID-19 Pandemie und Lesekompetenz von Viertklässler*innen: Ergebnisse der IFS-Schulpanelstudie 2016–2021 (Institut für Schulentwicklungsforschung, Univ. Dortmund, 2022).

Schult, J., Mahler, N., Fauth, B. & Lindner, M. A. Did students learn less during the COVID-19 pandemic? Reading and mathematics competencies before and after the first pandemic wave. Sch. Eff. Sch. Improv. https://doi.org/10.1080/09243453.2022.2061014 (2022).

Schult, J., Mahler, N., Fauth, B. & Lindner, M. A. Long-term consequences of repeated school closures during the COVID-19 pandemic for reading and mathematics competencies. Front. Educ. https://doi.org/10.3389/feduc.2022.867316 (2022).

Bazoli, N., Marzadro, S., Schizzerotto, A. & Vergolini, L. Learning Loss and Students’ Social Origins during the COVID-19 Pandemic in Italy (FBK-IRVAPP Working Papers 3, 2022).

Borgonovi, F. & Ferrara, A. The effects of COVID-19 on inequalities in educational achievement in Italy. Preprint at SSRN https://doi.org/10.2139/ssrn.4171968 (2022).

Contini, D., Di Tommaso, M. L., Muratori, C., Piazzalunga, D. & Schiavon, L. Who lost the most? Mathematics achievement during the COVID-19 pandemic. BE J. Econ. Anal. Policy 22 , 399–408 (2022).

Engzell, P., Frey, A. & Verhagen, M. D. Learning loss due to school closures during the COVID-19 pandemic. Proc. Natl Acad. Sci. USA https://doi.org/10.1073/pnas.2022376118 (2021).

Haelermans, C. Learning Growth and Inequality in Primary Education: Policy Lessons from the COVID-19 Crisis (The European Liberal Forum (ELF)-FORES, 2021).

Haelermans, C. et al. A Full Year COVID-19 Crisis with Interrupted Learning and Two School Closures: The Effects on Learning Growth and Inequality in Primary Education (Maastricht Univ., Research Centre for Education and the Labour Market (ROA), 2021).

Haelermans, C. et al. Sharp increase in inequality in education in times of the COVID-19-pandemic. PLoS ONE 17 , e0261114 (2022).

Schuurman, T. M., Henrichs, L. F., Schuurman, N. K., Polderdijk, S. & Hornstra, L. Learning loss in vulnerable student populations after the first COVID-19 school closure in the Netherlands. Scand. J. Educ. Res. https://doi.org/10.1080/00313831.2021.2006307 (2021).

Arenas, A. & Gortazar, L. Learning Loss One Year after School Closures (Esade Working Paper, 2022).

Hallin, A. E., Danielsson, H., Nordström, T. & Fälth, L. No learning loss in Sweden during the pandemic evidence from primary school reading assessments. Int. J. Educ. Res. 114 , 102011 (2022).

Blainey, K. & Hannay, T. The Impact of School Closures on Autumn 2020 Attainment (RS Assessment from Hodder Education and SchoolDash, 2021).

Blainey, K. & Hannay, T. The Impact of School Closures on Spring 2021 Attainment (RS Assessment from Hodder Education and SchoolDash, 2021).

Blainey, K. & Hannay, T. The Effects of Educational Disruption on Primary School Attainment in Summer 2021 (RS Assessment from Hodder Education and SchoolDash, 2021).

Understanding Progress in the 2020/21 Academic Year: Complete Findings from the Autumn Term (London: Department for Education, 2021).

Understanding Progress in the 2020/21 Academic Year: Initial Findings from the Spring Term (London: Department for Education, 2021).

Impact of COVID-19 on Attainment: Initial Analysis (Brentford: GL Assessment, 2021).

Rose, S. et al. Impact of School Closures and Subsequent Support Strategies on Attainment and Socio-emotional Wellbeing in Key Stage 1: Interim Paper 1 (National Foundation for Educational Research (NFER) and Education Endowment Foundation (EEF) , 2021).

Rose, S. et al. Impact of School Closures and Subsequent Support Strategies on Attainment and Socio-emotional Wellbeing in Key Stage 1: Interim Paper 2 (National Foundation for Educational Research (NFER) and Education Endowment Foundation (EEF), 2021).

Weidmann, B. et al. COVID-19 Disruptions: Attainment Gaps and Primary School Responses (Education Endowment Foundation, 2021).

Bielinski, J., Brown, R. & Wagner, K. No Longer a Prediction: What New Data Tell Us About the Effects of 2020 Learning Disruptions (Illuminate Education, 2021).

Domingue, B. W., Hough, H. J., Lang, D. & Yeatman, J. Changing Patterns of Growth in Oral Reading Fluency During the COVID-19 Pandemic. PACE Working Paper (Policy Analysis for California Education, 2021).

Domingue, B. et al. The effect of COVID on oral reading fluency during the 2020–2021 academic year. AERA Open https://doi.org/10.1177/23328584221120254 (2022).

Kogan, V. & Lavertu, S. The COVID-19 Pandemic and Student Achievement on Ohio’s Third-Grade English Language Arts Assessment (Ohio State Univ., 2021).

Kogan, V. & Lavertu, S. How the COVID-19 Pandemic Affected Student Learning in Ohio: Analysis of Spring 2021 Ohio State Tests (Ohio State Univ., 2021).

Kozakowski, W., Gill, B., Lavallee, P., Burnett, A. & Ladinsky, J. Changes in Academic Achievement in Pittsburgh Public Schools during Remote Instruction in the COVID-19 Pandemic (Institute of Education Sciences (IES), US Department of Education, 2020).

Kuhfeld, M. & Lewis, K. Student Achievement in 2021–2022: Cause for Hope and Continued Urgency (NWEA, 2022).

Lewis, K., Kuhfeld, M., Ruzek, E. & McEachin, A. Learning during COVID-19: Reading and Math Achievement in the 2020–21 School Year (NWEA, 2021).

Locke, V. N., Patarapichayatham, C. & Lewis, S. Learning Loss in Reading and Math in US Schools Due to the COVID-19 Pandemic (Istation, 2021).

Pier, L., Christian, M., Tymeson, H. & Meyer, R. H. COVID-19 Impacts on Student Learning: Evidence from Interim Assessments in California. PACE Working Paper (Policy Analysis for California Education, 2021).

Download references

Acknowledgements

Carlsberg Foundation grant CF19-0102 (A.M.B.-M.); Leverhulme Trust Large Centre Grant (P.E.), the Swedish Research Council for Health, Working Life and Welfare (FORTE) grant 2016-07099 (P.E.); the French National Research Agency (ANR) as part of the ‘Investissements d’Avenir’ programme LIEPP (ANR-11-LABX-0091 and ANR-11-IDEX-0005-02) and the Université Paris Cité IdEx (ANR-18-IDEX-0001) (P.E.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and affiliations.

Centre for Research on Social Inequalities (CRIS), Sciences Po, Paris, France

Bastian A. Betthäuser

Department of Social Policy and Intervention, University of Oxford, Oxford, UK

Bastian A. Betthäuser & Anders M. Bach-Mortensen

Nuffield College, University of Oxford, Oxford, UK

Bastian A. Betthäuser & Per Engzell

Social Research Institute, University College London, London, UK

Per Engzell

Swedish Institute for Social Research, Stockholm University, Stockholm, Sweden

You can also search for this author in PubMed   Google Scholar

Contributions

B.A.B., A.M.B.-M. and P.E. designed the study; B.A.B., A.M.B.-M. and P.E. planned and implemented the search and screened studies; B.A.B., A.M.B.-M. and P.E. extracted relevant data from studies; B.A.B., A.M.B.-M. and P.E. conducted the quality appraisal; B.A.B., A.M.B.-M. and P.E. conducted the data analysis and visualization; B.A.B., A.M.B.-M. and P.E. wrote the manuscript.

Corresponding author

Correspondence to Bastian A. Betthäuser .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Human Behaviour thanks Guilherme Lichand, Sébastien Goudeau and Christoph König for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information.

Supplementary methods, results, figures, tables, PRISMA Checklist and references.

Reporting Summary

Peer review file, rights and permissions.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Betthäuser, B.A., Bach-Mortensen, A.M. & Engzell, P. A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic. Nat Hum Behav 7 , 375–385 (2023). https://doi.org/10.1038/s41562-022-01506-4

Download citation

Received : 24 June 2022

Accepted : 30 November 2022

Published : 30 January 2023

Issue Date : March 2023

DOI : https://doi.org/10.1038/s41562-022-01506-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Investigating the effect of covid-19 disruption in education using reds data.

• Alice Bertoletti
• Zbigniew Karpiński

Large-scale Assessments in Education (2024)

A democratic curriculum for the challenges of post-truth

• David Nally

Curriculum Perspectives (2024)

SGS: SqueezeNet-guided Gaussian-kernel SVM for COVID-19 Diagnosis

• Fanfeng Shi
• Vishnuvarthanan Govindaraj

Mobile Networks and Applications (2024)

Electrochemical determination of budesonide: a common corticosteroid used to treat respiratory diseases such as COVID-19 and asthma

• Katarzyna Jedlińska
• Katarzyna Trojanowska
• Bogusław Baś

Journal of Applied Electrochemistry (2024)

“Follow the Science” in COVID-19 Policy: A Scoping Review

• Jacob R. Greenmyer

HEC Forum (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

A Systematic Review of the Research Topics in Online Learning During COVID-19: Documenting the Sudden Shift

• Min Young Doo Kangwon National University http://orcid.org/0000-0003-3565-2159
• Meina Zhu Wayne State University
• Curtis J. Bonk Indiana University Bloomington

Since most schools and learners had no choice but to learn online during the pandemic, online learning became the mainstream learning mode rather than a substitute for traditional face-to-face learning. Given this enormous change in online learning, we conducted a systematic review of 191 of the most recent online learning studies published during the COVID-19 era. The systematic review results indicated that the themes regarding “courses and instructors” became popular during the pandemic, whereas most online learning research has focused on “learners” pre-COVID-19. Notably, the research topics “course and instructors” and “course technology” received more attention than prior to COVID-19. We found that “engagement” remained the most common research theme even after the pandemic. New research topics included parents, technology acceptance or adoption of online learning, and learners’ and instructors’ perceptions of online learning.

An, H., Mongillo, G., Sung, W., & Fuentes, D. (2022). Factors affecting online learning during the COVID-19 pandemic: The lived experiences of parents, teachers, and administrators in U.S. high-needs K-12 schools. The Journal of Online Learning Research (JOLR), 8(2), 203-234. https://www.learntechlib.org/primary/p/220404/

Aslan, S., Li, Q., Bonk, C. J., & Nachman, L. (2022). An overnight educational transformation: How did the pandemic turn early childhood education upside down? Online Learning, 26(2), 52-77. DOI: http://dx.doi.org/10.24059/olj.v26i2.2748

Azizan, S. N., Lee, A. S. H., Crosling, G., Atherton, G., Arulanandam, B. V., Lee, C. E., &

Abdul Rahim, R. B. (2022). Online learning and COVID-19 in higher education: The value of IT models in assessing students’ satisfaction. International Journal of Emerging Technologies in Learning (iJET), 17(3), 245–278. https://doi.org/10.3991/ijet.v17i03.24871

Beatty, B. J. (2019). Hybrid-flexible course design (1st ed.). EdTech Books. https://edtechbooks.org/hyflex

Berge, Z., & Mrozowski, S. (2001). Review of research in distance education, 1990 to 1999. American Journal of Distance Education, 15(3), 5–19. https://doi.org/ 10.1080/08923640109527090

Bond, M. (2020). Schools and emergency remote education during the COVID-19 pandemic: A living rapid systematic review. Asian Journal of Distance Education, 15(2), 191-247. http://www.asianjde.com/ojs/index.php/AsianJDE/article/view/517

Bond, M., Bedenlier, S., Marín, V. I., & Händel, M. (2021). Emergency remote teaching in higher education: Mapping the first global online semester. International Journal of Educational Technology in Higher Education, 18(1), 1-24. https://doi.org/10.1186/s41239-021-00282-x

Bonk, C. J. (2020). Pandemic ponderings, 30 years to today: Synchronous signals, saviors, or survivors? Distance Education, 41(4), 589-599. https://doi.org/10.1080/01587919.2020.1821610

Bonk, C. J., & Graham, C. R. (Eds.) (2006). Handbook of blended learning: Global perspectives, local designs. Pfeiffer Publishing.

Bonk, C. J., Olson, T., Wisher, R. A., & Orvis, K. L. (2002). Learning from focus groups: An examination of blended learning. Journal of Distance Education, 17(3), 97-118.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Braun, V., Clarke, V., & Rance, N. (2014). How to use thematic analysis with interview data. In A. Vossler & N. Moller (Eds.), The counselling & psychotherapy research handbook, 183–197. Sage.

Canales-Romero, D., & Hachfeld, A (2021). Juggling school and work from home: Results from a survey on German families with school-aged children during the early COVID-19 lockdown. Frontiers in Psychology, 12. https://doi.org/10.3389/fpsyg.2021.734257

Cao, Y., Zhang, S., Chan, M.C.E., Kang. Y. (2021). Post-pandemic reflections: lessons from Chinese mathematics teachers about online mathematics instruction. Asia Pacific Education Review, 22, 157–168. https://doi.org/10.1007/s12564-021-09694-w

The Centers for Disease Control and Prevention (2022, May 4). COVID-19 forecasts: Deaths. Retrieved from https://www.cdc.gov/coronavirus/2019-ncov/science/forecasting/forecasting-us.html

Chang, H. M, & Kim. H. J. (2021). Predicting the pass probability of secondary school students taking online classes. Computers & Education, 164, 104110. https://doi.org/10.1016/j.compedu.2020.104110

Charumilind, S. Craven, M., Lamb, J., Sabow, A., Singhal, S., & Wilson, M. (2022, March 1). When will the COVID-19 pandemic end? McKinsey & Company. https://www.mckinsey.com/industries/healthcare-systems-and-services/our-insights/when-will-the-covid-19-pandemic-end

Cooper, H. (1988). The structure of knowledge synthesis: A taxonomy of literature reviews. Knowledge in Society, 1, 104–126.

Crompton, H., Burke, D., Jordan, K., & Wilson, S. W. (2021). Learning with technology during emergencies: A systematic review of K‐12 education. British Journal of Educational Technology, 52(4), 1554-1575. https://doi.org/10.1111/bjet.13114

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340. https://doi.org/10.2307/249008

Erwin, B. (2021, November). A policymaker’s guide to virtual schools. Education Commission of the States. https://www.ecs.org/wp-content/uploads/Policymakers-Guide-to-Virtual-Schools.pdf

Gross, B. (2021). Surging enrollment in virtual schools during the pandemic spurs new questions for policymakers. Center on Reinventing Public Education, Arizona State University. https://crpe.org/surging-enrollment-in-virtual-schools-during-the-pandemic-spurs-new-questions-for-policymakers/

Hamaidi, D. D. A., Arouri, D. Y. M., Noufal, R. K., & Aldrou, I. T. (2021). Parents’ perceptions of their children’s experiences with distance learning during the COVID-19 pandemic. The International Review of Research in Open and Distributed Learning, 22(2), 224-241. https://doi.org/10.19173/irrodl.v22i2.5154

Heo, H., Bonk, C. J., & Doo, M. Y. (2022). Influences of depression, self-efficacy, and resource management on learning engagement in blended learning during COVID-19. The Internet and Higher Education, 54, https://doi.org/10.1016/j.iheduc.2022.100856

Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020, March 27). The differences between emergency remote teaching and online learning. EDUCAUSE Review. https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teachingand-online-learning

Huang, L. & Zhang, T. (2021). Perceived social support, psychological capital, and subjective well-being among college students in the context of online learning during the COVID-19 pandemic. Asia-Pacific Education Researcher. https://doi.org/10.1007/s40299-021-00608-3

Kanwar, A., & Daniel, J. (2020). Report to Commonwealth education ministers: From response to resilience. Commonwealth of Learning. http://oasis.col.org/handle/11599/3592

Lederman, D. (2019). Online enrollments grow, but pace slows. Inside Higher Ed. https://www.insidehighered.com/digital-learning/article/2019/12/11/more-students-study-online-rate-growth-slowed-2018

Lee, K. (2019). Rewriting a history of open universities: (Hi)stories of distance teachers. The International Review of Research in Open and Distributed Learning, 20(4), 1-12. https://doi.org/10.19173/irrodl.v20i3.4070

Liu, Y., & Butzlaff, A. (2021). Where's the germs? The effects of using virtual reality on nursing students' hospital infection prevention during the COVID-19 pandemic. Journal of Computer Assisted Learning, 37(6), 1622–1628. https://doi.org/10.1111/jcal.12601

Maloney, E. J., & Kim, J. (2020, June 10). Learning in 2050. Inside Higher Ed. https://www.insidehighered.com/digital-learning/blogs/learning-innovation/learning-2050

Martin, F., Sun, T., & Westine, C. D. (2020). A systematic review of research on online teaching and learning from 2009 to 2018. Computers & Education, 159, 104009.

Miks, J., & McIlwaine, J. (2020, April 20). Keeping the world’s children learning through COVID-19. UNICEF. https://www.unicef.org/coronavirus/keeping-worlds-children-learning-through-covid-19

Mishra, S., Sahoo, S., & Pandey, S. (2021). Research trends in online distance learning during the COVID-19 pandemic. Distance Education, 42(4), 494-519. https://doi.org/10.1080/01587919.2021.1986373

Moore, M. G. (Ed.) (2007). The handbook of distance education (2nd Ed.). Lawrence Erlbaum Associates.

Moore, M. G., & Kearsley, G. (2012). Distance education: A systems view (3rd ed.). Wadsworth.

Munir, F., Anwar, A., & Kee, D. M. H. (2021). The online learning and students’ fear of COVID-19: Study in Malaysia and Pakistan. The International Review of Research in Open and Distributed Learning, 22(4), 1-21. https://doi.org/10.19173/irrodl.v22i4.5637

National Center for Education Statistics (2015). Number of virtual schools by state and school type, magnet status, charter status, and shared-time status: School year 2013–14. https://nces.ed.gov/ccd/tables/201314_Virtual_Schools_table_1.asp

National Center for Education Statistics (2020). Number of virtual schools by state and school type, magnet status, charter status, and shared-time status: School year 2018–19. https://nces.ed.gov/ccd/tables/201819_Virtual_Schools_table_1.asp

National Center for Education Statistics (2021). Number of virtual schools by state and school type, magnet status, charter status, and shared-time status: School year 2019–20. https://nces.ed.gov/ccd/tables/201920_Virtual_Schools_table_1.asp

Nguyen T., Netto, C.L.M., Wilkins, J.F., Bröker, P., Vargas, E.E., Sealfon, C.D., Puthipiroj, P., Li, K.S., Bowler, J.E., Hinson, H.R., Pujar, M. & Stein, G.M. (2021). Insights into students’ experiences and perceptions of remote learning methods: From the COVID-19 pandemic to best practice for the future. Frontiers in Education, 6, 647986. doi: 10.3389/feduc.2021.647986

Oinas, S., Hotulainen, R., Koivuhovi, S., Brunila, K., & Vainikainen, M-P. (2022). Remote learning experiences of girls, boys and non-binary students. Computers & Education, 183, [104499]. https://doi.org/10.1016/j.compedu.2022.104499

Park, A. (2022, April 29). The U.S. is in a 'Controlled Pandemic' Phase of COVID-19. But what does that mean? Time. https://time.com/6172048/covid-19-controlled-pandemic-endemic/

Petersen, G. B. L., Petkakis, G., & Makransky, G. (2022). A study of how immersion and interactivity drive VR learning. Computers & Education, 179, 104429, https://doi.org/10.1016/j.compedu.2021.104429

Picciano, A., Dziuban, C., & Graham, C. R. (Eds.) (2014). Blended learning: Research perspectives, Volume 2. Routledge.

Picciano, A., Dziuban, C., Graham, C. R. & Moskal, P. (Eds.) (2022). Blended learning: Research perspectives, Volume 3. Routledge.

Pollard, R., & Kumar, S. (2021). Mentoring graduate students online: Strategies and challenges. The International Review of Research in Open and Distributed Learning, 22(2), 267-284. https://doi.org/10.19173/irrodl.v22i2.5093

Salis-Pilco, S. Z., Yang. Y., Zhang. Z. (2022). Student engagement in online learning in Latin American higher education during the COVID-19 pandemic: A systematic review. British Journal of Educational Technology, 53(3), 593-619. https://doi.org/10.1111/bjet.13190

Shen, Y. W., Reynolds, T. H., Bonk, C. J., & Brush, T. A. (2013). A case study of applying blended learning in an accelerated post-baccalaureate teacher education program. Journal of Educational Technology Development and Exchange, 6(1), 59-78.

Seabra, F., Teixeira, A., Abelha, M., Aires, L. (2021). Emergency remote teaching and learning in Portugal: Preschool to secondary school teachers’ perceptions. Education Sciences, 11, 349. https://doi.org/ 10.3390/educsci11070349

Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, S., Ahern, T. C., Shaw, S. M., & Liu, X. (2006). Teaching courses online: A review of the research. Review of Educational Research, 76(1), 93–135. https://doi.org/10.3102/00346543076001093 .

Theirworld. (2020, March 20). Hundreds of millions of students now learning from home after coronavirus crisis shuts their schools. ReliefWeb. https://reliefweb.int/report/world/hundreds-millions-students-now-learning-home-after-coronavirus-crisis-shuts-their

UNESCO (2020). UNESCO rallies international organizations, civil society and private sector partners in a broad Coalition to ensure #LearningNeverStops. https://en.unesco.org/news/unesco-rallies-international-organizations-civil-society-and-private-sector-partners-broad

VanLeeuwen, C. A., Veletsianos, G., Johnson, N., & Belikov, O. (2021). Never-ending repetitiveness, sadness, loss, and “juggling with a blindfold on:” Lived experiences of Canadian college and university faculty members during the COVID-19

pandemic. British Journal of Educational Technology, 52, 1306-1322

https://doi.org/10.1111/bjet.13065

Wedemeyer, C. A. (1981). Learning at the back door: Reflections on non-traditional learning in the lifespan. University of Wisconsin Press.

Zawacki-Richter, O., Backer, E., & Vogt, S. (2009). Review of distance education research (2000 to 2008): Analysis of research areas, methods, and authorship patterns. International Review of Research in Open and Distance Learning, 10(6), 30. https://doi.org/10.19173/irrodl.v10i6.741

Zhan, Z., Li, Y., Yuan, X., & Chen, Q. (2021). To be or not to be: Parents’ willingness to send their children back to school after the COVID-19 outbreak. The Asia-Pacific Education Researcher. https://doi.org/10.1007/s40299-021-00610-9

As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/ .

This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.

Author(s) hold copyright in their work, and retain publishing rights without restrictions

The DOAJ Seal is awarded to journals that demonstrate best practice in open access publishing

OLC Membership

OLC Research Center

Information

• For Readers
• For Authors
• For Librarians

The COVID-19 pandemic has changed education forever. This is how 

With schools shut across the world, millions of children have had to adapt to new types of learning. Image:  REUTERS/Gonzalo Fuentes

.chakra .wef-1c7l3mo{-webkit-transition:all 0.15s ease-out;transition:all 0.15s ease-out;cursor:pointer;-webkit-text-decoration:none;text-decoration:none;outline:none;color:inherit;}.chakra .wef-1c7l3mo:hover,.chakra .wef-1c7l3mo[data-hover]{-webkit-text-decoration:underline;text-decoration:underline;}.chakra .wef-1c7l3mo:focus,.chakra .wef-1c7l3mo[data-focus]{box-shadow:0 0 0 3px rgba(168,203,251,0.5);} Cathy Li

Farah lalani.

.chakra .wef-9dduvl{margin-top:16px;margin-bottom:16px;line-height:1.388;font-size:1.25rem;}@media screen and (min-width:56.5rem){.chakra .wef-9dduvl{font-size:1.125rem;}} Explore and monitor how .chakra .wef-15eoq1r{margin-top:16px;margin-bottom:16px;line-height:1.388;font-size:1.25rem;color:#F7DB5E;}@media screen and (min-width:56.5rem){.chakra .wef-15eoq1r{font-size:1.125rem;}} COVID-19 is affecting economies, industries and global issues

.chakra .wef-1nk5u5d{margin-top:16px;margin-bottom:16px;line-height:1.388;color:#2846F8;font-size:1.25rem;}@media screen and (min-width:56.5rem){.chakra .wef-1nk5u5d{font-size:1.125rem;}} Get involved with our crowdsourced digital platform to deliver impact at scale

Stay up to date:.

• The COVID-19 has resulted in schools shut all across the world. Globally, over 1.2 billion children are out of the classroom.
• As a result, education has changed dramatically, with the distinctive rise of e-learning, whereby teaching is undertaken remotely and on digital platforms.
• Research suggests that online learning has been shown to increase retention of information, and take less time, meaning the changes coronavirus have caused might be here to stay.

While countries are at different points in their COVID-19 infection rates, worldwide there are currently more than 1.2 billion children in 186 countries affected by school closures due to the pandemic. In Denmark, children up to the age of 11 are returning to nurseries and schools after initially closing on 12 March , but in South Korea students are responding to roll calls from their teachers online .

With this sudden shift away from the classroom in many parts of the globe, some are wondering whether the adoption of online learning will continue to persist post-pandemic, and how such a shift would impact the worldwide education market.

Even before COVID-19, there was already high growth and adoption in education technology, with global edtech investments reaching US$18.66 billion in 2019 and the overall market for online education projected to reach $350 Billion by 2025 . Whether it is language apps , virtual tutoring , video conferencing tools, or online learning software , there has been a significant surge in usage since COVID-19.

How is the education sector responding to COVID-19?

In response to significant demand, many online learning platforms are offering free access to their services, including platforms like BYJU’S , a Bangalore-based educational technology and online tutoring firm founded in 2011, which is now the world’s most highly valued edtech company . Since announcing free live classes on its Think and Learn app, BYJU’s has seen a 200% increase in the number of new students using its product, according to Mrinal Mohit, the company's Chief Operating Officer.

Tencent classroom, meanwhile, has been used extensively since mid-February after the Chinese government instructed a quarter of a billion full-time students to resume their studies through online platforms. This resulted in the largest “online movement” in the history of education with approximately 730,000 , or 81% of K-12 students, attending classes via the Tencent K-12 Online School in Wuhan.

Have you read?

The future of jobs report 2023, how to follow the growth summit 2023.

Other companies are bolstering capabilities to provide a one-stop shop for teachers and students. For example, Lark, a Singapore-based collaboration suite initially developed by ByteDance as an internal tool to meet its own exponential growth, began offering teachers and students unlimited video conferencing time, auto-translation capabilities, real-time co-editing of project work, and smart calendar scheduling, amongst other features. To do so quickly and in a time of crisis, Lark ramped up its global server infrastructure and engineering capabilities to ensure reliable connectivity.

Alibaba’s distance learning solution, DingTalk, had to prepare for a similar influx: “To support large-scale remote work, the platform tapped Alibaba Cloud to deploy more than 100,000 new cloud servers in just two hours last month – setting a new record for rapid capacity expansion,” according to DingTalk CEO, Chen Hang.

Some school districts are forming unique partnerships, like the one between The Los Angeles Unified School District and PBS SoCal/KCET to offer local educational broadcasts, with separate channels focused on different ages, and a range of digital options. Media organizations such as the BBC are also powering virtual learning; Bitesize Daily , launched on 20 April, is offering 14 weeks of curriculum-based learning for kids across the UK with celebrities like Manchester City footballer Sergio Aguero teaching some of the content.

What does this mean for the future of learning?

While some believe that the unplanned and rapid move to online learning – with no training, insufficient bandwidth, and little preparation – will result in a poor user experience that is unconducive to sustained growth, others believe that a new hybrid model of education will emerge, with significant benefits. “I believe that the integration of information technology in education will be further accelerated and that online education will eventually become an integral component of school education,“ says Wang Tao, Vice President of Tencent Cloud and Vice President of Tencent Education.

There have already been successful transitions amongst many universities. For example, Zhejiang University managed to get more than 5,000 courses online just two weeks into the transition using “DingTalk ZJU”. The Imperial College London started offering a course on the science of coronavirus, which is now the most enrolled class launched in 2020 on Coursera .

Many are already touting the benefits: Dr Amjad, a Professor at The University of Jordan who has been using Lark to teach his students says, “It has changed the way of teaching. It enables me to reach out to my students more efficiently and effectively through chat groups, video meetings, voting and also document sharing, especially during this pandemic. My students also find it is easier to communicate on Lark. I will stick to Lark even after coronavirus, I believe traditional offline learning and e-learning can go hand by hand."

These 3 charts show the global growth in online learning

The challenges of online learning.

There are, however, challenges to overcome. Some students without reliable internet access and/or technology struggle to participate in digital learning; this gap is seen across countries and between income brackets within countries. For example, whilst 95% of students in Switzerland, Norway, and Austria have a computer to use for their schoolwork, only 34% in Indonesia do, according to OECD data .

In the US, there is a significant gap between those from privileged and disadvantaged backgrounds: whilst virtually all 15-year-olds from a privileged background said they had a computer to work on, nearly 25% of those from disadvantaged backgrounds did not. While some schools and governments have been providing digital equipment to students in need, such as in New South Wales , Australia, many are still concerned that the pandemic will widenthe digital divide .

Is learning online as effective?

For those who do have access to the right technology, there is evidence that learning online can be more effective in a number of ways. Some research shows that on average, students retain 25-60% more material when learning online compared to only 8-10% in a classroom. This is mostly due to the students being able to learn faster online; e-learning requires 40-60% less time to learn than in a traditional classroom setting because students can learn at their own pace, going back and re-reading, skipping, or accelerating through concepts as they choose.

Nevertheless, the effectiveness of online learning varies amongst age groups. The general consensus on children, especially younger ones, is that a structured environment is required , because kids are more easily distracted. To get the full benefit of online learning, there needs to be a concerted effort to provide this structure and go beyond replicating a physical class/lecture through video capabilities, instead, using a range of collaboration tools and engagement methods that promote “inclusion, personalization and intelligence”, according to Dowson Tong, Senior Executive Vice President of Tencent and President of its Cloud and Smart Industries Group.

Since studies have shown that children extensively use their senses to learn, making learning fun and effective through use of technology is crucial, according to BYJU's Mrinal Mohit. “Over a period, we have observed that clever integration of games has demonstrated higher engagement and increased motivation towards learning especially among younger students, making them truly fall in love with learning”, he says.

A changing education imperative

It is clear that this pandemic has utterly disrupted an education system that many assert was already losing its relevance . In his book, 21 Lessons for the 21st Century , scholar Yuval Noah Harari outlines how schools continue to focus on traditional academic skills and rote learning , rather than on skills such as critical thinking and adaptability, which will be more important for success in the future. Could the move to online learning be the catalyst to create a new, more effective method of educating students? While some worry that the hasty nature of the transition online may have hindered this goal, others plan to make e-learning part of their ‘new normal’ after experiencing the benefits first-hand.

The importance of disseminating knowledge is highlighted through COVID-19

Major world events are often an inflection point for rapid innovation – a clear example is the rise of e-commerce post-SARS . While we have yet to see whether this will apply to e-learning post-COVID-19, it is one of the few sectors where investment has not dried up . What has been made clear through this pandemic is the importance of disseminating knowledge across borders, companies, and all parts of society. If online learning technology can play a role here, it is incumbent upon all of us to explore its full potential.

Our education system is losing relevance. Here's how to unleash its potential

3 ways the coronavirus pandemic could reshape education, celebrities are helping the uk's schoolchildren learn during lockdown, don't miss any update on this topic.

Create a free account and access your personalized content collection with our latest publications and analyses.

License and Republishing

World Economic Forum articles may be republished in accordance with the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License, and in accordance with our Terms of Use.

The views expressed in this article are those of the author alone and not the World Economic Forum.

Related topics:

The agenda .chakra .wef-n7bacu{margin-top:16px;margin-bottom:16px;line-height:1.388;font-weight:400;} weekly.

A weekly update of the most important issues driving the global agenda

.chakra .wef-1dtnjt5{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;} More on COVID-19 .chakra .wef-17xejub{-webkit-flex:1;-ms-flex:1;flex:1;justify-self:stretch;-webkit-align-self:stretch;-ms-flex-item-align:stretch;align-self:stretch;} .chakra .wef-nr1rr4{display:-webkit-inline-box;display:-webkit-inline-flex;display:-ms-inline-flexbox;display:inline-flex;white-space:normal;vertical-align:middle;text-transform:uppercase;font-size:0.75rem;border-radius:0.25rem;font-weight:700;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;line-height:1.2;-webkit-letter-spacing:1.25px;-moz-letter-spacing:1.25px;-ms-letter-spacing:1.25px;letter-spacing:1.25px;background:none;padding:0px;color:#B3B3B3;-webkit-box-decoration-break:clone;box-decoration-break:clone;-webkit-box-decoration-break:clone;}@media screen and (min-width:37.5rem){.chakra .wef-nr1rr4{font-size:0.875rem;}}@media screen and (min-width:56.5rem){.chakra .wef-nr1rr4{font-size:1rem;}} See all

Winding down COVAX – lessons learnt from delivering 2 billion COVID-19 vaccinations to lower-income countries

Charlotte Edmond

January 8, 2024

Here’s what to know about the new COVID-19 Pirola variant

October 11, 2023

How the cost of living crisis affects young people around the world

Douglas Broom

August 8, 2023

From smallpox to COVID: the medical inventions that have seen off infectious diseases over the past century

Andrea Willige

May 11, 2023

COVID-19 is no longer a global health emergency. Here's what it means

Simon Nicholas Williams

May 9, 2023

New research shows the significant health harms of the pandemic 

Philip Clarke, Jack Pollard and Mara Violato

April 17, 2023

• Open access
• Published: 07 April 2024

Efficacy of a virtual nursing simulation-based education to provide psychological support for patients affected by infectious disease disasters: a randomized controlled trial

• Eunjung Ko 1 &
• Yun-Jung Choi 1  

BMC Nursing volume  23 , Article number:  230 ( 2024 ) Cite this article

138 Accesses

1 Altmetric

Metrics details

Virtual simulation-based education for healthcare professionals has emerged as a strategy for dealing with infectious disease disasters, particularly when training at clinical sites is restricted due to the risk of infection and a lack of personal protective equipment. This research evaluated a virtual simulation-based education program intended to increase nurses’ perceived competence in providing psychological support to patients affected by infectious disease disasters.

The efficacy of the program was evaluated via a randomized controlled trial. We recruited 104 nurses for participation in the study and allocated them randomly and evenly to an experimental group and a control group. The experimental group was given a web address through which they could access the program, whereas the control group was provided with a web address that directed them to text-based education materials. Data were then collected through an online survey of competence in addressing disaster mental health, after which the data were analyzed using the Statistical Package for the Social Sciences(version 23.0).

The analysis showed that the experimental group’s disaster mental health competence (F = 5.149, p  =.026), problem solving process (t = 3.024, p  =.003), self-leadership (t = 2.063, p  =.042), learning self-efficacy (t = 3.450, p  =.001), and transfer motivation (t = 2.095, p  =.039) significantly statistically differed from those of the control group.

Conclusions

A virtual nursing simulation-based education program for psychological support can overcome limitations of time and space. The program would also be an effective learning resource during infectious disease outbreaks.

Clinical trial registration

This Korean clinical trial was retrospectively registered (21/11/2023) in the Clinical Research Information Service ( https://cris.nih.go.kr ) with trial registration number KCT0008965.

Peer Review reports

The last two decades have confronted the world with a variety of infectious diseases, such as severe acute respiratory syndrome, which first occurred in Asia in 2003 before spreading worldwide, including Korea, in only a few months. Since then, infectious disease outbreaks began to be recognized as severe disasters. Other examples include the 2009 H1N1 influenza outbreak, which caused more than 10,000 deaths worldwide and 140 deaths in Korea; the proliferation of the Ebola virus, which resulted in a fatality rate of more than 90% in Africa in 2014; and the outbreak of Middle East respiratory syndrome in 2015, Zika virus disease in 2016, and coronavirus disease (COVID-19) in 2019 [ 1 ]. The COVID-19 pandemic, in particular, has caused infections among approximately 64 million people and the deaths of 1.5 million individuals as of December 2020 [ 2 ].

Direct victims of infectious disease disasters, infected patients, and quarantined individuals suffer from a fear of stigma or social blame and guilt, but even people who are unexposed to sources of infection experience psychological distress from anxiety and fear of disease or possible death [ 3 ]. They also blame infected people and harbor hatred toward them [ 3 ]. This assertion is supported by an examination of web search behaviors and infodemic attitudes toward COVID-19, which identified superficial and racist attitudes [ 4 ]. Additionally, in research using a health stigma and discrimination framework related to communicable diseases, the authors found that people exhibit negative stereotypes, biases, and discriminatory conduct toward infected groups owing to fears of contagion, concerns about potential harm, and perceptions that individuals violate central values [ 5 ]. Stigmatized individuals experience adverse effects on their health because of both the stress induced by stigma and the decreased use of available services [ 5 ].

Severe and prolonged anxiety, fear, blame, and aggression can lead to mental health problems, including depression, anxiety, panic attacks, somatic symptoms, post-traumatic stress disorder, psychosis, and even suicide and life-threatening behaviors [ 6 ]. Therefore, recovery from the psychological trauma caused by a disaster should be regarded as equally necessary as physical recovery, with emphasis placed on psychological support activities that prevent the deterioration of mental health [ 7 ].

Disasters pose a significant threat to mental health support systems, wherein the lack of healthcare professionals or psychologists trained to address these conditions exacerbates the psychological distress and psychopathological risk experienced by society [ 8 ]. When training at clinical sites is restricted due to infection risks and a lack of personal protective equipment (PPE), an emerging solution is virtual simulation [ 9 ].

A virtual simulation is a simulation modality developed on the basis of video or graphic recordings featuring virtual patients and delivered via either a static or mobile device. It replicates real-world clinical situations and affords learners an interactive experience [ 10 ]. Virtual simulation-based education provides an immersive clinical environment, as virtual patients respond to a learner’s assessments and interventions [ 11 , 12 ]. It enables two-way communication, and allows medical professionals to practice making clinical decisions [ 10 ]. Virtual patients are equipped with voice, intonation, and expressions that reinforce the educational narrative within the virtual environment, thereby enhancing the effectiveness of the learning experience [ 13 ]. One of the primary advantages of virtual simulation-based education is its provision of a safe and non-threatening environment in which learners can practice. It also offers flexible and reproducible learning experiences, thus catering to the diverse needs of learners [ 14 ].

Self-assessment is the most commonly used competence evaluation tool, as it is cost-effective and helps nurses improve their practice by identifying their strengths and weaknesses for development [ 15 ]. Self-assessed competence is also related to the quality of patient care because nurses promote continuous learning by determining educational needs through such evaluations [ 16 ]. The competence perceived by a nurse is inherently subjective given its self-reported nature and poses a challenge in establishing a direct correlation with the actual care of patients [ 17 , 18 ]. However, studies have indicated that increased levels of self-perceived competence are associated with a significant increase in core competencies related to patient care and frequent use of clinical skills [ 19 , 20 ]. Perceived competence likewise influences the job satisfaction and organizational citizenship behavior of nurses and is significantly related to absenteeism, one of the deterrents to the delivery of quality care [ 21 , 22 ].

Competence refers to the possession of qualifications and abilities to satisfy professional standards, as well as the capability to perform tasks and duties in a suitable and effective manner [ 23 ]. Competencies for disaster mental health are crucial for enhancing disaster response capabilities. These competencies encompass a range of skills, knowledge, and attitudes necessary for mental health professionals to effectively support individuals and communities affected by disasters [ 24 ]. Such competencies and how they are affected by simulation-based training have been explored in some studies, which reported a significant increase in competence after exposure to the aforementioned education [ 25 , 26 ].

The simulation education defined in mock training designs based on real situations provides opportunities to exercise problem-solving through various strategies. Problem-solving process is considered key competency through which learners are expected to enhance their relevant knowledge and clinical performance abilities [ 27 ]. In particular, problem-solving processes for identifying and assessing problems and finding solutions are psychological strategies that help people cope and recover after a disaster [ 28 ]. A scoping review on the effect of simulation-based education on the problem-solving process indicated that out of 32 studies reviewed, 21 demonstrated statistically significant improvement in people’s ability to resolve problems [ 29 ].

Simulation training can also address self-leadership, which is an essential self-learning quality that aids individuals in staying motivated and focused on their learning goals. It is also required as a basic qualification of professional nurses, who must be able to take initiative and make responsible decisions [ 30 , 31 ]. Previous studies have reported statistically significant improvements in self-leadership following simulation training [ 32 , 33 ].

Another aspect that benefits from simulation-driven education is learning self-efficacy, which plays a crucial role in predicting learners’ levels of engagement and academic success in online education. It reflects learners’ confidence in their ability to manage their own learning process. It is a significant predictor of both learners’ participation levels and their academic achievements in online education settings [ 34 , 35 ]. Several studies have demonstrated virtual simulation- or online education-induced significant improvements in learning self-efficacy [ 36 , 37 ]. Finally, virtual simulation-based education can also improve the motivation to transfer new knowledge and skills learned through education to clinical practice [ 38 ]. This motivation is considered an essential measure of effective learning for nurses working in the clinical field [ 38 ]. A previous study reported that psychiatric nursing simulation training combined with post-course debriefing significantly increases participants’ level of motivation to transfer [ 38 ].

On the basis of the discussion above, this study evaluated a virtual nursing simulation-based education program on disaster psychology designed to provide psychological support to patients affected by infectious disease disasters.

Study design

This study conducted a randomized controlled trial (RCT) to test the virtual nursing simulation-based education program of interest. The RCT protocol used was based on CONSORT guidelines.

Participants

We recruited nurses working at general hospitals in South Korea. With permission from the nurse managers of these hospitals, a participation notice was posted on the institutions’ internet bulletin boards for nurses for a week. The two-sided test criterion, with a significance level (α) of 0.05, a power (1-β) of 0.80, and a medium effect size of 0.6, dictates that the minimum number of participants per group be 90. The effect size was based on a virtual simulation intervention study conducted by Kim and Choi [ 36 ]. Taking the dropout rate into consideration, we recruited 104 nurses, who were assigned to an experimental group and a control group using the random sampling functionality of the Statistical Package for the Social Sciences (SPSS version 23.0). Out of the initial sample, 11 participants were excluded because they were on vacation, could not be contacted, or provided incomplete responses during data collection (Fig.  1 ).

Flowchart of the randomized controlled trial

The virtual nursing simulation-based education program

This study probed into the virtual nursing simulation-based education program developed by Ko [ 39 ]. The program is implemented using an e-learning development platform, Articulate Storyline, whose operating environment is compatible with all web browsers (Internet Explorer, Microsoft Edge, Firefox, Google Chrome, etc.). It is a mobile-friendly application that can run in devices with Android and iOS operating systems. When an individual uses their smartphone or personal computer to access the server via the web address corresponding to the education program, the content functions execute. Ko’s [ 39 ] program involves five stages of learning completed in 100 min: (1) preparatory learning (30 min), (2) pre-test (5 min), (3) pre-briefing (5 min), (4) simulation game (30 min), and (5) structured self-debriefing (30 min) (Fig.  2 ).

Preparatory learning comes with lecture materials on guidelines for providing psychological support to victims of infectious disease disasters, administering psychological first aid, donning and doffing PPE, and exercising mindfulness through videos and pictures. In the pretest stage, a learner answers five questions and can immediately check the correct responses, which come with detailed explanations. In the prebriefing stage, an overview of a nursing simulation scenario, patient information, learning objectives, and instructions on using the virtual simulation are provided. During the simulation game, a video of the simulation is presented. It starts with a 39-year-old female, a standardized patient who is age- and gender-matched to the scenario, confirmed to have contracted COVID-19 and transferred to a negative pressure isolation room. The patient presents with extreme anxiety and feeling of tightness in her chest. During the game, learners are expected to complete 12 quizzes. In the debriefing stage, a summary of the simulation quiz results and self-debriefing questions are provided, and the comments made by learners are saved in the Naver cloud platform.

The evaluated virtual nursing simulation-based education program (examples are our own work)

Measurements

Disaster mental health competence.

Disaster mental health competence was measured using the perceived competence scale for disaster mental health workforce (PCS-DMHW), which was developed by Yoon and Choi [ 40 ]. This tool consists of 24 questions related to knowledge (6 questions), attitudes (9 questions), and skills (9 questions). Each item is rated using a five-point Likert scale (0 = strongly disagree, 4 = strongly agree), and the responses are summed. The higher the score, the greater the perception of competence in a relevant area [ 40 ]. The Cronbach’s α values of the PCS-DMHW were 0.95 and 0.94 at the time of tool development and the present study, respectively.

Problem solving process

Problem solving process was determined using a tool modified and supplemented by Park and Woo [ 41 ] on the grounds of the problem solving process and behavior survey developed by Lee [ 42 ]. This tool is composed of 25 questions on five factors, namely, problem discovery, problem definition, problem solution design, problem solution execution, and problem solving review [ 41 ]. The reliability of the tool was 0.89 at the time of development [ 41 ], but the Cronbach’s α found in the current research was 0.94.

Self-leadership

Self-leadership was measured using a tool developed by Manz [ 43 ] and modified by Kim [ 44 ]. The tool consists of 18 questions distributed over six factors (three questions each): self-defense, rehearsal, goal setting, self-compensation, self-expense edition, and constructive thinking. The reliability of the tool at the time of development and the present research was (Cronbach’s α) 0.87 and 0.82, respectively.

Learning self-efficacy

To ascertain learning self-efficacy, we used the tool developed by Ayres [45] and translated by Park and Kweon [ 38 ]. This tool consists of 10 questions, and it had a reliability (Cronbach’s ⍺) of 0.94 and 0.93 at the time of development and the current study, respectively.

Motivation to transfer

We used Ayres’s [45] motivation to transfer scale, which was translated by Park and Kweon [ 38 ]. Its reliability at the time of development and the present research was (Cronbach’s ⍺) 0.80 and 0.93, respectively.

Data collection

The experimental and control groups were administered a pretest through an online survey. The web address through which the evaluated virtual simulation-based education program could be accessed was provided to the experimental group, whereas text-based education materials on psychological support for victims of infectious disease disasters were given to the control group. The groups were simultaneously sent the program’s instruction manual, and their inquiries were answered through chat. After the interventions, each participant was administered a posttest through another online survey.

Data analysis

The collected data were analyzed using SPSS version 23.0. The homogeneity test for general characteristics between the experimental and control groups was analyzed using a t-test, a chi-square test, and Fisher’s exact test. The normality of the dependent variables was analyzed using the Kolmogorov-Smirnov test. Changes in the dependent variables between the pretest and posttest were analyzed using a paired t-test. Differences in the dependent variables before and after the groups’ use of the interventions were examined via a t-test and ANCOVA.

Ethical considerations

We completed education in bioethics law prior to the research and obtained approval of the research proposal and questionnaire from the Institutional Review Board of the affiliated university (IRB approval number 1041078-202003-HRSB-070-01CC). A signed consent form was also obtained from each participant after the purpose and methods of the research, the confidentiality of personal information, and the voluntary nature of participation or their right to withdraw from the study were explained to them. All collected data were kept in a lockable cabinet, and electronic data were encrypted and stored. These data are to be discarded after three years.

A total of 93 participants (45 in the experimental group and 48 in the control group) were left after the exclusion of unsuitable respondents. of the between-group comparisons of the subjects indicated no significant differences between them (5% significance level) in terms of general characteristics, such as gender, age, work unit, and clinical experience (Table  1 ).

The score of the experimental group on disaster mental health competence increased from 48.13 in the pretest to 70.51 in the posttest (+ 22.38), whereas that of the control group increased from 53.33 in the pretest to 68.38 in the posttest (+ 15.04). These findings reflect a statistically significant difference in competence between the groups (F = 5.149, p  =.026). The scores of the experimental and control groups on problem solving process increased from 73.07 in the pretest to 88.24 in the posttest (+ 15.18) and from 75.75 in the pretest to 83.77 in the posttest (+ 8.02), respectively. As with the competence findings, these point to a significant difference between the groups in terms of the ability to resolve problems (t = 3.024, p  =.003) (Table  2 ).

The score of the experimental group on self-leadership increased from 54.87 in the pretest to 59.58 in the posttest (+ 4.71), and that of the control group increased from 57.48 in the pretest to 60.10 in the posttest (+ 2.63). These results denote a statistically significant difference in this ability between the groups (t = 2.063, p  =.042). The scores of the experimental and control participants on learning self-rose from 55.40 in the pretest to 58.84 in the posttest (+ 3.44) and from 56.81 in the pretest to 57.13 in the posttest (+ 0.31), respectively. Again, a statistically significant difference was found between the groups (t = 3.450, p  =.001). Their scores on motivation to transfer rose from 49.31 in the pretest to 54.29 in the posttest (+ 4.98) (experimental group) and the score increased from 50.50 in the pretest to 51.85 in the posttest (+ 1.35) (control group), pointing to a significant difference between the groups (t = 2.095, p  =.039).

As previously stated, this research was evaluated a virtual nursing simulation-based education program designed to provide psychological support to patients affected by infectious disease disasters. The results showed statistically significant increases in the experimental group’s pretest and posttest scores on disaster mental health competence, problem solving process, self-leadership, learning self-efficacy, and motivation to transfer.

The experimental group achieved more statistically significant improvements in disaster mental health competence than did the control group. This finding is similar to the statistically significant increase in the average disaster mental health competence shown by providers of disaster mental health services providers and non-expert groups after PFA training involving lecture and practice [ 46 ]. It is also consistent with the significant increase in the scores of school counselors on disaster mental health competence after a lecture and simulation on PFA [ 25 ]. In their study on disaster relief workers, Kang and Choi [ 26 ] measured the participants’ performance competence in PFA after the delivery of a lecture and simulation-based education using a standardized patient. The authors found a significant increase in PFA performance competence, consistent with the present research. Since there are currently no other virtual simulation-based education programs for disaster psychological support available, we compared the effectiveness of various PFA training methods with the program assessed in the present work.

In the current research, the posttest scores of the experimental group on problem solving process significantly increased, similar to the results of Kim et al.’s study on virtual simulation- and blended simulation-based education on asthmatic child nursing [ 47 ]. Both the control and experimental groups (virtual simulation only and blended simulation featuring high-fidelity and virtual simulations, respectively) showed an increase in their problem solving process scores. These results and those derived in the present work are similar because reading and pretest phases were incorporated into the design of the previous study. Given that researchers have used commercial virtual simulations featuring avatars rather than standardized patient videos available through English-based platforms, user experiences may differ, thus requiring a qualitative analysis to identify differences. However, Kim et al. [ 47 ] did not implement a debriefing after the virtual simulation program, rendering comparison impossible. Another research reported that a multimodality simulation education that combines such methods as virtual simulation, the use of mannequins, and part-task training increase increased the scores of hospital nurses’ on problem solving process [ 48 ].

In the present work, the experimental group’s self-leadership scores increased after they used the program, and these scores were higher [ 49 , 50 ]. This difference can be explained by the fact that our respondents voluntarily participated in our research given their interest in self-learning programs for disaster psychological support; even in the comparison studies, participants with stronger interest in leadership education typically exhibited heightened degrees of self-leadership [ 51 ]. The increase in self-leadership scores in the current research is consistent with a previous study involving a two-hour simulation education about PPE donning and doffing, medication administration, and medical specimen treatment in a scenario of patients suspected of having infectious diseases [ 32 ]. Another research showed that simulation education on high-risk pregnancy enhances nursing students’ problem-solving processes and self-leadership [ 52 ].

Learning self-efficacy is a key variable that enables the prediction of learners’ degrees of participation in online education and the prediction of their academic achievements, as it points to the ability to manage their learning processes [ 34 , 53 ]. The results of the current research in this regard are consistent with those of a study on the online practice of basic nursing skills, which increased participants’ learning self-efficacy [ 54 ]. The researchers included an online quiz about basic nursing skills and feedback sections for learners’ self-evaluations of their performance as avenues through which to encourage autonomy in learning. A similar approach was used in the present study, which involved both a pretest for self-evaluation, direct feedback on the virtual simulation, and a self-debriefing session, enabling the participants to reflect on their simulation experiences while reviewing other participants’ answers during self-debriefing. These functions of the evaluated program were expected to factor importantly in the significant increase in the participants’ learning self-efficacy scores.

Many studies on practice education have examined participants’ motivations to transfer knowledge and skills alongside their learning self-efficacies. In the current research, the motivation to transfer scores of the experimental increased, and the difference between the two groups was statistically meaningful. This result is consistent with the findings of Park and Kweon on the simulation education about psychiatric nursing, during which post-course debriefing increased the participants’ average scores on motivation to transfer and learning self-efficacy [ 38 ]. Conversely, Kang and Kim found that a six-week simulation program for alcoholic patient care did not generate a significant increase in the participants’ motivation to transfer and learning self-efficacy scores [ 55 ]. This finding was attributed to the unfamiliarity of the local community scenario used in the research to the participants, who were in their senior year of nursing school [ 55 ]. This limitation was overcome in the current research by administering a qualitative survey of nurses’ actual demand for education on psychological support for infectious disease patients. That is, the survey presented scenarios that the participants needed.

As with other studies, the present research was encumbered by several limitations. First, the self-assessment measures used in this study may be unreliable, because they are based on individuals’ subjective perceptions and interpretations of their abilities. There is also the possibility of respondent fatigue given that the participants were compelled to answer numerous questions. Future studies should incorporate both subjective and objective measures into data collection and consider as concise an evaluation method as possible to prevent respondent fatigue. Second, this study did not establish a direct link between the obtained results and actual changes in practice or improvements in patient outcomes. We propose a follow-up study to investigate the impact of the education program examined in this study on either the mental health of patients or the quality of patient care. Third, simulation-based education tends to be accompanied with more guidance than text-based program because the former has diverse components, including quiz games, and participants are predisposed to allocate more time to simulation-based education. These may potentially influence the results. In the future, we propose to conduct research by modifying education under the same time and guided condition.

This study proposed that a well-designed virtual nursing simulation-based education program can be an effective modality with which to satisfy the educational needs of nurses in the context of infectious disease outbreaks. Such programs can be easily used by nurses anywhere and anytime before they are deployed to provide psychological support to patients with infectious diseases. They are also expected to contribute to enhancing competence in addressing disaster mental health and improving the quality of care of patients afflicted with infectious diseases.

Data availability

The datasets used and/or analyzed in this study are available from the corresponding author upon reasonable request.

Abbreviations

Coronavirus disease 2019

Randomized controlled trial

Personal protective equipment

Statistical Package for the Social Sciences

Analysis of covariance

Psychological first aid

World Health Organization. WHO coronavirus disease (COVID-19) dashboard. 2020.

Oh EG, Yu SH, Kang JH, Oh YJ, Cha KI, Jeon YH, et al. Perspectives on nursing profession for a post-COVID-19 new normal. Korean J Adult Nurs. 2020;32(3):221–2.

Article   Google Scholar  

Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. Rapid review. Lancet. 2020;395:912–20.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Çalışkan C, Özsezer G, Pay M, Demir G, Çelebi I, Koçak H. Web search behaviors and infodemic attitudes regarding COVID-19 in Turkey: a framework study for improving response and informing policy on the COVID-19 infodemic. Front Public Health. 2022;10:948478.

Article   PubMed   PubMed Central   Google Scholar  

Stangl AL, Earnshaw VA, Logie CH, Van Brakel WC, Simbayi L, Barré I, et al. The health stigma and discrimination Framework: a global, crosscutting framework to inform research, intervention development, and policy on health-related stigmas. BMC Med. 2019;17:1–13.

Tucci V, Moukaddam N, Meadows J, Shah S, Galwankar SC, Kapur GB. The forgotten plague: psychiatric manifestations of Ebola, Zika, and emerging infectious diseases. J Global Infect Dis. 2017;9(4):151.

Korea Mental Health Technology R&D Project. (2017). Korean Disaster and Mental Health Support Guidelines.

Zhang J, Wu W, Zhao X, Zhang W. Recommended psychological crisis intervention response to the 2019 novel coronavirus pneumonia outbreak in China: a model of West China Hospital. Precision Clin Med. 2020;3(1):3–8.

Liu W. The effects of virtual simulation on undergraduate nursing students’ mental health literacy: a prospective cohort study. Issues Ment Health Nurs. 2020:1–10.

Cant R, Cooper S, Sussex R, Bogossian F. What’s in a name? Clarifying the nomenclature of virtual simulation. Clin Simul Nurs. 2019;27:26–30.

Foronda C, Bauman EB. Strategies to incorporate virtual simulation in nurse education. Clin Simul Nurs. 2014;10(8):412–812.

Berman NB, Durning SJ, Fischer MR, Huwendiek S, Triola MM. The role for virtual patients in the future of medical education. Acad Med. 2016;91(9):1217–22.

Article   PubMed   Google Scholar  

Verkuyl M, Hughes M, Tsui J, Betts L, St-Amant O, Lapum JL. Virtual gaming simulation in nursing education: a focus group study. J Nurs Educ. 2017;56(5):274–80.

Cobbett S, Snelgrove-Clarke E. Virtual versus face-to-face clinical simulation in relation to student knowledge, anxiety, and self-confidence in maternal-newborn nursing: a randomized controlled trial. Nurse Educ Today. 2016;45:179–84.

Bahreini M, Shahamat S, Hayatdavoudi P, Mirzaei M. Comparison of the clinical competence of nurses working in two university hospitals in Iran. Nurs Health Sci. 2011;13(3):282–8.

Goliroshan S, Babanataj R, Aziznejadroshan P. Investigating the self-assessment of clinical competency of nurses working in Babol University of Medical Sciences hospitals. Middle East J Family Med. 2018;7(10):279.

Google Scholar  

Bam V, Diji AKA, Asante E, Lomotey AY, Adade P, Akyeampong BA. Self-assessed competencies of nurses at an emergency department in Ghana. Afr J Emerg Med. 2020;10(1):8–12.

O’Leary J. Comparison of self-assessed competence and experience among critical care nurses. J Nurs Adm Manag. 2012;20(5):607–14. https://doi.org/10.1111/j.1365-2834.2012.01394.x .

Kim KM, Choi JS. Self-perceived competency of infection control nurses based on Benner’s framework: a nationwide survey in Korea. Appl Nurs Res. 2015;28(2):175–9.

Hassankhani H, Hasanzadeh F, Powers KA, Dadash Zadeh A, Rajaie R. Clinical skills performed by Iranian emergency nurses: perceived competency levels and attitudes toward expanding professional roles. J Emerg Nurs. 2018;44(2):156–63. https://doi.org/10.1016/j.jen.2017.06.007 .

Biagioli V, Prandi C, Nyatanga B, Fida R. The role of professional competency in influencing job satisfaction and organizational citizenship behavior among palliative care nurses. J Hospice Palliat Nurs. 2018;20(4):377–84.

Alreshidi NM, Alaseeri R, Garcia M. Factors influencing absenteeism among nursing staff in the primary health care centers in hail: a preliminary study for enhancing staff commitment. Health Sci J. 2019;13(3):658–1.

Rodolfa E, Bent R, Eisman E, Nelson P, Rehm L, Ritchie P. A cube model for competency development: implications for psychology educators and regulators. Prof Psychology: Res Pract. 2005;36(4):347.

King RV, Burkle FM, Walsh LE, North CS. Competencies for disaster mental health. Curr Psychiatry Rep. 2015;17(3).

Lee JS, You S, Choi YK, Youn Hy SH. A preliminary evaluation of the training effects of a didactic and simulation-based psychological first aid program in students and school counselors in South Korea. PLoS ONE. 2017;12(7):e0181271.

Kang JY, Choi YJ. Effects of a psychological first aid simulated training for pregnant flood victims on disaster relief worker’s knowledge, competence, and self-efficacy. Appl Nurs Res. 2020:1513.

Park S, Kim S. The effects of team-based simulation education on problem solving process. Korean J Health Communication. 2019;14(2):165–72.

Australian Psychological Society. Useful skills for disaster recovery: Problem solving. 2016.

Lim A, Song Y. A scoping review of instruments for measuring problem-solving ability in simulation nursing education in Korea: a focus on process behavior survey. J Korean Acad Fundamentals Nurs. 2022;29(3):269–83.

Lee M, Lee M, Kim S. A study on nursing students’ self-leadership and their perception of learning. J Korean Acad Soc Nurs Educ. 2015;21(3):417–25. https://doi.org/10.5977/jkasne.2015.21.3.417 .

Goldsby E, Goldsby M, Neck CB, Neck CP. Under pressure: time management, self-leadership, and the nurse manager. Administrative Sci. 2020;10(3):38.

Kim JK, Song MS. Effects of respiratory infectious disease simulation-based education on nursing students of clinical competency, self-leadership and critical thinking. J Korea Academia-Industrial Cooperation Soc. 2019;20(8):93–101.

Cho GY, Seo MK. Influencing factors of learning flow, self-leadership and debriefing satisfaction on problem solving ability of nursing students in simulation learning. J Fisheries Mar Sci Educ. 2020;32(2):409–19.

Artino AR. Motivational beliefs and perceptions of instructional quality: Predicting satisfaction with online training. J Comput Assist Learn. 2008;24(3):260–70.

Yu BM, Jeon JC, Park HJ. The effects on learning motivation and self-efficacy according to the type of reflection. J Educational Inform Media. 2013;19(4):837–59.

Kim KA, Choi DW. The effect of virtual simulation in nursing education: an application of care for acute heart disease patients. J Korean Soc Simul Nurs. 2018;6(2):1–13.

Sapiano AB, Sammut R, Trapani J. The effectiveness of virtual simulation in improving student nurses’ knowledge and performance during patient deterioration: a pre and post test design. Nurse Educ Today. 2018;62:128–33.

Park SY, Kweon YR. The effect of using standardized patients in psychiatric nursing practical training for nursing college students. J Korean Acad Psychiatric Mental Health Nurs. 2012;21(1):79–88.

Ko E. Development of a virtual simulation-based nurse education program to provide psychological support for patients affected by infectious disease disasters [dissertation]. [Seoul, Korea]: Chung-Ang University.

Yoon HY, Choi YK. The development and validation of the perceived competence scale for disaster mental health workforce. Psychiatry Invest. 2019;16(11):816.

Park JW, Woo OH. The effects of PBL (problem-based learning) on problem solving process by learner’s metacognitive level. J Educational Technol. 1999;15(3):55–81.

Lee JS. The effects of process behaviors on problem-solving performance in various tests. Dissertation Abstracts International Section A: Humanities and Social Sciences. 1978;39(4-A):2149.

Manz CC. The art of self-leadership: strategies for personal effectiveness in your life and work. Prentice-Hall; 1983.44. Kim HS. The relationship between self-leadership and job satisfaction of middle school teacher [master’s thesis]. [Seoul]: Soongsil University; 2003. 83 p. 45. Ayres HW. Factors related to motivation to learn and motivation to transfer learning in a nursing population [dissertation]. [Raleigh]: North Carolina State University.

Kim HS. The relationship between self-leadership and job satisfaction of middle school teacher [master?s thesis]. [Seoul]: Soongsil University; 2003. 83 p.

Ayres HW. Factors related to motivation to learn and motivation to transfer learning in a nursing population [dissertation]. [Raleigh]: North Carolina State University.

Park H, Choi S, Choi Y, Park S, You S, Baik M, et al. Effect of Korean version of psychological first aid training program on training disaster mental health service provider. J Korean Neuropsychiatric Association. 2020;59(2):123–35.

Kim M, Kim S, Lee WS. Effects of a virtual reality simulation and a blended simulation of care for pediatric patient with asthma. Child Health Nurs Res. 2019;25(4):496–506.

Noh J, Oh EG, Kim SS, Jang YS, Chung HS, Lee O. Development and evaluation of a multimodality simulation disaster education and training program for hospital nurses. Int J Nurs Pract. 2020:e12810.

Cho JL. The effect of self-leadership and communication barriers on nursing performance in hospital nurses. J Convergence Cult Technol. 2019;5(2):239–46.

Kwon SN, Park HJ. Effects of nurses’ positive psychological capital, self-leadership, and relational bonds on organizational commitment. J Korean Acad Nurs Adm. 2020;26(3):241–50.

Yoo JY, Lee YH, Ha YK. A convergence study on the effects of self-leadership and job satisfaction on nursing performance in general hospital nurses. J Convergence Inform Technol. 2019;9(10):28–38.

Park SY, Shim CS. Effects of a simulator-based delivery education on the major satisfaction, nursing professionalism and clinical competence of the nursing students. J Korea Entertainment Ind Association (JKEIA). 2018;12(5):199.

Jeon JC. The effects on learning motivation and self-efficacy according to the type of reflection. J Educational Inform Media. 2013;19(4):837–59.

Kim S, Cho Y. Proliferation of online learning and the implications for teaching and learning in future university education. J Lifelong Learn Soc. 2018;14(4):51–78.

Kang GS, Kim Y. Development and application of integrated-simulation practice program using standardized patients: caring for alcoholism with diabetes mellitus in the community. J Korea Academia-Industrial Cooperation Soc. 2016;17(8):662–72.

Download references

Acknowledgements

The authors would like to thank Eun-Joo Choi and Dong-Hee Cho for their contributions to the development of the simulation program.

This work was supported by the National Research Foundation of Korea (NRF) through a grant funded by the Korean government (Ministry of Science and ICT) (NRF-2020R1A2B5B0100208).

Author information

Authors and affiliations.

Chung-Ang University, Red Cross College of Nursing, Seoul, South Korea

Eunjung Ko & Yun-Jung Choi

You can also search for this author in PubMed   Google Scholar

Contributions

K.E. and Y.C. wrote the design, collected data, data analysis, results, discussion, and conclusion.

Corresponding author

Correspondence to Yun-Jung Choi .

Ethics declarations

Ethics approval and consent to participate.

This study was conducted in accordance with the Declaration of Helsinki (Association World Medical, 2013) and was part of a larger study. It was approved by the Institutional Review Board of Chung-Ang University (IRB approval number 1041078-202003-HRSB-070-01CC) and retrospectively registered (21/11/2023) in the Clinical Research Information Service ( https://cris.nih.go.kr ) with trial registration number KCT0008965. All the participants provided written informed consent and were informed of the right to withdraw from participation at any time during the research until publication. Data confidentiality was ensured, and the results were provided to the participants at their request.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Ko, E., Choi, YJ. Efficacy of a virtual nursing simulation-based education to provide psychological support for patients affected by infectious disease disasters: a randomized controlled trial. BMC Nurs 23 , 230 (2024). https://doi.org/10.1186/s12912-024-01901-4

Download citation

Received : 01 November 2023

Accepted : 29 March 2024

Published : 07 April 2024

DOI : https://doi.org/10.1186/s12912-024-01901-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Virtual simulation
• Psychological support
• Infectious diseases disaster
• Nursing education

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]

• COVID-19 and your mental health

Worries and anxiety about COVID-19 can be overwhelming. Learn ways to cope as COVID-19 spreads.

At the start of the COVID-19 pandemic, life for many people changed very quickly. Worry and concern were natural partners of all that change — getting used to new routines, loneliness and financial pressure, among other issues. Information overload, rumor and misinformation didn't help.

Worldwide surveys done in 2020 and 2021 found higher than typical levels of stress, insomnia, anxiety and depression. By 2022, levels had lowered but were still higher than before 2020.

Though feelings of distress about COVID-19 may come and go, they are still an issue for many people. You aren't alone if you feel distress due to COVID-19. And you're not alone if you've coped with the stress in less than healthy ways, such as substance use.

But healthier self-care choices can help you cope with COVID-19 or any other challenge you may face.

And knowing when to get help can be the most essential self-care action of all.

Recognize what's typical and what's not

Stress and worry are common during a crisis. But something like the COVID-19 pandemic can push people beyond their ability to cope.

In surveys, the most common symptoms reported were trouble sleeping and feeling anxiety or nervous. The number of people noting those symptoms went up and down in surveys given over time. Depression and loneliness were less common than nervousness or sleep problems, but more consistent across surveys given over time. Among adults, use of drugs, alcohol and other intoxicating substances has increased over time as well.

The first step is to notice how often you feel helpless, sad, angry, irritable, hopeless, anxious or afraid. Some people may feel numb.

Keep track of how often you have trouble focusing on daily tasks or doing routine chores. Are there things that you used to enjoy doing that you stopped doing because of how you feel? Note any big changes in appetite, any substance use, body aches and pains, and problems with sleep.

These feelings may come and go over time. But if these feelings don't go away or make it hard to do your daily tasks, it's time to ask for help.

Get help when you need it

If you're feeling suicidal or thinking of hurting yourself, seek help.

• Contact your healthcare professional or a mental health professional.
• Contact a suicide hotline. In the U.S., call or text 988 to reach the 988 Suicide & Crisis Lifeline , available 24 hours a day, seven days a week. Or use the Lifeline Chat . Services are free and confidential.

If you are worried about yourself or someone else, contact your healthcare professional or mental health professional. Some may be able to see you in person or talk over the phone or online.

You also can reach out to a friend or loved one. Someone in your faith community also could help.

And you may be able to get counseling or a mental health appointment through an employer's employee assistance program.

Another option is information and treatment options from groups such as:

• National Alliance on Mental Illness (NAMI).
• Substance Abuse and Mental Health Services Administration (SAMHSA).
• Anxiety and Depression Association of America.

Self-care tips

Some people may use unhealthy ways to cope with anxiety around COVID-19. These unhealthy choices may include things such as misuse of medicines or legal drugs and use of illegal drugs. Unhealthy coping choices also can be things such as sleeping too much or too little, or overeating. It also can include avoiding other people and focusing on only one soothing thing, such as work, television or gaming.

Unhealthy coping methods can worsen mental and physical health. And that is particularly true if you're trying to manage or recover from COVID-19.

Self-care actions can help you restore a healthy balance in your life. They can lessen everyday stress or significant anxiety linked to events such as the COVID-19 pandemic. Self-care actions give your body and mind a chance to heal from the problems long-term stress can cause.

Take care of your body

Healthy self-care tips start with the basics. Give your body what it needs and avoid what it doesn't need. Some tips are:

• Get the right amount of sleep for you. A regular sleep schedule, when you go to bed and get up at similar times each day, can help avoid sleep problems.
• Move your body. Regular physical activity and exercise can help reduce anxiety and improve mood. Any activity you can do regularly is a good choice. That may be a scheduled workout, a walk or even dancing to your favorite music.
• Choose healthy food and drinks. Foods that are high in nutrients, such as protein, vitamins and minerals are healthy choices. Avoid food or drink with added sugar, fat or salt.
• Avoid tobacco, alcohol and drugs. If you smoke tobacco or if you vape, you're already at higher risk of lung disease. Because COVID-19 affects the lungs, your risk increases even more. Using alcohol to manage how you feel can make matters worse and reduce your coping skills. Avoid taking illegal drugs or misusing prescriptions to manage your feelings.

Take care of your mind

Healthy coping actions for your brain start with deciding how much news and social media is right for you. Staying informed, especially during a pandemic, helps you make the best choices but do it carefully.

Set aside a specific amount of time to find information in the news or on social media, stay limited to that time, and choose reliable sources. For example, give yourself up to 20 or 30 minutes a day of news and social media. That amount keeps people informed but not overwhelmed.

For COVID-19, consider reliable health sources. Examples are the U.S. Centers for Disease Control and Prevention (CDC) and the World Health Organization (WHO).

Other healthy self-care tips are:

• Relax and recharge. Many people benefit from relaxation exercises such as mindfulness, deep breathing, meditation and yoga. Find an activity that helps you relax and try to do it every day at least for a short time. Fitting time in for hobbies or activities you enjoy can help manage feelings of stress too.
• Stick to your health routine. If you see a healthcare professional for mental health services, keep up with your appointments. And stay up to date with all your wellness tests and screenings.
• Stay in touch and connect with others. Family, friends and your community are part of a healthy mental outlook. Together, you form a healthy support network for concerns or challenges. Social interactions, over time, are linked to a healthier and longer life.

Avoid stigma and discrimination

Stigma can make people feel isolated and even abandoned. They may feel sad, hurt and angry when people in their community avoid them for fear of getting COVID-19. People who have experienced stigma related to COVID-19 include people of Asian descent, health care workers and people with COVID-19.

Treating people differently because of their medical condition, called medical discrimination, isn't new to the COVID-19 pandemic. Stigma has long been a problem for people with various conditions such as Hansen's disease (leprosy), HIV, diabetes and many mental illnesses.

People who experience stigma may be left out or shunned, treated differently, or denied job and school options. They also may be targets of verbal, emotional and physical abuse.

Communication can help end stigma or discrimination. You can address stigma when you:

• Get to know people as more than just an illness. Using respectful language can go a long way toward making people comfortable talking about a health issue.
• Get the facts about COVID-19 or other medical issues from reputable sources such as the CDC and WHO.
• Speak up if you hear or see myths about an illness or people with an illness.

COVID-19 and health

The virus that causes COVID-19 is still a concern for many people. By recognizing when to get help and taking time for your health, life challenges such as COVID-19 can be managed.

• Mental health during the COVID-19 pandemic. National Institutes of Health. https://covid19.nih.gov/covid-19-topics/mental-health. Accessed March 12, 2024.
• Mental Health and COVID-19: Early evidence of the pandemic's impact: Scientific brief, 2 March 2022. World Health Organization. https://www.who.int/publications/i/item/WHO-2019-nCoV-Sci_Brief-Mental_health-2022.1. Accessed March 12, 2024.
• Mental health and the pandemic: What U.S. surveys have found. Pew Research Center. https://www.pewresearch.org/short-reads/2023/03/02/mental-health-and-the-pandemic-what-u-s-surveys-have-found/. Accessed March 12, 2024.
• Taking care of your emotional health. Centers for Disease Control and Prevention. https://emergency.cdc.gov/coping/selfcare.asp. Accessed March 12, 2024.
• #HealthyAtHome—Mental health. World Health Organization. www.who.int/campaigns/connecting-the-world-to-combat-coronavirus/healthyathome/healthyathome---mental-health. Accessed March 12, 2024.
• Coping with stress. Centers for Disease Control and Prevention. www.cdc.gov/mentalhealth/stress-coping/cope-with-stress/. Accessed March 12, 2024.
• Manage stress. U.S. Department of Health and Human Services. https://health.gov/myhealthfinder/topics/health-conditions/heart-health/manage-stress. Accessed March 20, 2020.
• COVID-19 and substance abuse. National Institute on Drug Abuse. https://nida.nih.gov/research-topics/covid-19-substance-use#health-outcomes. Accessed March 12, 2024.
• COVID-19 resource and information guide. National Alliance on Mental Illness. https://www.nami.org/Support-Education/NAMI-HelpLine/COVID-19-Information-and-Resources/COVID-19-Resource-and-Information-Guide. Accessed March 15, 2024.
• Negative coping and PTSD. U.S. Department of Veterans Affairs. https://www.ptsd.va.gov/gethelp/negative_coping.asp. Accessed March 15, 2024.
• Health effects of cigarette smoking. Centers for Disease Control and Prevention. https://www.cdc.gov/tobacco/data_statistics/fact_sheets/health_effects/effects_cig_smoking/index.htm#respiratory. Accessed March 15, 2024.
• People with certain medical conditions. Centers for Disease Control and Prevention. https://www.cdc.gov/coronavirus/2019-ncov/need-extra-precautions/people-with-medical-conditions.html. Accessed March 15, 2024.
• Your healthiest self: Emotional wellness toolkit. National Institutes of Health. https://www.nih.gov/health-information/emotional-wellness-toolkit. Accessed March 15, 2024.
• World leprosy day: Bust the myths, learn the facts. Centers for Disease Control and Prevention. https://www.cdc.gov/leprosy/world-leprosy-day/. Accessed March 15, 2024.
• HIV stigma and discrimination. Centers for Disease Control and Prevention. https://www.cdc.gov/hiv/basics/hiv-stigma/. Accessed March 15, 2024.
• Diabetes stigma: Learn about it, recognize it, reduce it. Centers for Disease Control and Prevention. https://www.cdc.gov/diabetes/library/features/diabetes_stigma.html. Accessed March 15, 2024.
• Phelan SM, et al. Patient and health care professional perspectives on stigma in integrated behavioral health: Barriers and recommendations. Annals of Family Medicine. 2023; doi:10.1370/afm.2924.
• Stigma reduction. Centers for Disease Control and Prevention. https://www.cdc.gov/drugoverdose/od2a/case-studies/stigma-reduction.html. Accessed March 15, 2024.
• Nyblade L, et al. Stigma in health facilities: Why it matters and how we can change it. BMC Medicine. 2019; doi:10.1186/s12916-019-1256-2.
• Combating bias and stigma related to COVID-19. American Psychological Association. https://www.apa.org/topics/covid-19-bias. Accessed March 15, 2024.
• Yashadhana A, et al. Pandemic-related racial discrimination and its health impact among non-Indigenous racially minoritized peoples in high-income contexts: A systematic review. Health Promotion International. 2021; doi:10.1093/heapro/daab144.
• Sawchuk CN (expert opinion). Mayo Clinic. March 25, 2024.

Products and Services

• A Book: Endemic - A Post-Pandemic Playbook
• Begin Exploring Women's Health Solutions at Mayo Clinic Store
• A Book: Future Care
• Antibiotics: Are you misusing them?
• COVID-19 and vitamin D
• Convalescent plasma therapy
• Coronavirus disease 2019 (COVID-19)
• COVID-19: How can I protect myself?
• Herd immunity and coronavirus
• COVID-19 and pets
• COVID-19 antibody testing
• COVID-19, cold, allergies and the flu
• COVID-19 drugs: Are there any that work?
• Long-term effects of COVID-19
• COVID-19 tests
• COVID-19 in babies and children
• Coronavirus infection by race
• COVID-19 travel advice
• COVID-19 vaccine: Should I reschedule my mammogram?
• COVID-19 vaccines for kids: What you need to know
• COVID-19 vaccines
• COVID-19 variant
• COVID-19 vs. flu: Similarities and differences
• COVID-19: Who's at higher risk of serious symptoms?
• Debunking coronavirus myths
• Different COVID-19 vaccines
• Extracorporeal membrane oxygenation (ECMO)
• Fever: First aid
• Fever treatment: Quick guide to treating a fever
• Fight coronavirus (COVID-19) transmission at home
• Honey: An effective cough remedy?
• How do COVID-19 antibody tests differ from diagnostic tests?
• How to take your pulse
• How to measure your respiratory rate
• How to take your temperature
• How well do face masks protect against COVID-19?
• Is hydroxychloroquine a treatment for COVID-19?
• Loss of smell
• Mayo Clinic Minute: You're washing your hands all wrong
• Mayo Clinic Minute: How dirty are common surfaces?
• Multisystem inflammatory syndrome in children (MIS-C)
• Nausea and vomiting
• Pregnancy and COVID-19
• Safe outdoor activities during the COVID-19 pandemic
• Safety tips for attending school during COVID-19
• Sex and COVID-19
• Shortness of breath
• Thermometers: Understand the options
• Treating COVID-19 at home
• Unusual symptoms of coronavirus
• Vaccine guidance from Mayo Clinic
• Watery eyes

Related information

• Mental health: What's normal, what's not - Related information Mental health: What's normal, what's not
• Mental illness - Related information Mental illness

Your gift holds great power – donate today!

Make your tax-deductible gift and be a part of the cutting-edge research and care that's changing medicine.

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

The impact of online education during the Covid-19 pandemic on the professional identity formation of medical students: A systematic scoping review

Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliations Yong Loo Lin School of Medicine, National University Singapore, Singapore, Singapore, Division of Supportive and Palliative Care, National Cancer Centre Singapore, Singapore

Affiliation Division of Cancer Education, National Cancer Centre Singapore, Singapore, Singapore

Affiliations Division of Cancer Education, National Cancer Centre Singapore, Singapore, Singapore, Department of Pharmacy, National Cancer Centre Singapore, Singapore, Singapore

Affiliations Division of Supportive and Palliative Care, National Cancer Centre Singapore, Singapore, Division of Cancer Education, National Cancer Centre Singapore, Singapore, Singapore, Duke-NUS Medical School, Singapore, Singapore, Assisi Hospice, Singapore, Singapore

Affiliations Division of Supportive and Palliative Care, National Cancer Centre Singapore, Singapore, Duke-NUS Medical School, Singapore, Singapore, Lien Centre for Palliative Care, Duke-NUS Medical School, Singapore, Singapore

Affiliation Palliative Care Institute Liverpool, Academic Palliative & End of Life Care Centre, University of Liverpool, Liverpool, United Kingdom

Affiliations Yong Loo Lin School of Medicine, National University Singapore, Singapore, Singapore, Duke-NUS Medical School, Singapore, Singapore, Department of Rheumatology and Immunology, Singapore General Hospital, Singapore, Singapore

Affiliation Medical Social Services, Singapore General Hospital, Singapore, Singapore

Affiliation Department of Internal Medicine, Singapore General Hospital, Singapore, Singapore

Affiliations Division of Cancer Education, National Cancer Centre Singapore, Singapore, Singapore, Duke-NUS Medical School, Singapore, Singapore, Division of Medical Oncology, National Cancer Centre Singapore, Singapore, Singapore

•  [ ... ],

* E-mail: [email protected]

Affiliations Yong Loo Lin School of Medicine, National University Singapore, Singapore, Singapore, Division of Supportive and Palliative Care, National Cancer Centre Singapore, Singapore, Division of Cancer Education, National Cancer Centre Singapore, Singapore, Singapore, Duke-NUS Medical School, Singapore, Singapore, Palliative Care Institute Liverpool, Academic Palliative & End of Life Care Centre, University of Liverpool, Liverpool, United Kingdom, PalC, The Palliative Care Centre for Excellence in Research and Education, Singapore, Singapore

• [ view all ]
• [ view less ]
• Jonathan Zhen Liang, 
• Donovan Kai Wei Ng, 
• Vijayprasanth Raveendran, 
• Mac Yu Kai Teo, 
• Elaine Li Ying Quah, 
• Keith Zi Yuan Chua, 
• Jun Kiat Lua, 
• Jasmine Lerk Juan Owyong, 
• Andrew Vimal Vijayan, 

• Published: January 5, 2024
• https://doi.org/10.1371/journal.pone.0296367
• Reader Comments

Evolving individual, contextual, organizational, interactional and sociocultural factors have complicated efforts to shape the professional identity formation (PIF) of medical students or how they feel, act and think as professionals. However, an almost exclusive reliance on online learning during the COVID-19 pandemic offers a unique opportunity to study the elemental structures that shape PIF and the environmental factors nurturing it. We propose two independent Systematic Evidence-Based Approach guided systematic scoping reviews (SSR in SEBA)s to map accounts of online learning environment and netiquette that structure online programs. The data accrued was analysed using the clinically evidenced Krishna-Pisupati Model of Professional Identity Formation (KPM) to study the evolving concepts of professional identity. The results of each SSR in SEBA were evaluated separately with the themes and categories identified in the Split Approach combined to create richer and deeper ‘themes/categories’ using the Jigsaw Perspective. The ‘themes/categories’ from each review were combined using the Funnelling Process to create domains that guide the discussion. The ‘themes/categories’ identified from the 141 included full-text articles in the SSR in SEBA of online programs were the content and effects of online programs. The themes/categories identified from the 26 included articles in the SSR in SEBA of netiquette were guidelines, contributing factors, and implications. The Funnelling Process identified online programs (encapsulating the content, approach, structures and the support mechanisms); their effects; and PIF development that framed the domains guiding the discussion. This SSR in SEBA identifies the fundamental elements behind developing PIF including a structured program within a nurturing environment confined with netiquette-guided boundaries akin to a Community of Practice and the elemental aspect of a socialisation process within online programs. These findings ought to be applicable beyond online training and guide the design, support and assessment of efforts to nurture PIF.

Citation: Liang JZ, Ng DKW, Raveendran V, Teo MYK, Quah ELY, Chua KZY, et al. (2024) The impact of online education during the Covid-19 pandemic on the professional identity formation of medical students: A systematic scoping review. PLoS ONE 19(1): e0296367. https://doi.org/10.1371/journal.pone.0296367

Editor: Marsa Gholamzadeh, Tehran University of Medical Sciences, ISLAMIC REPUBLIC OF IRAN

Received: June 7, 2023; Accepted: December 9, 2023; Published: January 5, 2024

Copyright: © 2024 Liang et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript and its Supporting Information files.

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Developing altruistic, ethical, humanistic and accountable physicians pivots on nurturing a medical student’s professional identity formation (PIF) [ 1 , 2 ]. However, medical education continues to struggle to understand and shape how medical students feel, act and think as professionals [ 3 ]. Sarraf-Yazdi et al. [ 4 ] attribute current gaps in understanding PIF to a failure to understand the impact of environmental, organizational, educational, research, clinical, individual, psychosocial, and contextual factors on the PIF process.

The shift from in-person, multi-actor educational interactions to a pandemic-induced online medical education program, offered a unique opportunity to study the key influences shaping PIF [ 5 ]. The nature of online platforms creates physical boundaries between virtual and physical programs, oft-password controlled access, and structured approach access, shaping interactions and guiding progress, attenuating some of the many influences impacting learning. This allows the essential aspects shaping PIF to come to the fore [ 6 ]. With such insights likely to inform efforts to nurture PIF in any training situation in medical school and beyond [ 7 ], we ask the question “ how does an online education program shape a medical student’s PIF ?”.

Theoretical framework

The mentoring ecosystem.

Conceiving online training programs as self-contained, structured programs with clear boundaries, and a distinct training trajectory for its multiple learners, tutors, and the host organization (henceforth stakeholders) draws similarities with the mentoring ecosystem [ 8 ]. The lens of the mentoring ecosystem focuses attention to structural and environmental facets that map a medical student’s progress. It also allows characterization of PIF through use of the Krishna-Pisupati Model for Professional Identity Formation (henceforth KPM).

The mentoring ecosystem pivots on the presence of clear boundaries that limit the effects of external influences on the progress of mentees along its structured, stage-based trajectory. This structured approach includes its specified learning objectives [ 9 ], goals [ 10 , 11 ], timelines and professional standards [ 12 , 13 ], codes of conduct, roles, responsibilities, expectations [ 14 , 15 ], implicit norms [ 16 ], culture [ 17 ], artifacts, sociocultural norms and expectations and legal requirements [ 18 – 20 ] (henceforth netiquette ); longitudinal mentoring support, stage based assessment program and its nurturing mentoring environment. These features liken the mentoring ecosystem to a Community of Practice (CoP) or “persistent , sustaining social network of individuals who share and develop an overlapping knowledge base , set of beliefs , values , history and experiences focused on a common practice and/or mutual enterprise” [ 21 ]. Current thinking suggests that CoPs are fundamental to PIF.

Concurrently within a structured program, the mentoring ecosystem’s spiral trajectory and longitudinal support and assessment processes supports the Socialisation Process or the process by which medical students are introduced and integrate new experiences. This “process in which the characteristics , values , and norms of the medical profession are internalised , resulting in an individual thinking , acting and feeling like a physician” is another critical aspect in nurturing PIF. The KPM captures evolving notions of PIF amidst maturing competencies and insights, shifts in belief systems, contextual considerations, and psycho-emotional states along the spiral mentoring trajectory [ 22 – 27 ].

The Krishna-Pisupati Model of PIF

The KPM outlines adaptations to a medical student’s belief systems to create a context appropriate identity that is consistent with their current belief systems ( congruence ) and regnant social, organizational, and professional standards and beliefs ( social validation ) within a boundaried and structured program [ 6 ]. There are four aspects to an individual’s belief systems. These correspond to the Innate, Individual, Relational and Societal aspects of the individual’s self-concepts of identity or personhood depicted by the Ring Theory of Personhood (henceforth RToP) at the heart of the KPM [ 28 – 31 ] ( S1 Fig ).

When ‘life experiences’ are introduced and are integrated into the religious and cultural beliefs, moral values, and ethical principles in the Innate Ring; the beliefs system related to autonomous function and individual characteristics in the Individual Ring; the belief systems governing personal relationships are housed within the Relational Ring and/or the belief system guiding peripheral relationships and societal, professional, and legal expectations within the Societal Ring [ 29 , 30 , 32 , 33 ], an event occurs. An event that is in sync with current belief systems creates resonance . Synchrony occurs when resonant aspects of the belief system are reprioritised to better address an event . When an event clashes with prevailing beliefs, dissonance arises. Dissonance in one ring is termed disharmony , whilst dissonance in two or more rings generates dyssynchrony .

Sensitivity , or detecting the presence of resonance , synchrony , disharmony and dyssynchron y, prompts medical students to evaluate the need for adaptations to their current belief systems ( judgement ) and determine their ability and readiness to make the change ( willingness ). To sustain their overall identity, and ensure congruence and social validation , the medical student must prioritise adaptations and their iterations of the identity work suits the settings, context, and practice (balance) [ 34 ]. It is suggested that evidence of sensitivity , judgement , willingness , balance and identity work points to development of PIF.

Methodology

We carried out two independent systematic scoping reviews (SSR)s of netiquette and online environment. Focus on netiquette was informed by initial reviews showing significant overlap between structure and netiquette and that reviews of netiquette better captured accounts of codes of practice.

We adapted Krishna’s Systematic Evidence-Based Approach (SEBA) to guide the two SSRs (henceforth SSR in SEBA) [ 5 , 8 , 28 , 30 , 35 – 38 ]—the Dual-SEBA approach ( S2 Fig ). The Dual-SEBA’s constructivist approach [ 36 , 39 – 44 ] and relativist lens [ 45 – 48 ] acknowledges belief systems, narratives, developing competencies, new life experiences, PIF, and netiquette as sociocultural constructs shaped by regnant environmental considerations, desired characteristics and expectations; and the medical student’s narratives, contextual factors, values, beliefs, and principles [ 49 , 50 ].

Each stage of the Dual-SEBA approach was guided by an expert team which comprised of a librarian from the National University of Singapore’s (NUS) Yong Loo Lin School of Medicine (YLLSoM) and local educational experts and clinicians at YLLSoM, National Cancer Centre Singapore, Palliative Care Institute Liverpool, and Duke-NUS Medical School.

Stage 1 of SEBA: Systematic approach

Each research team employed the PCC (Population/Concept/Context Study design) format and PRISMA checklist (see S1 File ) to guide their primary research questions [ 51 ].

1. Netiquette.

With only a limited number of articles on the topic, the primary research question extended beyond the Covid-19 timeframe and focused on “ What is known about netiquette in online programs in medical schools ?” and the secondary research question was “ What are the features , causes and implications of lapses in netiquette in online programs in medical schools ?” ( Table 1 ).

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0296367.t001

Independent searches were conducted on PubMed, SCOPUS, ERIC, Google Scholar, Embase between 12 th September 2022 and 21 st January 2023 for articles published between 1 st January 2000 and 31 st December 2021 on online professionalism and standards of practice in online interactions within medical schools. The full search strategy is enclosed in the supplementary file ( S2 File ).

2. Online medical training during Covid-19.

To evaluate online medical training programs during the Covid-19 pandemic, the research and expert teams determined the primary research question to be “What is known of online medical training programs during the Covid-19 pandemic ? ” . The secondary research question was “How are online medical training programs structured , assessed and supported during the Covid-19 pandemic ? ” ( Table 2 ).

https://doi.org/10.1371/journal.pone.0296367.t002

In surveying extant literature on online medical training programs during the Covid-19 pandemic, the second research team extended Stojan et al. [ 52 ]’s review on online learning developments in undergraduate medical education during Covid-19, beyond articles published on the MedEdPublish portal. Snowballing of relevant articles from the included articles was also proposed to ensure a comprehensive review and the inclusion of key articles.

Members of the research team conducted independent searches on PubMed, Embase, ERIC and Scopus between 17 th December 2022 and 17 th February 2023 for articles published between 1 st January 2019 to 31 st December 2022.

Searching . To ensure a sustainable review the expert teams limited the inclusion criteria in keeping with Pham et al. [ 53 ]’s approach to scoping reviews. Each team independently studied the database and discussed their findings, adopting Sandelowski and Barroso [ 54 ]’s ‘negotiated consensual validation’ to attain consensus on the final list of titles to be reviewed.

Stage 2 of SEBA: Split approach

Krishna’s ‘Split Approach’ ensures that novel aspects of the area of interest are not omitted [ 39 , 53 – 57 ]. For each review, two independent groups of researchers analysed the included articles concurrently using Braun and Clarke [ 58 ]’s approach to thematic analysis and Hsieh and Shannon [ 59 ]’s approach to directed content analysis.

Employing Braun and Clarke [ 58 ]’s approach to thematic analysis, the first team of researchers independently reviewed the included articles to map patterns in the data and synthesise a code book to code the remaining articles. Guided by an inductive approach, subthemes were reorganised into themes that best described the data [ 60 ]. ‘Negotiated consensual validation’ determined the final list of themes.

The second research team adopted Hsieh and Shannon [ 59 ]’s approach to directed content analysis, deriving codes from Ahmed et al. [ 61 ]’s review entitled “Model for utilizing distance learning post COVID-19 using (PACT)™ a cross sectional qualitative study” to encapsulate key aspects of online education programs and netiquette. In the presence of a working theory, Hsieh and Shannon [ 59 ]’s approach to directed content analysis promises to capture all evidence of phenomena identified in the KPM and attenuate concerns regarding the omission of negative findings and new considerations attributed to thematic analysis [ 4 , 33 , 62 – 64 ]. Hsieh and Shannon [ 59 ]’s approach to directed content analysis also provides ‘ supporting and un-supporting evidence for a theory ’ which in turn allows for KPM to be ‘ supported and extended ’ [ 59 , 65 , 66 ]. The deductive approach adopted allows confirmation, expansion, retesting and study of the KPM theory beyond the mentoring setting [ 66 – 68 ]. This approach acts as a check and balance [ 69 ] to reflexive thematic analysis that pivots on coding reliability and use of Cohen’s Kappa to assess the degree of consensus between researchers coding the same piece of data; code books that contain a shared understanding of the codes and themes; reflexive thematic analysis which recognises the role of researcher’s interpretation of the codes; and the employ of multiple researchers to ‘ sense check’ the data.

Here, the Split Approach is useful particularly when Cohen’s Kappa is not employed, given that coding is seen as part of a training process for new researchers. The presence of independent data from different sources also reduces concerns about the trustworthiness [ 70 ].

Stage 3 of SEBA: Jigsaw perspective

Reimagined as pieces of a jigsaw puzzle, complementary elements of themes in each review were combined with the categories identified in direct content analysis to create bigger pieces of the puzzle or ‘themes/categories. This process was guided by Phases 4 to 6 of France et al. [ 71 ]’s approach to meta-ethnography.

Stage 4 of SEBA: Funnelling process

France et al. [ 71 ]’s approach also guided the Funnelling Process which juxtaposed the themes/categories from each review to form domains.

a. Online programs

12370 abstracts were reviewed, 4406 full text articles were evaluated and 134 articles were included. With snowballing identifying seven articles, 141 full text articles were included ( Fig 1 ). 65 were quantitative studies, five qualitative studies, two mixed studies, and 69 were descriptive/opinions/proceedings/reviews/perspectives/monographs. The Jigsaw Perspective identified two themes/categories—the content of current programs and effects of online programs.

https://doi.org/10.1371/journal.pone.0296367.g001

b. Netiquette

A total of 6115 abstracts were reviewed, 174 full text articles were evaluated, and 19 articles were initially included ( Fig 1 ). Seven additional articles containing the existing netiquette and online professionalism guidelines of medical schools were snowballed from a Google search and from existing articles, yielding a total of 26 final included articles. Six were quantitative studies, two were qualitative studies and 18 were descriptive/opinions/proceedings/reviews/perspectives/monographs. The themes/categories identified were current guidelines, contributing factors, and their implications.

The iterative process of SEBA.

With the initial findings suggesting the presence of features of CoPs and the Socialisation Process, Hsieh and Shannon [ 59 ]’s approach to directed content analysis was used to draw on codes and categories from current data on the KPM [ 66 , 72 , 73 ]. This process created two additional domains. The four domains were features of: 1) current programs; 2) netiquette, 3) CoP; and 4) KPM.

Domain 1. Features of current programs.

Often replacing traditional approaches, the Covid-19 pandemic-induced curricula boosted support of medical students at all stages of their training and catered to the individual needs of medical students from different backgrounds, settings, expectations and different levels of knowledge, skills, and experience [ 5 , 74 – 86 ]. Wooliscroft [ 87 ] and Alkhowailed et al. [ 88 ] suggested that the Covid-19 pandemic-induced curricula changes had cemented telemedicine, simulated learning, and extended reality learning’s role in modern medical education.

The content covered included the expansion of online content in emergency care [ 52 ], confidentiality, safety, awareness of online personas [ 89 – 91 ], and netiquette [ 89 , 90 ]. In addition, it has enhanced access to ‘knowledge banks’ [ 92 – 96 ], and encouraged more sustainable [ 97 – 106 ], innovative [ 82 , 107 – 113 ], rewarding [ 114 ], flexible [ 94 , 115 ], context-appropriate [ 76 , 105 , 112 , 116 – 120 ], engaging [ 76 , 121 ], interactive [ 81 , 122 ] and interprofessional educational approaches [ 123 , 124 ].

These enhancements better aligned expectations, structuring, assessment, and support of online programs [ 85 , 125 ], improved critical thinking, metacognitive and problem-based thinking; boosted engagement and teamwork; increased achievement of learning objectives [ 52 , 105 , 116 , 118 – 120 , 126 , 127 ], access to learning [ 52 , 94 , 97 , 128 – 130 ], and knowledge acquisition and satisfaction rates [ 131 ]. Gordon et al. [ 132 ], Daniel et al. [ 133 ], Dedeilia et al. [ 107 ], Stojan et al. [ 52 ] and Grafton-Clarke et al. [ 101 ] credited online programs with building confidence and skills, role modelling professional values [ 134 ], supporting reflective practice [ 135 ], and nurturing PIF. Rose [ 134 ], Aluri et al. [ 135 ], and Stetson et al. [ 136 ] reported that online interventions contextualised learning and provided users with authentic clinical experiences.

The approach to online teaching also impacted outcomes. Though asynchronous online sessions [ 93 , 101 , 106 , 137 , 138 ] offered convenient study [ 78 , 94 , 97 , 104 , 139 – 142 ] and fostered work-life balance [ 96 , 123 , 143 – 146 ], medical students preferred synchronous sessions [ 97 , 104 , 139 ]. Synchronous sessions countered social isolation [ 113 , 117 , 147 ], provided peer-mentoring and complemented face-to-face learning [ 93 , 113 , 147 – 149 ].

Domain 2. Features of netiquette.

The ill-effects of online education were often not discussed in depth and are summarised in Table 3 for ease of review. These varied considerations underpin the need for structuring and policing of practice. It also helps shape the training trajectory.

https://doi.org/10.1371/journal.pone.0296367.t003

However, differences in focus, duration, subject matter, level of sophistication, structure, assessment processes, and support and oversight of the program and participants across regnant online curricula, along with time and manpower constraints caused by the sudden shift to online education created differences in the content of published netiquette guidelines [ 89 , 90 ]. The context specific nature of netiquette is summarised in Table 4 for ease of review.

https://doi.org/10.1371/journal.pone.0296367.t004

Domain 3. Features of a CoP.

The physical separation between online and physical practice, the online approach, netiquette and structure of the program created boundaried programs. The program structures also advanced clear step-wise inculcation of knowledge, avenues to practice skills and an opportunity to introduce and integrate the values, beliefs, principles and attitudes espoused by the program. These structures served to gradually empower the medical students and give them more significant roles in the program reminiscent of the move from legitimate peripheral participation to key roles at the core of a CoP.

To validate the premise that online programs function like mentoring ecosystems or a modified CoP, the expert and research teams sought to identify features of CoPs drawn from Cruess et al. [ 276 ], Clement, Brown [ 277 ], Sherbino, Snell [ 278 ], Hean, Anderson [ 279 ], Hägg-Martinell, Hult [ 280 ], Buckley, Steinert [ 281 ] and de Carvalho-Filho, Tio [ 282 ] in accounts of online programs.

Whilst there was evidence of a ‘ persistent , sustaining social network’ and a ‘social network of individuals’; evidence for ‘an overlapping knowledge base , set of beliefs , values , history and experiences’ could only be inferred [ 96 , 110 , 155 , 156 , 158 , 161 , 173 , 182 , 194 , 199 , 204 – 207 ]. Here, shared values, culture, goals, a common identity and a welcoming environment were drawn from accounts of online programs seeking to engage and challenge medical students, set and align expectations, and nurture a conducive learning environment [ 150 , 224 , 225 ].

Similarly, evidence for a structured and guided approach, and flexible and adaptive support mechanisms were implied from the presence of a discrete online program confined by clear physical boundaries, supplemented by standards, netiquette, formal curriculum, learning objectives and a learning trajectory (henceforth structured online program ) [ 167 – 169 ].

Concurrently, data on the presence of a learning trajectory, akin to the notion of a mentoring trajectory, that guides progress is deduced from accounts of achievement of learning objectives [ 52 , 105 , 116 , 118 – 120 , 126 , 127 ], alignment of expectations, structuring, assessment, and support of online programs [ 85 , 125 ], and contextualised learning within authentic clinical experiences [ 167 , 168 , 169 ].

Structure was also evident from the provision of longitudinal role modelling of professional values [ 134 ], supervision, feedback and mentoring to accommodate the learner’s individual goals and needs and support of reflective practice [ 135 ]. Efforts to foster work-life balance [ 96 , 123 , 143 – 146 ], counter social isolation [ 113 , 117 , 147 ], complement face-to-face learning [ 93 , 113 , 147 – 149 ] and in nurturing PIF [ 97 , 208 , 220 , 241 , 244 – 248 ] hint at the presence of a flexible, personalised, responsive, assessment driven approach.

These features, however, were not consistent across the programs and the netiquette guidelines as evidenced by Tables 3 and 4 .

Domain 4. Features of KPM.

The impact of a structured online program on PIF on belief systems and identity is inferred. However, Stouffer et al. [ 283 ]’s account of a short week-long online arts and humanities course for second, third- and fourth-year medical students at John Hopkins University does merit attention. Here, the authors suggest that this intervention inspired the “process of psychological and social development that occurs within the larger context of overall identity formation” [ 296 ]. Other accounts also infer as much. Stojan et al. [ 52 ], Dedeilia et al. [ 107 ] and Grafton-Clarke et al. [ 111 ], for example, report that online learning enhanced cognitive capabilities, and facilitated greater engagement suggesting changes in the Individual and Societal Rings of the RToP [ 105 , 114 , 116 , 118 – 120 , 122 , 127 ]. Other accounts revealed online programs encouraged medical students to become ‘change agents’ and actively reshape the education landscape [ 97 , 113 , 284 – 286 ]. Changes to teamwork [ 287 ], practice, thinking [ 131 ] and wellbeing also imply influence upon sensitivity , judgement , willingness , balance , identity work and reflections within the KPM.

Conversely, disrupted and ineffective learning [ 105 , 158 , 249 , 250 , 288 – 291 ], and a failure to meet learning objectives [ 289 , 292 ] resulted in disharmony in the Individual Ring. There were also accounts of disharmony in the Relational Ring caused by poor tutor-learner relationships [ 61 , 247 , 252 , 253 , 269 ], reduced peer interactions [ 150 , 199 , 253 ] and increased isolation [ 206 , 253 , 293 , 294 ]. Disharmony in these rings cascaded into dyssynchrony across the Societal, Relational and Individual Rings exaggerated by gaps in knowledge, skills, and attitudes [ 251 ], and poor interprofessional practice [ 142 , 206 , 293 , 295 , 296 ]. Overall, when unsupported, such dissonance culminated in ineffectual adaptations, further indicating wider impact upon sensitivity , judgement , willingness , balance , identity work and reflections within the KPM [ 293 ]. The effects [ 135 , 220 ] of netiquette and online programs on the rings of the RToP within the KPM are summarised in Table 5 .

https://doi.org/10.1371/journal.pone.0296367.t005

Stage 5 of SEBA: Analysis of evidence-based and non-data driven literature.

The inclusion of non-data-based articles such as position, perspective, commentaries, conference, reflective and opinion papers, editorials, oral presentations, letters, posters, forum discussions, blogs, interviews, surveys, governmental reports and policy statements from PubMed, Embase, SCOPUS, ERIC and Google Scholar raised concerns over biases in the analysis. To allay these concerns, the research team compared the themes elicited from data-driven publications with those from non-data-based articles. Similarities between the two groups assuaged concerns of biases.

Stage 6 of SEBA: Synthesis of discussion

The “Best Evidence Medical Education (BEME) Collaboration Guide” [ 297 ] and the “Structured approach to the Reporting In healthcare education of Evidence Synthesis (STORIES)” [ 298 ] were used in the synthesis of responses to our primary research questions.

This Dual-SEBA review reveals online programs are comparable to mentoring ecosystems and capable of influencing a medical student’s narratives, their developing competencies, and their conduct and evolving PIF. In nurturing the “ transformative journey through which [a medical student] integrates the knowledge , skills , values , and behaviours of a competent , humanistic physician with [their] unique identity and core values ”, our Dual-SEBA approach addresses our overarching research objective of characterising the basic features required to support PIF in a structured program [ 299 – 302 ]. These include a structured curriculum; an established netiquette; an alignment of expectations; a consistent yet flexible approach; a clearly delineated learning trajectory; longitudinal support; a longitudinal assessment process; personalised and appropriate feedback; and a nurturing learning environment. These findings serve as a template for the design, support, and assessment of future programs in medical education and may be extrapolated to programs in different training settings and even beyond the medical student population.

Indeed, viewing PIF in online training programs as a series of interventions capable of shaping adaptations to belief systems, influencing identity work and asking questions of self-concepts of professional identity highlights several considerations. Whilst structure, a consistent approach, and a nurturing environment are pivotal, there must also be adequate acknowledgment of the individual needs of the participant population. Different narratives, belief systems, contextual considerations, abilities, levels of self-awareness and reflective capabilities undergird the need for a personalised support mechanism to run in tandem with a consistent training approach that seeks to cater for the needs of the general participant population. This also underscores the need for personalised, appropriate, specific, and timely assessments and mentoring support. Such support is essential to shaping a medical student’s sensitivity , judgement , willingness , balance , identity work and reflections and thus their belief systems, self-concepts of personhood and identity. Accessible personalised support is also pertinent when the ramifications of reflections may occur sometime after the experience and when evidence suggests that their effects impact all aspects of personhood and identity ( Table 5 ). Indeed, this Dual-SEBA highlights the potential hazards of unsupported training in Table 3 .

These findings underline the host organization’s role in ensuring effective design [ 4 ], oversight and support [ 5 , 303 ] of the program and supporting faculty training programs and interprofessional education in online programs. Here, the absence of ‘train the trainers’ programs, vis-à-vis holistic assessments and longitudinal evaluations of the education program, is concerning. A further worry is the lack of consideration for communication platforms for accessible support and feedback and indeed the protected time afforded to faculty to meet the individual needs of their student population. Missing too are accounts of the long-term impact of online programs on PIF, oversight and program evaluations that will further guide structure and oversight of online programs.

Limitations

Netiquette in medical education is a relatively under-reviewed and novel area in the existing literature. Gaps in current thinking are accentuated by our focus on the impact of online learning during Covid-19 on the PIF of only medical students.

Including articles in or translated into English may have also restricted the search results. With mainly North American and European-drawn data, these findings may not be as easily applicable beyond these regions.

The insights provided in this Dual-SEBA highlights a number of new considerations that require evaluation. The importance of assessing this longitudinal and holistic developmental process suggests the need for more effective assessment tools, appraisal of the learning environment, training programs for trainers and portfolio use. Similarly, the involvement of interprofessional educational initiatives and potential assessments and support mechanisms also require further study. In light of the flexibility within the online program structure and the potential for cascading effects in PIF, we will focus our immediate attention on creating adaptive and longitudinal assessments of PIF as we look forward to engaging in this exciting field of medical education.

Supporting information

S1 fig. krishna-pisupati model of pif..

https://doi.org/10.1371/journal.pone.0296367.s001

S2 Fig. The Dual-SEBA approach.

https://doi.org/10.1371/journal.pone.0296367.s002

S1 File. PRISMA checklist.

https://doi.org/10.1371/journal.pone.0296367.s003

S2 File. Full search strategy.

https://doi.org/10.1371/journal.pone.0296367.s004

Acknowledgments

The authors would like to dedicate this paper to the late Dr. S Radha Krishna and A/Prof Cynthia Goh whose advice and ideas were integral to the success of this review and Thondy, Maia Olivia and Raja Kamarul whose lives continue to inspire us. The authors would also like to thank the anonymous reviewers for their helpful comments which greatly enhanced this manuscript.

• View Article
• PubMed/NCBI
• Google Scholar
• 17. Stokes J, Price B. Social Media, Visual Culture and Contemporary Identity. In: Callaos N, Carrasquero JV, Sánchez B, Tremante As, Welsch F, editors. 11th International Multi-Conference on Society, Cybernetics and Informatics (IMSCI 2017): International Institute of Informatics and Cybernetics; 2017. p. 159–63.
• 27. Koon OE, Krishna LKR. Perspective from Singapore. NUS Press; 2014.
• 46. Crotty M. The foundations of social research: Meaning and perspective in the research process: Sage; 1998 Oct 15.
• 54. Sandelowski M, Barroso J. Handbook for synthesizing qualitative research: Springer Publishing Company; 2006.
• 218. BE H, SQ T, WT H, YK K, NB P, RA Z. How medical students can respond to the Covid-19 pandemic 2020 [Available from: https://www.thestar.com.my/opinion/letters/2020/04/21/how-medical-students-can-respond-to-the-covid-19-pandemic
• 219. Ali NuA. Students disappointed with online teaching system amid COVID-19 2020 [Available from: https://dailytimes.com.pk/587446/students-disappointed-with-online-teaching-system-amid-covid-19/ .
• 273. Smale T. Faculty of Health Netiquette Guidelines: Keele University; 2012 [updated 2012-06-25. Available from: https://www.keele.ac.uk/health/netiquette/Netiquette%20Guidelines.pdf .

WCU Stories

Business savvy student puts skills to use for ncur project.

 WCU Stories      April 9, 2024

Steven Hudspeth

By Julia Duvall

During his childhood, Western Carolina University student Steven Hudspeth, a junior from Gastonia, used to go on day trips with his grandmother and siblings to see the Blue Ridge Parkway from the Western North Carolina side.

He loved seeing the mountains each trip and kept those childhood memories close.

“We never went to the beach, it was always the mountains that we loved to go visit,” Hudspeth said.

Fast forward a few years later, Hudspeth knew that after graduating from Gaston Community College, he wanted to continue his education at WCU and be surrounded by the mountains from his childhood.

Having his school choice solidified, it was now time for Hudspeth to decide on a major and concentration. Always having an interest in business, he knew he wanted to help people achieve their financial goals, so he chose the financial planning concentration in WCU’s College of Business’ finance program.

“I was meeting with my adviser, Dr. (Gary) Curnutt, and he mentioned the National Conference on Undergraduate Research and all of the opportunities available to students,” Hudspeth said. “Initially, I was unsure if that was something I would want to do but learning more about NCUR and having Dr. Curnutt send me all of the information, I became very interested in submitting a research project.”

Hudspeth’s research paper, “The Association Between Online Payments Using Different Payment Types and Six Consumer Assessment Characteristics,” looks at the relationship between online payments using various types of payments, such as cash, checks, debit cards, credit cards, prepaid cards, money orders and bank account number payments during COVID-19, and six characteristics used to assess payment choice.  

“During COVID-19, there was a significant uptick in online payments due to the restrictions stemming from the pandemic, which shifted how businesses catered to their customers in person and online,” he said. “While researching, I ended up stumbling upon the 2020 Survey of Consumer Choice published by the Federal Reserve Bank of Atlanta,” Hudspeth said.

Hudspeth used that model for his study to see the association between online payments and the six consumer characteristics which are security, acceptance, getting and setting up, cost, convenience, and payment record.

“Collaborating on research with Steven has been incredibly rewarding,” Curnutt said. “Throughout our research, Steven has consistently demonstrated qualities that exemplify a great researcher. He possesses a profound intellectual curiosity, seeking to deepen his understanding of fundamental theories and their implications on model design, as well as the assumptions underlying our hypotheses.”

Preliminary results indicated a statistically significant association between payment records and cash, payment records and credit cards, security and prepaid cards, acceptance for payment and bank account number or routing number payments.

“After we ran the model characteristics some things lined up with our hypothesis,” Hudspeth said. “Essentially our hypothesis was based on perceived ease of use or perceived usefulness. The control variable for education level showed a significant statistical association between all education levels and all payment types.”

After working on this project, Hudspeth said he would like to continue his higher education journey in financial planning and conduct more research.

“I've thoroughly enjoyed working with Steven and I certainly look forward to future research collaborations,” Curnutt said.

Related Stories

Research and Discovery

Student Experience

College of Arts and Sciences

WCU student combines love of history and forensics for NCUR research project

Awards and Recognition

Alumnus gets second chance with Fulbright award

About the Blog

Western Carolina University is located in the beautiful Southern Appalachian mountains of Cullowhee, North Carolina. Our 600-acre mountain campus is surrounded by one of the most biodiverse regions in the state that provides students unparalleled learning and adventure opportunities. If you're looking for a university ready to fight as hard for your success as you are, choose WCU. Learn more about our campus...

• February 2024
• January 2024
• December 2023
• November 2023
• October 2023
• September 2023
• August 2023

Connect with Us

• Like Us on Facebook
• Follow us on Twitter
• Follow us on Instagram
• Subscribe to us on YouTube

ORIGINAL RESEARCH article

This article is part of the research topic.

Activating Academia for an Era of Colliding Crises

Seven Years of Embracing the Sustainable Development Goals: Perspectives from University of South Africa's Academic Staff Provisionally Accepted

• 1 University of South Africa, South Africa

The final, formatted version of the article will be published soon.

As this paper was being finalised, the world was left with less than seven of the 15 years of Sustainable Development Goals (SDGs) implementation to 2030. There were still huge gaps in the attainment of the SDGs in institutions of higher learning globally, especially that COVID-19 brought a barrier leading to a known pushback. However, the pandemic did not imply there was no work done prior, during and after COVID-19. This paper investigates the extent to which the University of South Africa's academic staff activated and mainstreamed the SDGs in their core mandates between 2016 and 2022. Data was generated through a survey (n=121), participatory action research, and document analysis. It emerged there is a greater degree of awareness of the SDGs, with 78% of academic respondents confirming this. However, the percentages drop across the four core mandate areas when it comes to SDGs implementation. About 52.6% of academics indicated they were promoting SDGs in their teaching, research (63.3%), community engagement (55.5%) and academic citizenship (54.5%). Findings further reveal key enabling institutional policies like the SDGS Localisation Declaration, and the Africa-Nuanced SDGs Research Support Programme. Large gaps remain on the publication front, where over 60% of the responding academics had not published an article explicitly on SDGs. There is also bias in publications towards certain SDGs. The work recommends that UNISA management continue raising awareness on the SDGs and systematically address barriers identified in the main paper to enhance the mainstreaming of the SDGs across all core mandate areas.

Keywords: Quality education, SDGs Stakeholders, sustainability, higher education, Academic Staff

Received: 13 Dec 2023; Accepted: 11 Apr 2024.

Copyright: © 2024 Nhamo and Chapungu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Prof. Godwell Nhamo, University of South Africa, Pretoria, South Africa

People also looked at