research paper on virtual reality

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Environ Res Public Health

How Virtual Reality Technology Has Changed Our Lives: An Overview of the Current and Potential Applications and Limitations

Associated data.

No new data were created or analyzed in this study. Data sharing is not applicable to this article.

Despite virtual reality (VR) being initially marketed toward gaming, there are many potential and existing VR applications in various sectors and fields, including education, training, simulations, and even in exercise and healthcare. Unfortunately, there is still a lack of general understanding of the strengths and limitations of VR as a technology in various application domains. Therefore, the aim of this literature review is to contribute to the library of literature concerning VR technology, its applications in everyday use, and some of its existing drawbacks. Key VR applications were discussed in terms of how they are currently utilized or can be utilized in the future, spanning fields such as medicine, engineering, education, and entertainment. The main benefits of VR are expressed through the text, followed by a discussion of some of the main limitations of current VR technologies and how they can be mitigated or improved. Overall, this literature review shows how virtual reality technology has the potential to be a greatly beneficial tool in a multitude of applications and a wide variety of fields. VR as a technology is still in its early stages, but more people are becoming interested in it and are optimistic about seeing what kind of changes VR can make in their everyday lives. With how rapidly modern society has adapted to personal computers and smartphones, VR has the opportunity to become the next big technological turning point that will eventually become commonplace in most households.

1. Introduction

This literature review aims to contribute to the library of literature on the applications of virtual reality (VR), how they are currently used and can be used in the future, and some of the strengths and difficulties that come with using VR.

Virtual reality (VR) refers to a computer-generated, three-dimensional virtual environment that users can interact with, typically accessed via a computer that is capable of projecting 3D information via a display, which can be isolated screens or a wearable display, e.g., a head-mounted display (HMD), along with user identification sensors [ 1 ]. VR can mainly be divided into two categories: non-immersive, and immersive [ 2 ]. Non-immersive VR utilizes a combination of screens surrounding the user to present virtual information [ 3 ]. A typical example of this is driving or flight simulations in which the user sits in a chair with multiple screens around them, giving them the feeling of being in the cockpit or driver’s seat without being fully immersed. Immersive VR refers to using a wearable display, e.g., HMD, to track a user’s movement and present the VR information based on the position of users [ 4 ], which allows them to experience 360 degrees of the virtual environment. This immersive experience is what most people think of when it comes to VR and is one of the most marketable aspects of VR technology. In between immersive and non-immersive VR, there is also augmented reality (AR). AR makes use of computer-generated imagery that is overlayed on physical elements in the real world, which can be found in many applications, such as stores providing a virtual fitting application for people to “try on” clothes. Mixed reality (XR) represents the spectrum between the physical and digital worlds, combining AR and VR to allow users to both immerse themselves in a virtual world while also being somewhat grounded in reality.

The concept of VR was first introduced in the 1960s, with Morton’s creation of the Telesphere Mask and the Sensorama [ 5 ]. The original technologies served the purpose of immersing the user in the video display around them, making them feel like they are a part of the video. The Ultimate display was an idea developed by Ivan Sutherland [ 6 ], operating on a similar concept of allowing the user to feel immersed in a computer-generated environment using multiple input and output devices [ 7 , 8 ]. Following the creation of the Sensorama and the idea of the Ultimate display in the 1960s, the next large boom in VR technology development occurred in the early 2010s. During this period of time, VR was still considered a gimmick—it was expensive and was not considered a technology that would ever become popular with the general public. This, however, started to shift in 2012, when Palmer Luckey debuted his prototype for the first Oculus [ 9 ]. In 2014, Facebook acquired Oculus after seeing the interest it garnered, leading to a significant increase in the popularity of VR devices for home use. Since then, VR has grown to become more popular and accessible to the everyday consumer, with more VR headsets available on the market, such as the HTC Vive, Samsung VR, Oculus, Google Cardboard, and more.

Despite VR being initially marketed toward gaming, there are many potential and existing VR applications in various sectors and fields, including education, training, simulations, and even in exercise and healthcare. Unfortunately, there is still a lack of general understanding of the strengths and limitations of VR as a technology in various application domains. Some of the largest issues with current VR technology are hard to overcome and can span from technical to financial and health issues. Technological limitations regarding users feeling uncomfortable or ill while using a VR headset, the inaccessibility of this technology to most people due to the high price of the associated hardware, and the lack of technical standardization are all current issues that the tech industry is hoping to overcome with research and future improvements.

Overall, this literature review serves the purpose of covering how different types of VR applications can be utilized, as well as providing information on the advantages and drawbacks of using VR technology in various application domains.

In order to present a reliable literature review, an extensive search was performed using common journal search engines/websites, e.g., Google Scholar, JSTOR, MDPI, ResearchGate, PubMed, and Science Direct, which includes peer-reviewed studies and articles. Keywords and phrases used in searching for sources include a combination of “VR” or “virtual reality” with “Education”, “Simulation,” “Games”, “Virtual”, “Immersive”, “Non-immersive”, “Training”, “Application”, “Manufacturing”, “Industrial”, “Medical”, “Healthcare”, and “Entertainment”. The variety in keywords helped yield different results for VR not only as a technology but also in major use cases where it has already been utilized for different industries and fields. The gathered papers and articles were then reviewed to further select representative and up-to-date evidence.

Papers were selected with the goal of providing sufficient coverage of the topic by presenting an overarching summary rather than an exhaustive review of every type of application within VR. Having a large variety of papers does not guarantee that every particular use case of VR is covered, but it does provide a wide breadth of use cases of VR that are currently applied, as well as opportunity spaces for VR applications in the future. As shown in Figure 1 , 145 papers were initially collected, but only 77 were thoroughly reviewed to provide enough coverage without unnecessary advanced technical details. Five additional papers and articles were added after review to accommodate additional information, resulting in a total of 82 sources used for the final literature review.

General structure of the paper selection and literature review.

Included papers were those that clearly presented a specific VR application, those that showed clear negative or positive outcomes of VR usage, or papers that provided relevant background information on a specific VR technology. Exclusion criteria included disregarding papers that had an overt focus on VR hardware components, excluding studies that may have mentioned VR without it being the focus, and rejecting papers that became repetitive after utilizing other papers on similar topics. The following sections provide detailed reviews based on various VR applications and domains.

3. Reviews of VR Technology Applications

The technological applications of VR have advanced to a point where they can be applied to an extensive range of fields and industries outside of just gaming or entertainment. Many have started to take advantage of VR in performing tasks that are hard to practice due to limited resources or the inherent risks and dangers associated with said tasks that can sometimes lead to catastrophic consequences. The greatest strength of VR is that it opens up opportunities for people to practice these tasks in a safe capacity while also being immersed enough for it to feel realistic and transferable to the real world and depict almost any situation accurately [ 10 ]. This section covers some of the main categories of VR applications and provides examples of how these applications are applied or can be applied to different use cases across various fields.

One of the most widely used and largely applicable applications of VR is the simulation aspect, which can be uniquely created and customized to suit users’ needs. There are two main types of simulations: immersive and non-immersive. As mentioned above, non-immersive VR simulations usually include multiple screens and some type of platform or apparatus that mimics the activities or tasks in reality [ 3 ]. Immersive VR simulations differ in terms of using HMDs in place of screens and can either utilize a control platform or apparatus such as the ones used in non-immersive simulations [ 11 ] or can instead be fully contained within a virtual setup and require no external setups or platforms. Whether users opt for immersive or non-immersive VR simulations, there is no significant difference in the performance, and the results appear to be very similar in fulfilling the simulation’s purpose [ 12 ]. There is, however, a slight advantage to using immersive VR simulations with HMDs, as they are capable of fully immersing the user in the simulated environment and giving them a more thorough experience [ 13 ].

3.1. Industrial Simulation Applications

VR simulations have many applications that can span from training simulation to prototyping, designing, and testing tools and objects. Some commonly used VR simulations in the industrial domain include driving simulators, flight simulators for pilots, and combat simulators for military personnel, all of which provide training to users in highly dangerous circumstances without putting them at risk during the training process [ 14 ]. Among the many use cases, two typical simulation applications are further discussed in the following sections.

3.1.1. Driving Simulations

One major use of VR simulations is driving simulations for both driving training and within the automotive industry; VR provides the ability to create driving simulations in which users can be placed in risky driving scenarios without real danger [ 15 ]. Driving simulators can be useful in multiple capacities, such as observing driving behavior to collect data or training inexperienced drivers in a low-stress environment.

VR driving simulations can be used to train young or novice drivers and help them understand their mistakes or point out some bad driving habits they need to adjust. Within a simulation, drivers can be placed in a virtual vehicle within an environment resembling a cityscape, with their behaviors and actions observed and recorded to later analyze for any issues or mistakes or to see if the drivers made the correct decisions in a given scenario [ 16 ]. After conducting the simulation, drivers can be informed of their mistakes and receive feedback about how to improve their behaviors in an actual driving situation. These driving simulations can also be beneficial in training young drivers with neurodevelopmental disorders such as autism spectrum disorder (ASD) [ 17 ], who may otherwise have difficulties learning in an uncontrolled environment.

Another application of VR driving simulations is the ability to collect real-time data on how users react to different scenarios as drivers on the road in a simulated environment. This data can be used in multiple capacities, such as designing better safety features in a vehicle, providing a better user experience for drivers, developing training modules for drivers, and for use in autonomous vehicle (AV) research and development. AVs have been an emerging field of technology that will continue to develop and advance, with VR simulations continuously providing opportunities for safe and efficient data collection and user testing [ 18 ]. One common issue in the field is developing trust between users and autonomous vehicles and understanding how to mitigate the distrust most people have in this technology [ 19 ]. It is important to ensure users have a certain level of trust in an AV so as to ensure drivers take over when appropriate. Accordingly, putting users in a VR driving simulation in which they interact with an autonomous vehicle virtually can yield substantial amounts of data on how users behave within that environment while also ensuring that users feel safe in the process and can become accustomed to being in an AV [ 20 ].

3.1.2. Product Design and Prototyping

One application of VR that can be useful is the ability to look at 3D models in a virtual space in a way that is difficult to visualize via a screen. Prototypes or preliminary designs for products can be modeled and shown in a virtual environment for test and evaluation purposes [ 21 ]. One significant advantage of showing these models in VR is presenting a virtual prototype or part without spending a lot of time, money, effort, or material on building the prototype in real life. Through simulations, VR can also show how the product would react under different conditions. Simulations can be run in VR to show the effect of different interactions between the prototype and surrounding subjects [ 22 ]. This can help the prototype designers determine if any areas of the prototype need to be improved based on the simulated interaction results. The ability to see the product in a virtual environment can also provide the ability to make changes to VR design for a quick turnaround and faster results, which could increase the speed of prototyping, reduce prototype production waste, and increase the understanding of the functions of the prototype.

3.2. Education

Educational applications of VR have not been utilized much yet, but there are many promising examples and studies of how beneficial VR can be in an educational environment. Using VR can help increase student attention by keeping them engaged with what is happening inside the VR environment [ 23 , 24 ]. Most teenage students find it challenging to pay attention in class, especially when they feel that the discussed topics are not relevant to them. When students use exciting technologies such as VR, they are more interested and engaged with what they are learning while immersed in a virtual environment [ 25 , 26 ]. VR headsets are also useful in blocking out visual and auditory distractions, creating an opportunity for the student to focus on teaching materials better. Such VR approaches open up more opportunities for teachers to interact one-on-one with students and have more useful and beneficial teacher–student interactions [ 27 ].

VR also provides the opportunity for students to construct and practice their own knowledge by being able to engage in meaningful experiences. Students are able to immersively engage in educational activities and gain a better understanding of the topic at hand [ 28 ]. VR also has the capability of transporting students to different environments, allowing them to learn and explore various concepts safely and efficiently. This can be especially useful to demonstrate environments that are impossible to visit in reality, such as underwater or space [ 29 , 30 ].

Mixed reality can be considered an extended VR application, which can be applied to real learning environments, such as exploring laboratory experiments [ 31 ]. Students can wear an HMD that shows information and instructions about the laboratory they will experience and can interact with items in reality to recreate what is simulated to them in VR. Essentially, students are still fully aware of their surroundings while also having a better visual understanding and representation of their task, which can help reduce mistakes, allow students to be more independent, and keep students interested and engaged.

With the start of the COVID-19 pandemic, there has been a sudden increase in virtual learning, with many classes being held via online meeting platforms and others being fully asynchronous. VR offers a new, unique approach to asynchronous learning; VR can create a learning environment in which a student can participate in lectures and ask questions to virtual instructors with pre-generated answers [ 32 ]. It is particularly important for students to feel immersed in the virtual environment in order to keep them engaged [ 33 ]. Virtual environments can be created to look just like real-life classrooms where students can walk around and work with other students on assignments [ 34 ]. The issue with asynchronous classroom experiences is that not all of a student’s questions will necessarily be answered; information will be limited to what is currently updated within the virtual experience. Thus, VR-based virtual education does provide a better experience to students than watching videos online, but it cannot replace the experience of being in a classroom with teachers who can directly engage with students.

With VR technology further advancing, VR could also be used for live, synchronous classes where students can engage with classmates and teachers from the comfort of their homes in real time. This would have been especially beneficial when schools were closed due to the pandemic, but it can also provide a way for students to attend classes while experiencing health difficulties, traveling, or living in other countries, etc. Even though live classes have not yet really been held using VR, such applications can be developed in the future, especially with some of the current development being made in both asynchronous learning and social interaction.

3.3. Public Health

Another domain in which VR has been utilized is within public health and wellness. Due to the immersive nature of VR, it can be used to simulate experiences that can directly impact people’s health. Some examples include providing immersive training simulations to medical personnel, offering a new method of exercise or meditation, and presenting therapists with opportunities to better help and understand their patients.

3.3.1. Medical Training

VR simulations provide the opportunity for medical professionals to practice procedures before operating on a patient, which has proven to help provide patients with better outcomes more consistently and reduce the incidence of mistakes. Preparation and practice in VR help improve patient outcomes because medical personnel are better prepared for each patient’s unique circumstances before operating [ 35 , 36 ].

In terms of learning how to perform procedures, medical students can train in an interactive virtual environment that can be programmed with different scenarios, which allows a student to experience real-life scenarios with virtual patients [ 37 ]. The virtual environment can be programmed in a multitude of diverse ways so the student can be prepared and better accustomed to different types of scenarios they may face with future patients. The simulation can be programmed so that a video can be played, showing how to effectively use a tool or object when the user looks at it [ 38 ]. The simulation can also provide hints or step-by-step instructions to students so they know how to perform the surgery properly. All these practices are much more hands-on than reading a textbook and more realistic than practicing on mannequins with minimal risks to a real patient, which makes VR a perfect tool to assist student learning.

Medical students are not the only ones who can benefit from VR simulations; seasoned medical professionals and surgeons can also benefit from this technology. Patient-specific virtual reality simulations (PSVR) are a technology that allows doctors to practice actual upcoming operations in VR [ 39 ]. This technology allows surgeons to practice customized procedures to match their patients’ specific needs and circumstances. A patient’s medical history and physical attributes can be created in the simulation and programmed with the most likely outcomes. When a surgeon performs a task or action in the simulation, the appropriate or most likely reaction can be programmed to simulate what would occur in real life under the same circumstance. This provides an opportunity for surgeons to plan out their surgery beforehand in a virtual environment, allowing them to be better prepared and more confident in their plan for the surgery ahead [ 40 ].

3.3.2. Exergaming, Fitness and Sports

With the initial focus of VR being on gaming, developers saw an opportunity for the emergence of a genre of games called exergames, in which users participate in physical activities to achieve the goals of the game. “The core concept of exergaming rests on the idea of using vigorous body activity as the input for interacting with engaging digital game content with the hope of supplanting the sedentary activity that typifies traditional game interaction that relies on keyboards, gamepads, and joysticks” [ 41 ]. VR games tend to fall under the category of exergames by requiring the user to stand up and move around in order to interact with the environment. Games such as Beat Saber (Beat Games, Prague, Czech Republic) make the user move around frequently to fulfill the game’s requirements.

Using VR as a workout tool helps gamify exercise, which can greatly assist users in staying motivated and engaged by providing them with goals to achieve during their workout. A study performed by Segura-Orti on dialysis patients shows that patients that used VR exercises instead of conventional physical activities had an increased level of physical activity compared to those who worked out using conventional methods [ 42 , 43 ]. This is probably due to the more enjoyable experience of getting exercise in game form that real life has failed to achieve with exercise apps and challenges. Some current examples include the implementation of treadmills and stationary bicycles with VR applications that allow users to physically run/cycle in place while virtually traveling through a virtual environment. These types of immersive experiences can make users’ workouts more enjoyable and can help encourage those new to fitness to start exercising from home in a new and exciting fashion.

VR technology is also being utilized in sports, where it is used to train athletes to improve their skills and can help provide them with physical therapy and rehabilitation. In terms of athletic training, VR presents a great method of perceptual-cognitive skills training [ 44 ], where users are able to experience and learn from video-based playback in an immersive environment rather than on a screen. This can be especially useful in customizing training for players in large team sports, such as football, basketball, or soccer [ 45 ]. VR allows individuals to repeatedly practice skills with lower risks of harm, which helps reduce injury. When injuries do occur in the real world, VR can be used in the rehabilitation process by allowing athletes to train from anywhere and at any time, even in the absence of a trainer or facility.

3.3.3. Therapy and Meditation

Another use of VR is in mental health therapy and meditation. The immersive nature of VR provides the flexibility to create various types of environments or experiences. Accordingly, VR can be used to experience situations that are hard to come by in real life, or that can be dangerous to go through in real life. For example, for those who suffer from post-traumatic stress disorder (PTSD), VR can be a way to experience situations that can trigger traumatic events within a safe, controlled capacity. Specific scenarios can be recreated in a virtual environment, and the patient can experience them in the presence of a therapist in order to receive help dealing with their trauma [ 46 ]. This type of therapy is similar to exposure therapy, in which patients confront what triggers them in order to slowly heal from their trauma [ 47 ].

For people who have certain disorders that may be hard to explain with words, VR can be a safe way to put people in scenarios that may trigger their disorders and observe their behaviors. Allowing a therapist to observe the situation can give them a better insight into why their patient is reacting in a certain way, which will allow them to better treat their patient [ 48 ].

Another application of VR is to use the immersive nature of the technology for meditation purposes. With the ability to experience a calm virtual environment that fully blocks distractions, VR presents a unique form of meditation that may be otherwise difficult to achieve at home. Studies on the use of VR in meditation have shown a slight increase in positive effects and a state of mindfulness in users after the meditation experience [ 49 ]. One study showed that VR meditation was more successful in reducing pre-exam anxiety in college students than watching a meditation video, where 71% of those using VR reported lower anxiety levels compared to 47% of the control group [ 50 ]. VR mediation has been shown to be useful in calming healthcare workers, especially during the COVID-19 pandemic. Virtual reality plus neurofeedback (VR + NF) meditation was shown to decrease the user’s anger, tension, depression, vigor, fatigue, and confusion [ 51 ]. Navarro-Haro et al. experienced an immersive VR mediation simulation and reported an increase in mindfulness and a reduction in negative emotional stress [ 52 ]. They were also less sad and less angry after the simulation. Mediation experts acknowledge that meditation with VR can be an immensely helpful and unique experience that is not yet fully utilized, and studies such as the one discussed here show promising results for this use of VR.

3.4. Social Interaction

VR provides the ability to transport users to a virtual environment in which they can interact with other users. This provides an opportunity to create social connections that may otherwise be hard to create or maintain. Social interaction via VR can be especially helpful for those with autism, as it provides a way for them to practice their communication skills. Users are able to participate in virtual cognition training to better improve their social skills, such as emotion recognition, social attribution, and analogical reasoning [ 53 ]. There are even programs in which young adults with high-functioning autism can participate that are designed with the purpose of increasing their social skills. These programs train users to better recognize facial expressions, body language, and emotions from a person’s voice [ 54 ]. These programs have lasting effects on the users, as they gain the ability to recognize other people’s emotions within the training that they can carry forward in their lives.

Social virtual reality also provides a new way for people to connect over long distances. Virtual spaces can be created in a VR environment and allow users to interact with each other in a realistic setting; users can have realistic avatars and talk to each other as if they were face-to-face [ 55 ]. This method of communication can be as effective as talking to another person in real life as long as the users feel immersed in the environment. When the users are immersed in the virtual environment, they have a better sense of presence, and their responses are more genuine [ 56 ]. This was especially popular during the COVID-19 pandemic when social distancing and travel restrictions made it much harder for people to see and speak with their loved ones [ 57 ]. Being able to attend events and experience activities with others via VR has provided a substitute for real-life interactions that is more realistic than merely speaking over the phone or via video chat [ 58 ].

3.5. Entertainment

The most prominent application of VR among the general public is within the sphere of entertainment, with VR offering new ways for users to experience several types of media in an immersive capacity.

One such form of media consumption within VR is watching movies, shows, or videos. VR offers new ways for users to experience visual media due to its ability to immerse users in a virtual world. VR displays are able to play 360° videos and allow the users to move around in the virtual environment, which provides the user with a more immersive experience and allows them to interact with the world as they see fit [ 59 ]. Users now have more control over what they want to pay attention to in a video and can experience videos in a whole new way.

Another application is virtual travel and tourism. Virtual tourism allows users to experience immersive tourism in simulated environments based on real landscapes or locations. This can make travel attainable to many people that would otherwise not be able to afford the time or money needed to physically visit faraway destinations. Examples of VR tourism include virtual museum visits, navigating areas using applications such as Google Street View, and virtual tours of popular destinations such as the Grand Canyon or the Great Wall of China. The concept of virtually visiting other countries or worlds has existed since the 90s [ 60 ], but there was a boost in interest recently due to travel constraints during the COVID-19 pandemic [ 61 ], with more people seeking travel experiences from the confines of their homes.

Live music is another form of entertainment that seems to be gaining traction as another large application of VR. Virtual reality has the ability to change the way people experience concerts, offering users the ability to attend and enjoy concerts from anywhere in the world. Prerecorded concerts are already available as a VR experience, with videos of the concerts filmed in 360 using omnidirectional cameras, allowing users to move their heads around and feel like they are physically present at the concert [ 62 ]. This can be an opportunity for users who do not have the ability to travel or could not get tickets to still enjoy the show. This will also allow users to see parts of the concert they could not see even if they were there due to cameras either being positioned on stage or close to the stage. The livestreaming of concerts in VR is still not technologically applicable, but it seems like the music industry is aiming to make it a reality at some point in the future with further VR development. As part of the most significant applications of VR, gaming has gained huge popularity recently, with headsets becoming more accessible and game developers investing more in the VR landscape. Many users have purchased VR headsets to play popular games such as Beat Saber , Super-Hot , and Job Simulator (Menlo Park, Prague, Czech Republic), some of the top-selling VR games. Besides designated VR games, many other games that were not initially made for VR are also being developed to include this capability and expand the options gamers have concerning their in-game experience. The rise of VR gaming popularity in recent years owes to the immersive capabilities of HMDs to immerse the users in the game environment, blocking out all external distractions [ 63 ] and giving the users a better sense of presence [ 64 ]. Players can experience the game from their point of view, which allows users to experience games in a whole new way [ 65 ].

4. Limitations and Side Effects of VR

Despite VR being a powerful and versatile tool, current VR technology has some evident limitations and drawbacks. These limitations include technological limits on what VR can do, how accessible VR is to the general public, and some of the side effects of using VR devices.

4.1. Technological Limitations

As a technology still in the earlier stages of development on a grand scale, VR has made significant leaps in evolution. Still, more substantial progress must occur before VR can be fully utilized in all possible applications and purposes.

Right now, the standardization of VR technology and presentation is still limited [ 66 ]; every developer may have their own interface specifications and functionality associated with their technology, and applications are not easily transferable between devices. The only standardization that can be observed as of now tends to be with popular games that are developed to be used across different VR platforms. It is also hard to troubleshoot bugs and receive proper support for any issues due to the lack of standardization. Hopefully, with time and progress in VR development, the technology can become more streamlined and provide better usability for users and transferability between devices. There are currently efforts to standardize VR, but these efforts are new, and the process is still in its infancy [ 67 ].

Other issues include hardware and software requirements for professional VR development, as most VR development software tends to take up a lot of data space on computers and have high-power consumption [ 68 ]. VR headsets also tend to be very heavy and can cause physical strain on users, causing headaches and pain, especially around the neck and shoulders [ 69 ]. As of now, it is not yet known what kind of detrimental effects VR use will have on users’ eyesight, but it is known that it can cause strain, especially with prolonged usage [ 70 ].

Another common issue is the lag between the user’s movements and the visual display within a VR headset [ 71 ]. A lot of the time, the headset’s tracking does not keep up properly with the user’s movements, which not only decreases their immersion but can also cause dizziness or “cybersickness,” which is explained in more detail below [ 71 , 72 ].

Cybersickness

One of the crucial issues with VR usage is VR-induced motion sickness, or “cybersickness” [ 73 , 74 ]. Cybersickness is a phenomenon where users will feel symptoms similar to motion sickness (i.e., nausea, dizziness, lightheadedness) as a result of using a VR device [ 71 ]. It is not yet known exactly why this occurs, but there are a few theories to explain this phenomenon. The most likely theory is known as the “sensory conflict theory,” which states that the excessive mismatch between the motion a user perceives visually and the lack of the corresponding movement in their body causes a conflict [ 71 , 72 , 75 ]. This happens when there is a disparity between the user’s visual system and vestibular system, which is the sensory system responsible for providing the brain with information about motion, head position, and spatial orientation [ 76 ]. Another explanation for cybersickness is the “ecological hypothesis”, which states that when people are not able to perceive or react to new dynamic situations, postural instability occurs [ 77 ].

Cybersickness does not always come with virtual experiences, but the issue can be exacerbated by several factors. Some individual factors include prolonged VR exposure; the user’s predisposition to motion sickness, fatigue, or nausea; and how adapted a user is to VR applications [ 71 , 78 ]. Cybersickness symptoms also seem to be less frequent when users are sitting instead of standing. Symptoms tend to worsen when a user is experiencing a high-speed simulation or game. Being a passive participant makes users more susceptible to symptoms than when they are in control of the simulation [ 71 , 79 , 80 ].

There are also some technical factors that can increase the likelihood of cybersickness occurring. These issues include noticeable lags (delays in the visual display can cause symptoms), position tracking errors (better head tracking reduces symptoms), and flicker in the visual display [ 71 , 72 ].

Cybersickness is one of the most uncomfortable issues that comes with VR usage, and if users continue to experience these uncomfortable symptoms, this can present a huge hindrance to the widespread development and utilization of VR applications [ 72 , 77 ].

4.2. Accessibility

As VR technology evolves, it is becoming more accessible, especially compared to its earlier stages. The cost of VR headsets on the market is still higher than most people can afford, but their current pricing is on par with most gaming consoles. Headsets such as Oculus Quest 2 cost about $300 for the base model and can be fully operated without the need for a computer, making it one of the more accessible headsets on the market. Most other headsets require using a computer that is “VR-ready”, meaning a high-end computer with a powerful graphics card that can manage VR applications. VR-ready computers tend to be more expensive than most computers, making this type of VR headset more expensive overall and out of reach for most people. This makes cost one of the larger barriers for people to get into VR as regular consumers, which is a hindrance to the growth of VR as a household technology.

VR as a field also includes augmented reality (AR) and mixed reality (XR), which are less immersive forms of virtual experiences where users still operate in the real world with a virtual overlay. AR and XR applications are more accessible to people due to their development for use on mobile devices, which are much more common with most people owning or having access to one. A common example of this type of application is AR games such as the popular Pokémon Go , which combines using a smartphone with a physical exploration of the real world [ 81 ] in search of “Pokémon” around them that can only be observed via their phones. Distances are tracked based on a user’s steps, and users can connect fitness apps to the game in order to increase rewards gained from crossing long distances. These types of games and applications can encourage people to be more physically active by gamifying the walking experience [ 82 ]. Similar smartphone games and applications can be a more accessible entry point for people interested in VR but who lack the funds to invest in an immersive headset and computer setup.

5. Conclusions

This literature review has shown how virtual reality technology has the potential to be a greatly beneficial tool in a multitude of applications and a wide variety of fields. Current applications span different domains such as engineering, education, medicine, and entertainment. With VR technology gaining popularity and traction, more VR applications can be further utilized in the future, both in improving current use cases as well as expanding to more domains. The hope is that with more VR technological breakthroughs and development, the current limitations and issues can be overcome, making long-term VR usage more realistic and accessible to more people.

Overall, VR as a technology is still in its early stages, but more people are becoming interested in it and are optimistic about seeing what kind of changes VR can make in their everyday lives. However, more and more application scenarios are under development by experts from different fields, which allows for more specific applications and development. With how rapidly modern society has adapted to personal computers and smartphones, VR has the opportunity to become the next big technological turning point that will eventually become commonplace in most households.

Funding Statement

This research received no external funding.

Author Contributions

Conceptualization, A.H. and B.J. methodology, A.H. and B.J. validation, B.J.; formal analysis, A.H.; investigation, A.H.; resources, A.H.; data curation, A.H.; writing—original draft preparation, A.H.; writing—review and editing, B.J.; visualization, A.H.; supervision, B.J. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Data availability statement, conflicts of interest.

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

ORIGINAL RESEARCH article

The past, present, and future of virtual and augmented reality research: a network and cluster analysis of the literature.

• 1 Applied Technology for Neuro-Psychology Lab, Istituto Auxologico Italiano, Milan, Italy
• 2 Department of Psychology, Catholic University of the Sacred Heart, Milan, Italy
• 3 Instituto de Investigación e Innovación en Bioingeniería, Universitat Politècnica de València, Valencia, Spain

The recent appearance of low cost virtual reality (VR) technologies – like the Oculus Rift, the HTC Vive and the Sony PlayStation VR – and Mixed Reality Interfaces (MRITF) – like the Hololens – is attracting the attention of users and researchers suggesting it may be the next largest stepping stone in technological innovation. However, the history of VR technology is longer than it may seem: the concept of VR was formulated in the 1960s and the first commercial VR tools appeared in the late 1980s. For this reason, during the last 20 years, 100s of researchers explored the processes, effects, and applications of this technology producing 1000s of scientific papers. What is the outcome of this significant research work? This paper wants to provide an answer to this question by exploring, using advanced scientometric techniques, the existing research corpus in the field. We collected all the existent articles about VR in the Web of Science Core Collection scientific database, and the resultant dataset contained 21,667 records for VR and 9,944 for augmented reality (AR). The bibliographic record contained various fields, such as author, title, abstract, country, and all the references (needed for the citation analysis). The network and cluster analysis of the literature showed a composite panorama characterized by changes and evolutions over the time. Indeed, whether until 5 years ago, the main publication media on VR concerned both conference proceeding and journals, more recently journals constitute the main medium of communication. Similarly, if at first computer science was the leading research field, nowadays clinical areas have increased, as well as the number of countries involved in VR research. The present work discusses the evolution and changes over the time of the use of VR in the main areas of application with an emphasis on the future expected VR’s capacities, increases and challenges. We conclude considering the disruptive contribution that VR/AR/MRITF will be able to get in scientific fields, as well in human communication and interaction, as already happened with the advent of mobile phones by increasing the use and the development of scientific applications (e.g., in clinical areas) and by modifying the social communication and interaction among people.

Introduction

In the last 5 years, virtual reality (VR) and augmented reality (AR) have attracted the interest of investors and the general public, especially after Mark Zuckerberg bought Oculus for two billion dollars ( Luckerson, 2014 ; Castelvecchi, 2016 ). Currently, many other companies, such as Sony, Samsung, HTC, and Google are making huge investments in VR and AR ( Korolov, 2014 ; Ebert, 2015 ; Castelvecchi, 2016 ). However, if VR has been used in research for more than 25 years, and now there are 1000s of papers and many researchers in the field, comprising a strong, interdisciplinary community, AR has a more recent application history ( Burdea and Coiffet, 2003 ; Kim, 2005 ; Bohil et al., 2011 ; Cipresso and Serino, 2014 ; Wexelblat, 2014 ). The study of VR was initiated in the computer graphics field and has been extended to several disciplines ( Sutherland, 1965 , 1968 ; Mazuryk and Gervautz, 1996 ; Choi et al., 2015 ). Currently, videogames supported by VR tools are more popular than the past, and they represent valuables, work-related tools for neuroscientists, psychologists, biologists, and other researchers as well. Indeed, for example, one of the main research purposes lies from navigation studies that include complex experiments that could be done in a laboratory by using VR, whereas, without VR, the researchers would have to go directly into the field, possibly with limited use of intervention. The importance of navigation studies for the functional understanding of human memory in dementia has been a topic of significant interest for a long time, and, in 2014, the Nobel Prize in “Physiology or Medicine” was awarded to John M. O’Keefe, May-Britt Moser, and Edvard I. Moser for their discoveries of nerve cells in the brain that enable a sense of place and navigation. Journals and magazines have extended this knowledge by writing about “the brain GPS,” which gives a clear idea of the mechanism. A huge number of studies have been conducted in clinical settings by using VR ( Bohil et al., 2011 ; Serino et al., 2014 ), and Nobel Prize winner, Edvard I. Moser commented about the use of VR ( Minderer et al., 2016 ), highlighting its importance for research and clinical practice. Moreover, the availability of free tools for VR experimental and computational use has made it easy to access any field ( Riva et al., 2011 ; Cipresso, 2015 ; Brown and Green, 2016 ; Cipresso et al., 2016 ).

Augmented reality is a more recent technology than VR and shows an interdisciplinary application framework, in which, nowadays, education and learning seem to be the most field of research. Indeed, AR allows supporting learning, for example increasing-on content understanding and memory preservation, as well as on learning motivation. However, if VR benefits from clear and more definite fields of application and research areas, AR is still emerging in the scientific scenarios.

In this article, we present a systematic and computational analysis of the emerging interdisciplinary VR and AR fields in terms of various co-citation networks in order to explore the evolution of the intellectual structure of this knowledge domain over time.

Virtual Reality Concepts and Features

The concept of VR could be traced at the mid of 1960 when Ivan Sutherland in a pivotal manuscript attempted to describe VR as a window through which a user perceives the virtual world as if looked, felt, sounded real and in which the user could act realistically ( Sutherland, 1965 ).

Since that time and in accordance with the application area, several definitions have been formulated: for example, Fuchs and Bishop (1992) defined VR as “real-time interactive graphics with 3D models, combined with a display technology that gives the user the immersion in the model world and direct manipulation” ( Fuchs and Bishop, 1992 ); Gigante (1993) described VR as “The illusion of participation in a synthetic environment rather than external observation of such an environment. VR relies on a 3D, stereoscopic head-tracker displays, hand/body tracking and binaural sound. VR is an immersive, multi-sensory experience” ( Gigante, 1993 ); and “Virtual reality refers to immersive, interactive, multi-sensory, viewer-centered, 3D computer generated environments and the combination of technologies required building environments” ( Cruz-Neira, 1993 ).

As we can notice, these definitions, although different, highlight three common features of VR systems: immersion, perception to be present in an environment, and interaction with that environment ( Biocca, 1997 ; Lombard and Ditton, 1997 ; Loomis et al., 1999 ; Heeter, 2000 ; Biocca et al., 2001 ; Bailenson et al., 2006 ; Skalski and Tamborini, 2007 ; Andersen and Thorpe, 2009 ; Slater, 2009 ; Sundar et al., 2010 ). Specifically, immersion concerns the amount of senses stimulated, interactions, and the reality’s similarity of the stimuli used to simulate environments. This feature can depend on the properties of the technological system used to isolate user from reality ( Slater, 2009 ).

Higher or lower degrees of immersion can depend by three types of VR systems provided to the user:

• Non-immersive systems are the simplest and cheapest type of VR applications that use desktops to reproduce images of the world.

• Immersive systems provide a complete simulated experience due to the support of several sensory outputs devices such as head mounted displays (HMDs) for enhancing the stereoscopic view of the environment through the movement of the user’s head, as well as audio and haptic devices.

• Semi-immersive systems such as Fish Tank VR are between the two above. They provide a stereo image of a three dimensional (3D) scene viewed on a monitor using a perspective projection coupled to the head position of the observer ( Ware et al., 1993 ). Higher technological immersive systems have showed a closest experience to reality, giving to the user the illusion of technological non-mediation and feeling him or her of “being in” or present in the virtual environment ( Lombard and Ditton, 1997 ). Furthermore, higher immersive systems, than the other two systems, can give the possibility to add several sensory outputs allowing that the interaction and actions were perceived as real ( Loomis et al., 1999 ; Heeter, 2000 ; Biocca et al., 2001 ).

Finally, the user’s VR experience could be disclosed by measuring presence, realism, and reality’s levels. Presence is a complex psychological feeling of “being there” in VR that involves the sensation and perception of physical presence, as well as the possibility to interact and react as if the user was in the real world ( Heeter, 1992 ). Similarly, the realism’s level corresponds to the degree of expectation that the user has about of the stimuli and experience ( Baños et al., 2000 , 2009 ). If the presented stimuli are similar to reality, VR user’s expectation will be congruent with reality expectation, enhancing VR experience. In the same way, higher is the degree of reality in interaction with the virtual stimuli, higher would be the level of realism of the user’s behaviors ( Baños et al., 2000 , 2009 ).

From Virtual to Augmented Reality

Looking chronologically on VR and AR developments, we can trace the first 3D immersive simulator in 1962, when Morton Heilig created Sensorama, a simulated experience of a motorcycle running through Brooklyn characterized by several sensory impressions, such as audio, olfactory, and haptic stimuli, including also wind to provide a realist experience ( Heilig, 1962 ). In the same years, Ivan Sutherland developed The Ultimate Display that, more than sound, smell, and haptic feedback, included interactive graphics that Sensorama didn’t provide. Furthermore, Philco developed the first HMD that together with The Sword of Damocles of Sutherland was able to update the virtual images by tracking user’s head position and orientation ( Sutherland, 1965 ). In the 70s, the University of North Carolina realized GROPE, the first system of force-feedback and Myron Krueger created VIDEOPLACE an Artificial Reality in which the users’ body figures were captured by cameras and projected on a screen ( Krueger et al., 1985 ). In this way two or more users could interact in the 2D-virtual space. In 1982, the US’ Air Force created the first flight simulator [Visually Coupled Airbone System Simulator (VCASS)] in which the pilot through an HMD could control the pathway and the targets. Generally, the 80’s were the years in which the first commercial devices began to emerge: for example, in 1985 the VPL company commercialized the DataGlove, glove sensors’ equipped able to measure the flexion of fingers, orientation and position, and identify hand gestures. Another example is the Eyephone, created in 1988 by the VPL Company, an HMD system for completely immerging the user in a virtual world. At the end of 80’s, Fake Space Labs created a Binocular-Omni-Orientational Monitor (BOOM), a complex system composed by a stereoscopic-displaying device, providing a moving and broad virtual environment, and a mechanical arm tracking. Furthermore, BOOM offered a more stable image and giving more quickly responses to movements than the HMD devices. Thanks to BOOM and DataGlove, the NASA Ames Research Center developed the Virtual Wind Tunnel in order to research and manipulate airflow in a virtual airplane or space ship. In 1992, the Electronic Visualization Laboratory of the University of Illinois created the CAVE Automatic Virtual Environment, an immersive VR system composed by projectors directed on three or more walls of a room.

More recently, many videogames companies have improved the development and quality of VR devices, like Oculus Rift, or HTC Vive that provide a wider field of view and lower latency. In addition, the actual HMD’s devices can be now combined with other tracker system as eye-tracking systems (FOVE), and motion and orientation sensors (e.g., Razer Hydra, Oculus Touch, or HTC Vive).

Simultaneously, at the beginning of 90’, the Boing Corporation created the first prototype of AR system for showing to employees how set up a wiring tool ( Carmigniani et al., 2011 ). At the same time, Rosenberg and Feiner developed an AR fixture for maintenance assistance, showing that the operator performance enhanced by added virtual information on the fixture to repair ( Rosenberg, 1993 ). In 1993 Loomis and colleagues produced an AR GPS-based system for helping the blind in the assisted navigation through adding spatial audio information ( Loomis et al., 1998 ). Always in the 1993 Julie Martin developed “Dancing in Cyberspace,” an AR theater in which actors interacted with virtual object in real time ( Cathy, 2011 ). Few years later, Feiner et al. (1997) developed the first Mobile AR System (MARS) able to add virtual information about touristic buildings ( Feiner et al., 1997 ). Since then, several applications have been developed: in Thomas et al. (2000) , created ARQuake, a mobile AR video game; in 2008 was created Wikitude that through the mobile camera, internet, and GPS could add information about the user’s environments ( Perry, 2008 ). In 2009 others AR applications, like AR Toolkit and SiteLens have been developed in order to add virtual information to the physical user’s surroundings. In 2011, Total Immersion developed D’Fusion, and AR system for designing projects ( Maurugeon, 2011 ). Finally, in 2013 and 2015, Google developed Google Glass and Google HoloLens, and their usability have begun to test in several field of application.

Virtual Reality Technologies

Technologically, the devices used in the virtual environments play an important role in the creation of successful virtual experiences. According to the literature, can be distinguished input and output devices ( Burdea et al., 1996 ; Burdea and Coiffet, 2003 ). Input devices are the ones that allow the user to communicate with the virtual environment, which can range from a simple joystick or keyboard to a glove allowing capturing finger movements or a tracker able to capture postures. More in detail, keyboard, mouse, trackball, and joystick represent the desktop input devices easy to use, which allow the user to launch continuous and discrete commands or movements to the environment. Other input devices can be represented by tracking devices as bend-sensing gloves that capture hand movements, postures and gestures, or pinch gloves that detect the fingers movements, and trackers able to follow the user’s movements in the physical world and translate them in the virtual environment.

On the contrary, the output devices allow the user to see, hear, smell, or touch everything that happens in the virtual environment. As mentioned above, among the visual devices can be found a wide range of possibilities, from the simplest or least immersive (monitor of a computer) to the most immersive one such as VR glasses or helmets or HMD or CAVE systems.

Furthermore, auditory, speakers, as well as haptic output devices are able to stimulate body senses providing a more real virtual experience. For example, haptic devices can stimulate the touch feeling and force models in the user.

Virtual Reality Applications

Since its appearance, VR has been used in different fields, as for gaming ( Zyda, 2005 ; Meldrum et al., 2012 ), military training ( Alexander et al., 2017 ), architectural design ( Song et al., 2017 ), education ( Englund et al., 2017 ), learning and social skills training ( Schmidt et al., 2017 ), simulations of surgical procedures ( Gallagher et al., 2005 ), assistance to the elderly or psychological treatments are other fields in which VR is bursting strongly ( Freeman et al., 2017 ; Neri et al., 2017 ). A recent and extensive review of Slater and Sanchez-Vives (2016) reported the main VR application evidences, including weakness and advantages, in several research areas, such as science, education, training, physical training, as well as social phenomena, moral behaviors, and could be used in other fields, like travel, meetings, collaboration, industry, news, and entertainment. Furthermore, another review published this year by Freeman et al. (2017) focused on VR in mental health, showing the efficacy of VR in assessing and treating different psychological disorders as anxiety, schizophrenia, depression, and eating disorders.

There are many possibilities that allow the use of VR as a stimulus, replacing real stimuli, recreating experiences, which in the real world would be impossible, with a high realism. This is why VR is widely used in research on new ways of applying psychological treatment or training, for example, to problems arising from phobias (agoraphobia, phobia to fly, etc.) ( Botella et al., 2017 ). Or, simply, it is used like improvement of the traditional systems of motor rehabilitation ( Llorens et al., 2014 ; Borrego et al., 2016 ), developing games that ameliorate the tasks. More in detail, in psychological treatment, Virtual Reality Exposure Therapy (VRET) has showed its efficacy, allowing to patients to gradually face fear stimuli or stressed situations in a safe environment where the psychological and physiological reactions can be controlled by the therapist ( Botella et al., 2017 ).

Augmented Reality Concept

Milgram and Kishino (1994) , conceptualized the Virtual-Reality Continuum that takes into consideration four systems: real environment, augmented reality (AR), augmented virtuality, and virtual environment. AR can be defined a newer technological system in which virtual objects are added to the real world in real-time during the user’s experience. Per Azuma et al. (2001) an AR system should: (1) combine real and virtual objects in a real environment; (2) run interactively and in real-time; (3) register real and virtual objects with each other. Furthermore, even if the AR experiences could seem different from VRs, the quality of AR experience could be considered similarly. Indeed, like in VR, feeling of presence, level of realism, and the degree of reality represent the main features that can be considered the indicators of the quality of AR experiences. Higher the experience is perceived as realistic, and there is congruence between the user’s expectation and the interaction inside the AR environments, higher would be the perception of “being there” physically, and at cognitive and emotional level. The feeling of presence, both in AR and VR environments, is important in acting behaviors like the real ones ( Botella et al., 2005 ; Juan et al., 2005 ; Bretón-López et al., 2010 ; Wrzesien et al., 2013 ).

Augmented Reality Technologies

Technologically, the AR systems, however various, present three common components, such as a geospatial datum for the virtual object, like a visual marker, a surface to project virtual elements to the user, and an adequate processing power for graphics, animation, and merging of images, like a pc and a monitor ( Carmigniani et al., 2011 ). To run, an AR system must also include a camera able to track the user movement for merging the virtual objects, and a visual display, like glasses through that the user can see the virtual objects overlaying to the physical world. To date, two-display systems exist, a video see-through (VST) and an optical see-though (OST) AR systems ( Botella et al., 2005 ; Juan et al., 2005 , 2007 ). The first one, disclosures virtual objects to the user by capturing the real objects/scenes with a camera and overlaying virtual objects, projecting them on a video or a monitor, while the second one, merges the virtual object on a transparent surface, like glasses, through the user see the added elements. The main difference between the two systems is the latency: an OST system could require more time to display the virtual objects than a VST system, generating a time lag between user’s action and performance and the detection of them by the system.

Augmented Reality Applications

Although AR is a more recent technology than VR, it has been investigated and used in several research areas such as architecture ( Lin and Hsu, 2017 ), maintenance ( Schwald and De Laval, 2003 ), entertainment ( Ozbek et al., 2004 ), education ( Nincarean et al., 2013 ; Bacca et al., 2014 ; Akçayır and Akçayır, 2017 ), medicine ( De Buck et al., 2005 ), and psychological treatments ( Juan et al., 2005 ; Botella et al., 2005 , 2010 ; Bretón-López et al., 2010 ; Wrzesien et al., 2011a , b , 2013 ; see the review Chicchi Giglioli et al., 2015 ). More in detail, in education several AR applications have been developed in the last few years showing the positive effects of this technology in supporting learning, such as an increased-on content understanding and memory preservation, as well as on learning motivation ( Radu, 2012 , 2014 ). For example, Ibáñez et al. (2014) developed a AR application on electromagnetism concepts’ learning, in which students could use AR batteries, magnets, cables on real superficies, and the system gave a real-time feedback to students about the correctness of the performance, improving in this way the academic success and motivation ( Di Serio et al., 2013 ). Deeply, AR system allows the possibility to learn visualizing and acting on composite phenomena that traditionally students study theoretically, without the possibility to see and test in real world ( Chien et al., 2010 ; Chen et al., 2011 ).

As well in psychological health, the number of research about AR is increasing, showing its efficacy above all in the treatment of psychological disorder (see the reviews Baus and Bouchard, 2014 ; Chicchi Giglioli et al., 2015 ). For example, in the treatment of anxiety disorders, like phobias, AR exposure therapy (ARET) showed its efficacy in one-session treatment, maintaining the positive impact in a follow-up at 1 or 3 month after. As VRET, ARET provides a safety and an ecological environment where any kind of stimulus is possible, allowing to keep control over the situation experienced by the patients, gradually generating situations of fear or stress. Indeed, in situations of fear, like the phobias for small animals, AR applications allow, in accordance with the patient’s anxiety, to gradually expose patient to fear animals, adding new animals during the session or enlarging their or increasing the speed. The various studies showed that AR is able, at the beginning of the session, to activate patient’s anxiety, for reducing after 1 h of exposition. After the session, patients even more than to better manage animal’s fear and anxiety, ware able to approach, interact, and kill real feared animals.

Materials and Methods

Data collection.

The input data for the analyses were retrieved from the scientific database Web of Science Core Collection ( Falagas et al., 2008 ) and the search terms used were “Virtual Reality” and “Augmented Reality” regarding papers published during the whole timespan covered.

Web of science core collection is composed of: Citation Indexes, Science Citation Index Expanded (SCI-EXPANDED) –1970-present, Social Sciences Citation Index (SSCI) –1970-present, Arts and Humanities Citation Index (A&HCI) –1975-present, Conference Proceedings Citation Index- Science (CPCI-S) –1990-present, Conference Proceedings Citation Index- Social Science & Humanities (CPCI-SSH) –1990-present, Book Citation Index– Science (BKCI-S) –2009-present, Book Citation Index– Social Sciences & Humanities (BKCI-SSH) –2009-present, Emerging Sources Citation Index (ESCI) –2015-present, Chemical Indexes, Current Chemical Reactions (CCR-EXPANDED) –2009-present (Includes Institut National de la Propriete Industrielle structure data back to 1840), Index Chemicus (IC) –2009-present.

The resultant dataset contained a total of 21,667 records for VR and 9,944 records for AR. The bibliographic record contained various fields, such as author, title, abstract, and all of the references (needed for the citation analysis). The research tool to visualize the networks was Cite space v.4.0.R5 SE (32 bit) ( Chen, 2006 ) under Java Runtime v.8 update 91 (build 1.8.0_91-b15). Statistical analyses were conducted using Stata MP-Parallel Edition, Release 14.0, StataCorp LP. Additional information can be found in Supplementary Data Sheet 1 .

The betweenness centrality of a node in a network measures the extent to which the node is part of paths that connect an arbitrary pair of nodes in the network ( Freeman, 1977 ; Brandes, 2001 ; Chen, 2006 ).

Structural metrics include betweenness centrality, modularity, and silhouette. Temporal and hybrid metrics include citation burstness and novelty. All the algorithms are detailed ( Chen et al., 2010 ).

The analysis of the literature on VR shows a complex panorama. At first sight, according to the document-type statistics from the Web of Science (WoS), proceedings papers were used extensively as outcomes of research, comprising almost 48% of the total (10,392 proceedings), with a similar number of articles on the subject amounting to about 47% of the total of 10, 199 articles. However, if we consider only the last 5 years (7,755 articles representing about 36% of the total), the situation changes with about 57% for articles (4,445) and about 33% for proceedings (2,578). Thus, it is clear that VR field has changed in areas other than at the technological level.

About the subject category, nodes and edges are computed as co-occurring subject categories from the Web of Science “Category” field in all the articles.

According to the subject category statistics from the WoS, computer science is the leading category, followed by engineering, and, together, they account for 15,341 articles, which make up about 71% of the total production. However, if we consider just the last 5 years, these categories reach only about 55%, with a total of 4,284 articles (Table 1 and Figure 1 ).

TABLE 1. Category statistics from the WoS for the entire period and the last 5 years.

FIGURE 1. Category from the WoS: network for the last 5 years.

The evidence is very interesting since it highlights that VR is doing very well as new technology with huge interest in hardware and software components. However, with respect to the past, we are witnessing increasing numbers of applications, especially in the medical area. In particular, note its inclusion in the top 10 list of rehabilitation and clinical neurology categories (about 10% of the total production in the last 5 years). It also is interesting that neuroscience and neurology, considered together, have shown an increase from about 12% to about 18.6% over the last 5 years. However, historic areas, such as automation and control systems, imaging science and photographic technology, and robotics, which had accounted for about 14.5% of the total articles ever produced were not even in the top 10 for the last 5 years, with each one accounting for less than 4%.

About the countries, nodes and edges are computed as networks of co-authors countries. Multiple occurrency of a country in the same paper are counted once.

The countries that were very involved in VR research have published for about 47% of the total (10,200 articles altogether). Of the 10,200 articles, the United States, China, England, and Germany published 4921, 2384, 1497, and 1398, respectively. The situation remains the same if we look at the articles published over the last 5 years. However, VR contributions also came from all over the globe, with Japan, Canada, Italy, France, Spain, South Korea, and Netherlands taking positions of prominence, as shown in Figure 2 .

FIGURE 2. Country network (node dimension represents centrality).

Network analysis was conducted to calculate and to represent the centrality index ( Freeman, 1977 ; Brandes, 2001 ), i.e., the dimension of the node in Figure 2 . The top-ranked country, with a centrality index of 0.26, was the United States (2011), and England was second, with a centrality index of 0.25. The third, fourth, and fifth countries were Germany, Italy, and Australia, with centrality indices of 0.15, 0.15, and 0.14, respectively.

About the Institutions, nodes and edges are computed as networks of co-authors Institutions (Figure 3 ).

FIGURE 3. Network of institutions: the dimensions of the nodes represent centrality.

The top-level institutions in VR were in the United States, where three universities were ranked as the top three in the world for published articles; these universities were the University of Illinois (159), the University of South California (147), and the University of Washington (146). The United States also had the eighth-ranked university, which was Iowa State University (116). The second country in the ranking was Canada, with the University of Toronto, which was ranked fifth with 125 articles and McGill University, ranked 10 th with 103 articles.

Other countries in the top-ten list were Netherlands, with the Delft University of Technology ranked fourth with 129 articles; Italy, with IRCCS Istituto Auxologico Italiano, ranked sixth (with the same number of publication of the institution ranked fifth) with 125 published articles; England, which was ranked seventh with 125 articles from the University of London’s Imperial College of Science, Technology, and Medicine; and China with 104 publications, with the Chinese Academy of Science, ranked ninth. Italy’s Istituto Auxologico Italiano, which was ranked fifth, was the only non-university institution ranked in the top-10 list for VR research (Figure 3 ).

About the Journals, nodes, and edges are computed as journal co-citation networks among each journals in the corresponding field.

The top-ranked Journals for citations in VR are Presence: Teleoperators & Virtual Environments with 2689 citations and CyberPsychology & Behavior (Cyberpsychol BEHAV) with 1884 citations; however, looking at the last 5 years, the former had increased the citations, but the latter had a far more significant increase, from about 70% to about 90%, i.e., an increase from 1029 to 1147.

Following the top two journals, IEEE Computer Graphics and Applications ( IEEE Comput Graph) and Advanced Health Telematics and Telemedicine ( St HEAL T) were both left out of the top-10 list based on the last 5 years. The data for the last 5 years also resulted in the inclusion of Experimental Brain Research ( Exp BRAIN RES) (625 citations), Archives of Physical Medicine and Rehabilitation ( Arch PHYS MED REHAB) (622 citations), and Plos ONE (619 citations) in the top-10 list of three journals, which highlighted the categories of rehabilitation and clinical neurology and neuroscience and neurology. Journal co-citation analysis is reported in Figure 4 , which clearly shows four distinct clusters.

FIGURE 4. Co-citation network of journals: the dimensions of the nodes represent centrality. Full list of official abbreviations of WoS journals can be found here: https://images.webofknowledge.com/images/help/WOS/A_abrvjt.html .

Network analysis was conducted to calculate and to represent the centrality index, i.e., the dimensions of the nodes in Figure 4 . The top-ranked item by centrality was Cyberpsychol BEHAV, with a centrality index of 0.29. The second-ranked item was Arch PHYS MED REHAB, with a centrality index of 0.23. The third was Behaviour Research and Therapy (Behav RES THER), with a centrality index of 0.15. The fourth was BRAIN, with a centrality index of 0.14. The fifth was Exp BRAIN RES, with a centrality index of 0.11.

Who’s Who in VR Research

Authors are the heart and brain of research, and their roles in a field are to define the past, present, and future of disciplines and to make significant breakthroughs to make new ideas arise (Figure 5 ).

FIGURE 5. Network of authors’ numbers of publications: the dimensions of the nodes represent the centrality index, and the dimensions of the characters represent the author’s rank.

Virtual reality research is very young and changing with time, but the top-10 authors in this field have made fundamentally significant contributions as pioneers in VR and taking it beyond a mere technological development. The purpose of the following highlights is not to rank researchers; rather, the purpose is to identify the most active researchers in order to understand where the field is going and how they plan for it to get there.

The top-ranked author is Riva G, with 180 publications. The second-ranked author is Rizzo A, with 101 publications. The third is Darzi A, with 97 publications. The forth is Aggarwal R, with 94 publications. The six authors following these three are Slater M, Alcaniz M, Botella C, Wiederhold BK, Kim SI, and Gutierrez-Maldonado J with 90, 90, 85, 75, 59, and 54 publications, respectively (Figure 6 ).

FIGURE 6. Authors’ co-citation network: the dimensions of the nodes represent centrality index, and the dimensions of the characters represent the author’s rank. The 10 authors that appear on the top-10 list are considered to be the pioneers of VR research.

Considering the last 5 years, the situation remains similar, with three new entries in the top-10 list, i.e., Muhlberger A, Cipresso P, and Ahmed K ranked 7th, 8th, and 10th, respectively.

The authors’ publications number network shows the most active authors in VR research. Another relevant analysis for our focus on VR research is to identify the most cited authors in the field.

For this purpose, the authors’ co-citation analysis highlights the authors in term of their impact on the literature considering the entire time span of the field ( White and Griffith, 1981 ; González-Teruel et al., 2015 ; Bu et al., 2016 ). The idea is to focus on the dynamic nature of the community of authors who contribute to the research.

Normally, authors with higher numbers of citations tend to be the scholars who drive the fundamental research and who make the most meaningful impacts on the evolution and development of the field. In the following, we identified the most-cited pioneers in the field of VR Research.

The top-ranked author by citation count is Gallagher (2001), with 694 citations. Second is Seymour (2004), with 668 citations. Third is Slater (1999), with 649 citations. Fourth is Grantcharov (2003), with 563 citations. Fifth is Riva (1999), with 546 citations. Sixth is Aggarwal (2006), with 505 citations. Seventh is Satava (1994), with 477 citations. Eighth is Witmer (2002), with 454 citations. Ninth is Rothbaum (1996), with 448 citations. Tenth is Cruz-neira (1995), with 416 citations.

Citation Network and Cluster Analyses for VR

Another analysis that can be used is the analysis of document co-citation, which allows us to focus on the highly-cited documents that generally are also the most influential in the domain ( Small, 1973 ; González-Teruel et al., 2015 ; Orosz et al., 2016 ).

The top-ranked article by citation counts is Seymour (2002) in Cluster #0, with 317 citations. The second article is Grantcharov (2004) in Cluster #0, with 286 citations. The third is Holden (2005) in Cluster #2, with 179 citations. The 4th is Gallagher et al. (2005) in Cluster #0, with 171 citations. The 5th is Ahlberg (2007) in Cluster #0, with 142 citations. The 6th is Parsons (2008) in Cluster #4, with 136 citations. The 7th is Powers (2008) in Cluster #4, with 134 citations. The 8th is Aggarwal (2007) in Cluster #0, with 121 citations. The 9th is Reznick (2006) in Cluster #0, with 121 citations. The 10th is Munz (2004) in Cluster #0, with 117 citations.

The network of document co-citations is visually complex (Figure 7 ) because it includes 1000s of articles and the links among them. However, this analysis is very important because can be used to identify the possible conglomerate of knowledge in the area, and this is essential for a deep understanding of the area. Thus, for this purpose, a cluster analysis was conducted ( Chen et al., 2010 ; González-Teruel et al., 2015 ; Klavans and Boyack, 2015 ). Figure 8 shows the clusters, which are identified with the two algorithms in Table 2 .

FIGURE 7. Network of document co-citations: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank, and the numbers represent the strengths of the links. It is possible to identify four historical phases (colors: blue, green, yellow, and red) from the past VR research to the current research.

FIGURE 8. Document co-citation network by cluster: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank and the red writing reports the name of the cluster with a short description that was produced with the mutual information algorithm; the clusters are identified with colored polygons.

TABLE 2. Cluster ID and silhouettes as identified with two algorithms ( Chen et al., 2010 ).

The identified clusters highlight clear parts of the literature of VR research, making clear and visible the interdisciplinary nature of this field. However, the dynamics to identify the past, present, and future of VR research cannot be clear yet. We analysed the relationships between these clusters and the temporal dimensions of each article. The results are synthesized in Figure 9 . It is clear that cluster #0 (laparoscopic skill), cluster #2 (gaming and rehabilitation), cluster #4 (therapy), and cluster #14 (surgery) are the most popular areas of VR research. (See Figure 9 and Table 2 to identify the clusters.) From Figure 9 , it also is possible to identify the first phase of laparoscopic skill (cluster #6) and therapy (cluster #7). More generally, it is possible to identify four historical phases (colors: blue, green, yellow, and red) from the past VR research to the current research.

FIGURE 9. Network of document co-citation: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank and the red writing on the right hand side reports the number of the cluster, such as in Table 2 , with a short description that was extracted accordingly.

We were able to identify the top 486 references that had the most citations by using burst citations algorithm. Citation burst is an indicator of a most active area of research. Citation burst is a detection of a burst event, which can last for multiple years as well as a single year. A citation burst provides evidence that a particular publication is associated with a surge of citations. The burst detection was based on Kleinberg’s algorithm ( Kleinberg, 2002 , 2003 ). The top-ranked document by bursts is Seymour (2002) in Cluster #0, with bursts of 88.93. The second is Grantcharov (2004) in Cluster #0, with bursts of 51.40. The third is Saposnik (2010) in Cluster #2, with bursts of 40.84. The fourth is Rothbaum (1995) in Cluster #7, with bursts of 38.94. The fifth is Holden (2005) in Cluster #2, with bursts of 37.52. The sixth is Scott (2000) in Cluster #0, with bursts of 33.39. The seventh is Saposnik (2011) in Cluster #2, with bursts of 33.33. The eighth is Burdea et al. (1996) in Cluster #3, with bursts of 32.42. The ninth is Burdea and Coiffet (2003) in Cluster #22, with bursts of 31.30. The 10th is Taffinder (1998) in Cluster #6, with bursts of 30.96 (Table 3 ).

TABLE 3. Cluster ID and references of burst article.

Citation Network and Cluster Analyses for AR

Looking at Augmented Reality scenario, the top ranked item by citation counts is Azuma (1997) in Cluster #0, with citation counts of 231. The second one is Azuma et al. (2001) in Cluster #0, with citation counts of 220. The third is Van Krevelen (2010) in Cluster #5, with citation counts of 207. The 4th is Lowe (2004) in Cluster #1, with citation counts of 157. The 5th is Wu (2013) in Cluster #4, with citation counts of 144. The 6th is Dunleavy (2009) in Cluster #4, with citation counts of 122. The 7th is Zhou (2008) in Cluster #5, with citation counts of 118. The 8th is Bay (2008) in Cluster #1, with citation counts of 117. The 9th is Newcombe (2011) in Cluster #1, with citation counts of 109. The 10th is Carmigniani et al. (2011) in Cluster #5, with citation counts of 104.

The network of document co-citations is visually complex (Figure 10 ) because it includes 1000s of articles and the links among them. However, this analysis is very important because can be used to identify the possible conglomerate of knowledge in the area, and this is essential for a deep understanding of the area. Thus, for this purpose, a cluster analysis was conducted ( Chen et al., 2010 ; González-Teruel et al., 2015 ; Klavans and Boyack, 2015 ). Figure 11 shows the clusters, which are identified with the two algorithms in Table 3 .

FIGURE 10. Network of document co-citations: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank, and the numbers represent the strengths of the links. It is possible to identify four historical phases (colors: blue, green, yellow, and red) from the past AR research to the current research.

FIGURE 11. Document co-citation network by cluster: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank and the red writing reports the name of the cluster with a short description that was produced with the mutual information algorithm; the clusters are identified with colored polygons.

The identified clusters highlight clear parts of the literature of AR research, making clear and visible the interdisciplinary nature of this field. However, the dynamics to identify the past, present, and future of AR research cannot be clear yet. We analysed the relationships between these clusters and the temporal dimensions of each article. The results are synthesized in Figure 12 . It is clear that cluster #1 (tracking), cluster #4 (education), and cluster #5 (virtual city environment) are the current areas of AR research. (See Figure 12 and Table 3 to identify the clusters.) It is possible to identify four historical phases (colors: blue, green, yellow, and red) from the past AR research to the current research.

FIGURE 12. Network of document co-citation: the dimensions of the nodes represent centrality, the dimensions of the characters represent the rank of the article rank and the red writing on the right hand side reports the number of the cluster, such as in Table 2 , with a short description that was extracted accordingly.

We were able to identify the top 394 references that had the most citations by using burst citations algorithm. Citation burst is an indicator of a most active area of research. Citation burst is a detection of a burst event, which can last for multiple years as well as a single year. A citation burst provides evidence that a particular publication is associated with a surge of citations. The burst detection was based on Kleinberg’s algorithm ( Kleinberg, 2002 , 2003 ). The top ranked document by bursts is Azuma (1997) in Cluster #0, with bursts of 101.64. The second one is Azuma et al. (2001) in Cluster #0, with bursts of 84.23. The third is Lowe (2004) in Cluster #1, with bursts of 64.07. The 4th is Van Krevelen (2010) in Cluster #5, with bursts of 50.99. The 5th is Wu (2013) in Cluster #4, with bursts of 47.23. The 6th is Hartley (2000) in Cluster #0, with bursts of 37.71. The 7th is Dunleavy (2009) in Cluster #4, with bursts of 33.22. The 8th is Kato (1999) in Cluster #0, with bursts of 32.16. The 9th is Newcombe (2011) in Cluster #1, with bursts of 29.72. The 10th is Feiner (1993) in Cluster #8, with bursts of 29.46 (Table 4 ).

TABLE 4. Cluster ID and silhouettes as identified with two algorithms ( Chen et al., 2010 ).

Our findings have profound implications for two reasons. At first the present work highlighted the evolution and development of VR and AR research and provided a clear perspective based on solid data and computational analyses. Secondly our findings on VR made it profoundly clear that the clinical dimension is one of the most investigated ever and seems to increase in quantitative and qualitative aspects, but also include technological development and article in computer science, engineer, and allied sciences.

Figure 9 clarifies the past, present, and future of VR research. The outset of VR research brought a clearly-identifiable development in interfaces for children and medicine, routine use and behavioral-assessment, special effects, systems perspectives, and tutorials. This pioneering era evolved in the period that we can identify as the development era, because it was the period in which VR was used in experiments associated with new technological impulses. Not surprisingly, this was exactly concomitant with the new economy era in which significant investments were made in information technology, and it also was the era of the so-called ‘dot-com bubble’ in the late 1990s. The confluence of pioneering techniques into ergonomic studies within this development era was used to develop the first effective clinical systems for surgery, telemedicine, human spatial navigation, and the first phase of the development of therapy and laparoscopic skills. With the new millennium, VR research switched strongly toward what we can call the clinical-VR era, with its strong emphasis on rehabilitation, neurosurgery, and a new phase of therapy and laparoscopic skills. The number of applications and articles that have been published in the last 5 years are in line with the new technological development that we are experiencing at the hardware level, for example, with so many new, HMDs, and at the software level with an increasing number of independent programmers and VR communities.

Finally, Figure 12 identifies clusters of the literature of AR research, making clear and visible the interdisciplinary nature of this field. The dynamics to identify the past, present, and future of AR research cannot be clear yet, but analyzing the relationships between these clusters and the temporal dimensions of each article tracking, education, and virtual city environment are the current areas of AR research. AR is a new technology that is showing its efficacy in different research fields, and providing a novel way to gather behavioral data and support learning, training, and clinical treatments.

Looking at scientific literature conducted in the last few years, it might appear that most developments in VR and AR studies have focused on clinical aspects. However, the reality is more complex; thus, this perception should be clarified. Although researchers publish studies on the use of VR in clinical settings, each study depends on the technologies available. Industrial development in VR and AR changed a lot in the last 10 years. In the past, the development involved mainly hardware solutions while nowadays, the main efforts pertain to the software when developing virtual solutions. Hardware became a commodity that is often available at low cost. On the other hand, software needs to be customized each time, per each experiment, and this requires huge efforts in term of development. Researchers in AR and VR today need to be able to adapt software in their labs.

Virtual reality and AR developments in this new clinical era rely on computer science and vice versa. The future of VR and AR is becoming more technological than before, and each day, new solutions and products are coming to the market. Both from software and hardware perspectives, the future of AR and VR depends on huge innovations in all fields. The gap between the past and the future of AR and VR research is about the “realism” that was the key aspect in the past versus the “interaction” that is the key aspect now. First 30 years of VR and AR consisted of a continuous research on better resolution and improved perception. Now, researchers already achieved a great resolution and need to focus on making the VR as realistic as possible, which is not simple. In fact, a real experience implies a realistic interaction and not just great resolution. Interactions can be improved in infinite ways through new developments at hardware and software levels.

Interaction in AR and VR is going to be “embodied,” with implication for neuroscientists that are thinking about new solutions to be implemented into the current systems ( Blanke et al., 2015 ; Riva, 2018 ; Riva et al., 2018 ). For example, the use of hands with contactless device (i.e., without gloves) makes the interaction in virtual environments more natural. The Leap Motion device 1 allows one to use of hands in VR without the use of gloves or markers. This simple and low-cost device allows the VR users to interact with virtual objects and related environments in a naturalistic way. When technology is able to be transparent, users can experience increased sense of being in the virtual environments (the so-called sense of presence).

Other forms of interactions are possible and have been developing continuously. For example, tactile and haptic device able to provide a continuous feedback to the users, intensifying their experience also by adding components, such as the feeling of touch and the physical weight of virtual objects, by using force feedback. Another technology available at low cost that facilitates interaction is the motion tracking system, such as Microsoft Kinect, for example. Such technology allows one to track the users’ bodies, allowing them to interact with the virtual environments using body movements, gestures, and interactions. Most HMDs use an embedded system to track HMD position and rotation as well as controllers that are generally placed into the user’s hands. This tracking allows a great degree of interaction and improves the overall virtual experience.

A final emerging approach is the use of digital technologies to simulate not only the external world but also the internal bodily signals ( Azevedo et al., 2017 ; Riva et al., 2017 ): interoception, proprioception and vestibular input. For example, Riva et al. (2017) recently introduced the concept of “sonoception” ( www.sonoception.com ), a novel non-invasive technological paradigm based on wearable acoustic and vibrotactile transducers able to alter internal bodily signals. This approach allowed the development of an interoceptive stimulator that is both able to assess interoceptive time perception in clinical patients ( Di Lernia et al., 2018b ) and to enhance heart rate variability (the short-term vagally mediated component—rMSSD) through the modulation of the subjects’ parasympathetic system ( Di Lernia et al., 2018a ).

In this scenario, it is clear that the future of VR and AR research is not just in clinical applications, although the implications for the patients are huge. The continuous development of VR and AR technologies is the result of research in computer science, engineering, and allied sciences. The reasons for which from our analyses emerged a “clinical era” are threefold. First, all clinical research on VR and AR includes also technological developments, and new technological discoveries are being published in clinical or technological journals but with clinical samples as main subject. As noted in our research, main journals that publish numerous articles on technological developments tested with both healthy and patients include Presence: Teleoperators & Virtual Environments, Cyberpsychology & Behavior (Cyberpsychol BEHAV), and IEEE Computer Graphics and Applications (IEEE Comput Graph). It is clear that researchers in psychology, neuroscience, medicine, and behavioral sciences in general have been investigating whether the technological developments of VR and AR are effective for users, indicating that clinical behavioral research has been incorporating large parts of computer science and engineering. A second aspect to consider is the industrial development. In fact, once a new technology is envisioned and created it goes for a patent application. Once the patent is sent for registration the new technology may be made available for the market, and eventually for journal submission and publication. Moreover, most VR and AR research that that proposes the development of a technology moves directly from the presenting prototype to receiving the patent and introducing it to the market without publishing the findings in scientific paper. Hence, it is clear that if a new technology has been developed for industrial market or consumer, but not for clinical purpose, the research conducted to develop such technology may never be published in a scientific paper. Although our manuscript considered published researches, we have to acknowledge the existence of several researches that have not been published at all. The third reason for which our analyses highlighted a “clinical era” is that several articles on VR and AR have been considered within the Web of Knowledge database, that is our source of references. In this article, we referred to “research” as the one in the database considered. Of course, this is a limitation of our study, since there are several other databases that are of big value in the scientific community, such as IEEE Xplore Digital Library, ACM Digital Library, and many others. Generally, the most important articles in journals published in these databases are also included in the Web of Knowledge database; hence, we are convinced that our study considered the top-level publications in computer science or engineering. Accordingly, we believe that this limitation can be overcome by considering the large number of articles referenced in our research.

Considering all these aspects, it is clear that clinical applications, behavioral aspects, and technological developments in VR and AR research are parts of a more complex situation compared to the old platforms used before the huge diffusion of HMD and solutions. We think that this work might provide a clearer vision for stakeholders, providing evidence of the current research frontiers and the challenges that are expected in the future, highlighting all the connections and implications of the research in several fields, such as clinical, behavioral, industrial, entertainment, educational, and many others.

Author Contributions

PC and GR conceived the idea. PC made data extraction and the computational analyses and wrote the first draft of the article. IG revised the introduction adding important information for the article. PC, IG, MR, and GR revised the article and approved the last version of the article after important input to the article rationale.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer GC declared a shared affiliation, with no collaboration, with the authors PC and GR to the handling Editor at the time of the review.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2018.02086/full#supplementary-material

• ^ https://www.leapmotion.com/

Akçayır, M., and Akçayır, G. (2017). Advantages and challenges associated with augmented reality for education: a systematic review of the literature. Educ. Res. Rev. 20, 1–11. doi: 10.1016/j.edurev.2016.11.002

CrossRef Full Text | Google Scholar

Alexander, T., Westhoven, M., and Conradi, J. (2017). “Virtual environments for competency-oriented education and training,” in Advances in Human Factors, Business Management, Training and Education , (Berlin: Springer International Publishing), 23–29. doi: 10.1007/978-3-319-42070-7_3

Andersen, S. M., and Thorpe, J. S. (2009). An if–thEN theory of personality: significant others and the relational self. J. Res. Pers. 43, 163–170. doi: 10.1016/j.jrp.2008.12.040

Azevedo, R. T., Bennett, N., Bilicki, A., Hooper, J., Markopoulou, F., and Tsakiris, M. (2017). The calming effect of a new wearable device during the anticipation of public speech. Sci. Rep. 7:2285. doi: 10.1038/s41598-017-02274-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., and MacIntyre, B. (2001). Recent advances in augmented reality. IEEE Comp. Graph. Appl. 21, 34–47. doi: 10.1109/38.963459

Bacca, J., Baldiris, S., Fabregat, R., and Graf, S. (2014). Augmented reality trends in education: a systematic review of research and applications. J. Educ. Technol. Soc. 17, 133.

Google Scholar

Bailenson, J. N., Yee, N., Merget, D., and Schroeder, R. (2006). The effect of behavioral realism and form realism of real-time avatar faces on verbal disclosure, nonverbal disclosure, emotion recognition, and copresence in dyadic interaction. Presence 15, 359–372. doi: 10.1162/pres.15.4.359

Baños, R. M., Botella, C., Garcia-Palacios, A., Villa, H., Perpiñá, C., and Alcaniz, M. (2000). Presence and reality judgment in virtual environments: a unitary construct? Cyberpsychol. Behav. 3, 327–335. doi: 10.1089/10949310050078760

Baños, R., Botella, C., García-Palacios, A., Villa, H., Perpiñá, C., and Gallardo, M. (2009). Psychological variables and reality judgment in virtual environments: the roles of absorption and dissociation. Cyberpsychol. Behav. 2, 143–148. doi: 10.1089/cpb.1999.2.143

Baus, O., and Bouchard, S. (2014). Moving from virtual reality exposure-based therapy to augmented reality exposure-based therapy: a review. Front. Hum. Neurosci. 8:112. doi: 10.3389/fnhum.2014.00112

Biocca, F. (1997). The cyborg’s dilemma: progressive embodiment in virtual environments. J. Comput. Mediat. Commun. 3. doi: 10.1111/j.1083-6101.1997

Biocca, F., Harms, C., and Gregg, J. (2001). “The networked minds measure of social presence: pilot test of the factor structure and concurrent validity,” in 4th Annual International Workshop on Presence , Philadelphia, PA, 1–9.

Blanke, O., Slater, M., and Serino, A. (2015). Behavioral, neural, and computational principles of bodily self-consciousness. Neuron 88, 145–166. doi: 10.1016/j.neuron.2015.09.029

Bohil, C. J., Alicea, B., and Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy. Nat. Rev. Neurosci. 12:752. doi: 10.1038/nrn3122

Borrego, A., Latorre, J., Llorens, R., Alcañiz, M., and Noé, E. (2016). Feasibility of a walking virtual reality system for rehabilitation: objective and subjective parameters. J. Neuroeng. Rehabil. 13:68. doi: 10.1186/s12984-016-0174-171

Botella, C., Bretón-López, J., Quero, S., Baños, R. M., and García-Palacios, A. (2010). Treating cockroach phobia with augmented reality. Behav. Ther. 41, 401–413. doi: 10.1016/j.beth.2009.07.002

Botella, C., Fernández-Álvarez, J., Guillén, V., García-Palacios, A., and Baños, R. (2017). Recent progress in virtual reality exposure therapy for phobias: a systematic review. Curr. Psychiatry Rep. 19:42. doi: 10.1007/s11920-017-0788-4

Botella, C. M., Juan, M. C., Baños, R. M., Alcañiz, M., Guillén, V., and Rey, B. (2005). Mixing realities? An application of augmented reality for the treatment of cockroach phobia. Cyberpsychol. Behav. 8, 162–171. doi: 10.1089/cpb.2005.8.162

Brandes, U. (2001). A faster algorithm for betweenness centrality. J. Math. Sociol. 25, 163–177. doi: 10.1080/0022250X.2001.9990249

Bretón-López, J., Quero, S., Botella, C., García-Palacios, A., Baños, R. M., and Alcañiz, M. (2010). An augmented reality system validation for the treatment of cockroach phobia. Cyberpsychol. Behav. Soc. Netw. 13, 705–710. doi: 10.1089/cyber.2009.0170

Brown, A., and Green, T. (2016). Virtual reality: low-cost tools and resources for the classroom. TechTrends 60, 517–519. doi: 10.1007/s11528-016-0102-z

Bu, Y., Liu, T. Y., and Huang, W. B. (2016). MACA: a modified author co-citation analysis method combined with general descriptive metadata of citations. Scientometrics 108, 143–166. doi: 10.1007/s11192-016-1959-5

Burdea, G., Richard, P., and Coiffet, P. (1996). Multimodal virtual reality: input-output devices, system integration, and human factors. Int. J. Hum. Compu. Interact. 8, 5–24. doi: 10.1080/10447319609526138

Burdea, G. C., and Coiffet, P. (2003). Virtual Reality Technology , Vol. 1, Hoboken, NJ: John Wiley & Sons.

Carmigniani, J., Furht, B., Anisetti, M., Ceravolo, P., Damiani, E., and Ivkovic, M. (2011). Augmented reality technologies, systems and applications. Multimed. Tools Appl. 51, 341–377. doi: 10.1007/s11042-010-0660-6

Castelvecchi, D. (2016). Low-cost headsets boost virtual reality’s lab appeal. Nature 533, 153–154. doi: 10.1038/533153a

Cathy (2011). The History of Augmented Reality. The Optical Vision Site. Available at: http://www.theopticalvisionsite.com/history-of-eyewear/the-history-of-augmented-reality/#.UelAUmeAOyA

Chen, C. (2006). CiteSpace II: detecting and visualizing emerging trends and transient patterns in scientific literature. J. Assoc. Inform. Sci. Technol. 57, 359–377. doi: 10.1002/asi.20317

Chen, C., Ibekwe-SanJuan, F., and Hou, J. (2010). The structure and dynamics of cocitation clusters: a multipleperspective cocitation analysis. J. Assoc. Inform. Sci. Technol. 61, 1386–1409. doi: 10.1002/jez.b.22741

Chen, Y. C., Chi, H. L., Hung, W. H., and Kang, S. C. (2011). Use of tangible and augmented reality models in engineering graphics courses. J. Prof. Issues Eng. Educ. Pract. 137, 267–276. doi: 10.1061/(ASCE)EI.1943-5541.0000078

Chicchi Giglioli, I. A., Pallavicini, F., Pedroli, E., Serino, S., and Riva, G. (2015). Augmented reality: a brand new challenge for the assessment and treatment of psychological disorders. Comput. Math. Methods Med. 2015:862942. doi: 10.1155/2015/862942

Chien, C. H., Chen, C. H., and Jeng, T. S. (2010). “An interactive augmented reality system for learning anatomy structure,” in Proceedings of the International Multiconference of Engineers and Computer Scientists , Vol. 1, (Hong Kong: International Association of Engineers), 17–19.

Choi, S., Jung, K., and Noh, S. D. (2015). Virtual reality applications in manufacturing industries: past research, present findings, and future directions. Concurr. Eng. 23, 40–63. doi: 10.1177/1063293X14568814

Cipresso, P. (2015). Modeling behavior dynamics using computational psychometrics within virtual worlds. Front. Psychol. 6:1725. doi: 10.3389/fpsyg.2015.01725

Cipresso, P., and Serino, S. (2014). Virtual Reality: Technologies, Medical Applications and Challenges. Hauppauge, NY: Nova Science Publishers, Inc.

Cipresso, P., Serino, S., and Riva, G. (2016). Psychometric assessment and behavioral experiments using a free virtual reality platform and computational science. BMC Med. Inform. Decis. Mak. 16:37. doi: 10.1186/s12911-016-0276-5

Cruz-Neira, C. (1993). “Virtual reality overview,” in SIGGRAPH 93 Course Notes 21st International Conference on Computer Graphics and Interactive Techniques, Orange County Convention Center , Orlando, FL.

De Buck, S., Maes, F., Ector, J., Bogaert, J., Dymarkowski, S., Heidbuchel, H., et al. (2005). An augmented reality system for patient-specific guidance of cardiac catheter ablation procedures. IEEE Trans. Med. Imaging 24, 1512–1524. doi: 10.1109/TMI.2005.857661

Di Lernia, D., Cipresso, P., Pedroli, E., and Riva, G. (2018a). Toward an embodied medicine: a portable device with programmable interoceptive stimulation for heart rate variability enhancement. Sensors (Basel) 18:2469. doi: 10.3390/s18082469

Di Lernia, D., Serino, S., Pezzulo, G., Pedroli, E., Cipresso, P., and Riva, G. (2018b). Feel the time. Time perception as a function of interoceptive processing. Front. Hum. Neurosci. 12:74. doi: 10.3389/fnhum.2018.00074

Di Serio,Á., Ibáñez, M. B., and Kloos, C. D. (2013). Impact of an augmented reality system on students’ motivation for a visual art course. Comput. Educ. 68, 586–596. doi: 10.1016/j.compedu.2012.03.002

Ebert, C. (2015). Looking into the future. IEEE Softw. 32, 92–97. doi: 10.1109/MS.2015.142

Englund, C., Olofsson, A. D., and Price, L. (2017). Teaching with technology in higher education: understanding conceptual change and development in practice. High. Educ. Res. Dev. 36, 73–87. doi: 10.1080/07294360.2016.1171300

Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., and Pappas, G. (2008). Comparison of pubmed, scopus, web of science, and Google scholar: strengths and weaknesses. FASEB J. 22, 338–342. doi: 10.1096/fj.07-9492LSF

Feiner, S., MacIntyre, B., Hollerer, T., and Webster, A. (1997). “A touring machine: prototyping 3D mobile augmented reality systems for exploring the urban environment,” in Digest of Papers. First International Symposium on Wearable Computers , (Cambridge, MA: IEEE), 74–81. doi: 10.1109/ISWC.1997.629922

Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark, D., Spanlang, B., et al. (2017). Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol. Med. 47, 2393–2400. doi: 10.1017/S003329171700040X

Freeman, L. C. (1977). A set of measures of centrality based on betweenness. Sociometry 40, 35–41. doi: 10.2307/3033543

Fuchs, H., and Bishop, G. (1992). Research Directions in Virtual Environments. Chapel Hill, NC: University of North Carolina at Chapel Hill.

Gallagher, A. G., Ritter, E. M., Champion, H., Higgins, G., Fried, M. P., Moses, G., et al. (2005). Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Ann. Surg. 241:364. doi: 10.1097/01.sla.0000151982.85062.80

Gigante, M. A. (1993). Virtual reality: definitions, history and applications. Virtual Real. Syst. 3–14. doi: 10.1016/B978-0-12-227748-1.50009-3

González-Teruel, A., González-Alcaide, G., Barrios, M., and Abad-García, M. F. (2015). Mapping recent information behavior research: an analysis of co-authorship and co-citation networks. Scientometrics 103, 687–705. doi: 10.1007/s11192-015-1548-z

Heeter, C. (1992). Being there: the subjective experience of presence. Presence 1, 262–271. doi: 10.1162/pres.1992.1.2.262

Heeter, C. (2000). Interactivity in the context of designed experiences. J. Interact. Adv. 1, 3–14. doi: 10.1080/15252019.2000.10722040

Heilig, M. (1962). Sensorama simulator. U.S. Patent No - 3, 870. Virginia: United States Patent and Trade Office.

Ibáñez, M. B., Di Serio,Á., Villarán, D., and Kloos, C. D. (2014). Experimenting with electromagnetism using augmented reality: impact on flow student experience and educational effectiveness. Comput. Educ. 71, 1–13. doi: 10.1016/j.compedu.2013.09.004

Juan, M. C., Alcañiz, M., Calatrava, J., Zaragozá, I., Baños, R., and Botella, C. (2007). “An optical see-through augmented reality system for the treatment of phobia to small animals,” in Virtual Reality, HCII 2007 Lecture Notes in Computer Science , Vol. 4563, ed. R. Schumaker (Berlin: Springer), 651–659.

Juan, M. C., Alcaniz, M., Monserrat, C., Botella, C., Baños, R. M., and Guerrero, B. (2005). Using augmented reality to treat phobias. IEEE Comput. Graph. Appl. 25, 31–37. doi: 10.1109/MCG.2005.143

Kim, G. J. (2005). A SWOT analysis of the field of virtual reality rehabilitation and therapy. Presence 14, 119–146. doi: 10.1162/1054746053967094

Klavans, R., and Boyack, K. W. (2015). Which type of citation analysis generates the most accurate taxonomy of scientific and technical knowledge? J. Assoc. Inform. Sci. Technol. 68, 984–998. doi: 10.1002/asi.23734

Kleinberg, J. (2002). “Bursty and hierarchical structure in streams,” in Paper Presented at the Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2002; Edmonton , Alberta, NT. doi: 10.1145/775047.775061

Kleinberg, J. (2003). Bursty and hierarchical structure in streams. Data Min. Knowl. Discov. 7, 373–397. doi: 10.1023/A:1024940629314

Korolov, M. (2014). The real risks of virtual reality. Risk Manag. 61, 20–24.

Krueger, M. W., Gionfriddo, T., and Hinrichsen, K. (1985). “Videoplace—an artificial reality,” in Proceedings of the ACM SIGCHI Bulletin , Vol. 16, New York, NY: ACM, 35–40. doi: 10.1145/317456.317463

Lin, C. H., and Hsu, P. H. (2017). “Integrating procedural modelling process and immersive VR environment for architectural design education,” in MATEC Web of Conferences , Vol. 104, Les Ulis: EDP Sciences. doi: 10.1051/matecconf/201710403007

Llorens, R., Noé, E., Ferri, J., and Alcañiz, M. (2014). Virtual reality-based telerehabilitation program for balance recovery. A pilot study in hemiparetic individuals with acquired brain injury. Brain Inj. 28:169.

Lombard, M., and Ditton, T. (1997). At the heart of it all: the concept of presence. J. Comput. Mediat. Commun. 3. doi: 10.1111/j.1083-6101.1997.tb00072.x

Loomis, J. M., Blascovich, J. J., and Beall, A. C. (1999). Immersive virtual environment technology as a basic research tool in psychology. Behav. Res. Methods Instr. Comput. 31, 557–564. doi: 10.3758/BF03200735

Loomis, J. M., Golledge, R. G., and Klatzky, R. L. (1998). Navigation system for the blind: auditory display modes and guidance. Presence 7, 193–203. doi: 10.1162/105474698565677

Luckerson, V. (2014). Facebook Buying Oculus Virtual-Reality Company for $2 Billion. Available at: http://time.com/37842/facebook-oculus-rift

Maurugeon, G. (2011). New D’Fusion Supports iPhone4S and 3xDSMax 2012. Available at: http://www.t-immersion.com/blog/2011-12-07/augmented-reality-dfusion-iphone-3dsmax

Mazuryk, T., and Gervautz, M. (1996). Virtual Reality-History, Applications, Technology and Future. Vienna: Institute of Computer Graphics Vienna University of Technology.

Meldrum, D., Glennon, A., Herdman, S., Murray, D., and McConn-Walsh, R. (2012). Virtual reality rehabilitation of balance: assessment of the usability of the nintendo Wii ® fit plus. Disabil. Rehabil. 7, 205–210. doi: 10.3109/17483107.2011.616922

Milgram, P., and Kishino, F. (1994). A taxonomy of mixed reality visual displays. IEICE Trans. Inform. Syst. 77, 1321–1329.

Minderer, M., Harvey, C. D., Donato, F., and Moser, E. I. (2016). Neuroscience: virtual reality explored. Nature 533, 324–325. doi: 10.1038/nature17899

Neri, S. G., Cardoso, J. R., Cruz, L., Lima, R. M., de Oliveira, R. J., Iversen, M. D., et al. (2017). Do virtual reality games improve mobility skills and balance measurements in community-dwelling older adults? Systematic review and meta-analysis. Clin. Rehabil. 31, 1292–1304. doi: 10.1177/0269215517694677

Nincarean, D., Alia, M. B., Halim, N. D. A., and Rahman, M. H. A. (2013). Mobile augmented reality: the potential for education. Procedia Soc. Behav. Sci. 103, 657–664. doi: 10.1016/j.sbspro.2013.10.385

Orosz, K., Farkas, I. J., and Pollner, P. (2016). Quantifying the changing role of past publications. Scientometrics 108, 829–853. doi: 10.1007/s11192-016-1971-9

Ozbek, C. S., Giesler, B., and Dillmann, R. (2004). “Jedi training: playful evaluation of head-mounted augmented reality display systems,” in Proceedings of SPIE. The International Society for Optical Engineering , Vol. 5291, eds R. A. Norwood, M. Eich, and M. G. Kuzyk (Denver, CO), 454–463.

Perry, S. (2008). Wikitude: Android App with Augmented Reality: Mind Blow-Ing. Digital Lifestyles.

Radu, I. (2012). “Why should my students use AR? A comparative review of the educational impacts of augmented-reality,” in Mixed and Augmented Reality (ISMAR), 2012 IEEE International Symposium on , (IEEE), 313–314. doi: 10.1109/ISMAR.2012.6402590

Radu, I. (2014). Augmented reality in education: a meta-review and cross-media analysis. Pers. Ubiquitous Comput. 18, 1533–1543. doi: 10.1007/s00779-013-0747-y

Riva, G. (2018). The neuroscience of body memory: From the self through the space to the others. Cortex 104, 241–260. doi: 10.1016/j.cortex.2017.07.013

Riva, G., Gaggioli, A., Grassi, A., Raspelli, S., Cipresso, P., Pallavicini, F., et al. (2011). NeuroVR 2-A free virtual reality platform for the assessment and treatment in behavioral health care. Stud. Health Technol. Inform. 163, 493–495.

PubMed Abstract | Google Scholar

Riva, G., Serino, S., Di Lernia, D., Pavone, E. F., and Dakanalis, A. (2017). Embodied medicine: mens sana in corpore virtuale sano. Front. Hum. Neurosci. 11:120. doi: 10.3389/fnhum.2017.00120

Riva, G., Wiederhold, B. K., and Mantovani, F. (2018). Neuroscience of virtual reality: from virtual exposure to embodied medicine. Cyberpsychol. Behav. Soc. Netw. doi: 10.1089/cyber.2017.29099.gri [Epub ahead of print].

Rosenberg, L. (1993). “The use of virtual fixtures to enhance telemanipulation with time delay,” in Proceedings of the ASME Winter Anual Meeting on Advances in Robotics, Mechatronics, and Haptic Interfaces , Vol. 49, (New Orleans, LA), 29–36.

Schmidt, M., Beck, D., Glaser, N., and Schmidt, C. (2017). “A prototype immersive, multi-user 3D virtual learning environment for individuals with autism to learn social and life skills: a virtuoso DBR update,” in International Conference on Immersive Learning , Cham: Springer, 185–188. doi: 10.1007/978-3-319-60633-0_15

Schwald, B., and De Laval, B. (2003). An augmented reality system for training and assistance to maintenance in the industrial context. J. WSCG 11.

Serino, S., Cipresso, P., Morganti, F., and Riva, G. (2014). The role of egocentric and allocentric abilities in Alzheimer’s disease: a systematic review. Ageing Res. Rev. 16, 32–44. doi: 10.1016/j.arr.2014.04.004

Skalski, P., and Tamborini, R. (2007). The role of social presence in interactive agent-based persuasion. Media Psychol. 10, 385–413. doi: 10.1080/15213260701533102

Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364, 3549–3557. doi: 10.1098/rstb.2009.0138

Slater, M., and Sanchez-Vives, M. V. (2016). Enhancing our lives with immersive virtual reality. Front. Robot. AI 3:74. doi: 10.3389/frobt.2016.00074

Small, H. (1973). Co-citation in the scientific literature: a new measure of the relationship between two documents. J. Assoc. Inform. Sci. Technol. 24, 265–269. doi: 10.1002/asi.4630240406

Song, H., Chen, F., Peng, Q., Zhang, J., and Gu, P. (2017). Improvement of user experience using virtual reality in open-architecture product design. Proc. Inst. Mech. Eng. B J. Eng. Manufact. 232.

Sundar, S. S., Xu, Q., and Bellur, S. (2010). “Designing interactivity in media interfaces: a communications perspective,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , (Boston, MA: ACM), 2247–2256. doi: 10.1145/1753326.1753666

Sutherland, I. E. (1965). The Ultimate Display. Multimedia: From Wagner to Virtual Reality. New York, NY: Norton.

Sutherland, I. E. (1968). “A head-mounted three dimensional display,” in Proceedings of the December 9-11, 1968, Fall Joint Computer Conference, Part I , (ACM), 757–764. doi: 10.1145/1476589.1476686

Thomas, B., Close, B., Donoghue, J., Squires, J., De Bondi, P., Morris, M., et al. (2000). “ARQuake: an outdoor/indoor augmented reality first person application,” in Digest of Papers. Fourth International Symposium on Wearable Computers , (Atlanta, GA: IEEE), 139–146. doi: 10.1109/ISWC.2000.888480

Ware, C., Arthur, K., and Booth, K. S. (1993). “Fish tank virtual reality,” in Proceedings of the INTERACT’93 and CHI’93 Conference on Human Factors in Computing Systems , (Amsterdam: ACM), 37–42. doi: 10.1145/169059.169066

Wexelblat, A. (ed.) (2014). Virtual Reality: Applications and Explorations. Cambridge, MA: Academic Press.

White, H. D., and Griffith, B. C. (1981). Author cocitation: a literature measure of intellectual structure. J. Assoc. Inform. Sci. Technol. 32, 163–171. doi: 10.1002/asi.4630320302

Wrzesien, M., Alcañiz, M., Botella, C., Burkhardt, J. M., Bretón-López, J., Ortega, M., et al. (2013). The therapeutic lamp: treating small-animal phobias. IEEE Comput. Graph. Appl. 33, 80–86. doi: 10.1109/MCG.2013.12

Wrzesien, M., Burkhardt, J. M., Alcañiz, M., and Botella, C. (2011a). How technology influences the therapeutic process: a comparative field evaluation of augmented reality and in vivo exposure therapy for phobia of small animals. Hum. Comput. Interact. 2011, 523–540.

Wrzesien, M., Burkhardt, J. M., Alcañiz Raya, M., and Botella, C. (2011b). “Mixing psychology and HCI in evaluation of augmented reality mental health technology,” in CHI’11 Extended Abstracts on Human Factors in Computing Systems , (Vancouver, BC: ACM), 2119–2124.

Zyda, M. (2005). From visual simulation to virtual reality to games. Computer 38, 25–32. doi: 10.1109/MC.2005.297

Keywords : virtual reality, augmented reality, quantitative psychology, measurement, psychometrics, scientometrics, computational psychometrics, mathematical psychology

Citation: Cipresso P, Giglioli IAC, Raya MA and Riva G (2018) The Past, Present, and Future of Virtual and Augmented Reality Research: A Network and Cluster Analysis of the Literature. Front. Psychol. 9:2086. doi: 10.3389/fpsyg.2018.02086

Received: 14 December 2017; Accepted: 10 October 2018; Published: 06 November 2018.

Reviewed by:

Copyright © 2018 Cipresso, Giglioli, Raya and Riva. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Pietro Cipresso, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Advertisement

Immersive virtual reality as a pedagogical tool in education: a systematic literature review of quantitative learning outcomes and experimental design

• Open access
• Published: 11 July 2020
• Volume 8 , pages 1–32, ( 2021 )

Cite this article

You have full access to this open access article

• D. Hamilton   ORCID: orcid.org/0000-0002-8659-4385 1 ,
• J. McKechnie 1 ,
• E. Edgerton   ORCID: orcid.org/0000-0001-7389-527X 1 &
• C. Wilson   ORCID: orcid.org/0000-0003-1054-4928 1  

77k Accesses

323 Citations

64 Altmetric

Explore all metrics

The adoption of immersive virtual reality (I-VR) as a pedagogical method in education has challenged the conceptual definition of what constitutes a learning environment. High fidelity graphics and immersive content using head-mounted-displays (HMD) have allowed students to explore complex subjects in a way that traditional teaching methods cannot. Despite this, research focusing on learning outcomes, intervention characteristics, and assessment measures associated with I-VR use has been sparse. To explore this, the current systematic review examined experimental studies published since 2013, where quantitative learning outcomes using HMD based I-VR were compared with less immersive pedagogical methods such as desktop computers and slideshows. A literature search yielded 29 publications that were deemed suitable for inclusion. Included papers were quality assessed using the Medical Education Research Study Quality Instrument (MERSQI). Most studies found a significant advantage of utilising I-VR in education, whilst a smaller number found no significant differences in attainment level regardless of whether I-VR or non-immersive methods were utilised. Only two studies found clear detrimental effects of using I-VR. However, most studies used short interventions, did not examine information retention, and were focused mainly on the teaching of scientific topics such as biology or physics. In addition, the MERSQI showed that the methods used to evaluate learning outcomes are often inadequate and this may affect the interpretation of I-VR’s utility. The review highlights that a rigorous methodological approach through the identification of appropriate assessment measures, intervention characteristics, and learning outcomes is essential to understanding the potential of I-VR as a pedagogical method.

Similar content being viewed by others

Systematic literature review and bibliometric analysis on virtual reality and education

Mario A. Rojas-Sánchez, Pedro R. Palos-Sánchez & José A. Folgado-Fernández

Gamification and Game Based Learning for Vocational Education and Training: A Systematic Literature Review

Fazlida Dahalan, Norlidah Alias & Mohd Shahril Nizam Shaharom

The Cognitive Affective Model of Immersive Learning (CAMIL): a Theoretical Research-Based Model of Learning in Immersive Virtual Reality

Guido Makransky & Gustav B. Petersen

Avoid common mistakes on your manuscript.

Introduction

The increasing financial feasibility of virtual reality (VR) has allowed for educational institutions to incorporate the technology into their teaching. According to research, 96% of universities and 79% of colleges in the UK are now utilising augmented or virtual reality in some capacity (UKAuthority 2019 ). In addition, the rising power of personal computers and associated hardware has led to a revolution in graphical fidelity, with ever more complex and realistic simulations and virtual worlds (Slater 2018 ). As Dickey ( 2005 ) alludes to, this has both challenged and expanded the very conceptual definition of what is defined as a learning environment. Where once this would have been restricted to classroom teaching or field trips, VR’s innate ability to give users a sense of presence and immersion has opened new possibilities in education if implemented appropriately (Häfner et al. 2018 ).

The use of technology-aided education as a pedagogical method is not a modern phenomenon, and investigations into its utility have been studied for almost half a century. As far back as the 1970s, Ellinger and Frankland ( 1976 ) found that the use of early computers to teach economic principles produced comparative learning outcomes with traditional didactic methods such as lectures. However, as Jensen and Konradsen ( 2018 ) allude to, it was with the release of the Oculus Rift in 2013 that VR became synonymous with head-mounted-display (HMD) based VR. This had several ramifications. First, HMDs became economically feasible for consumers and educational institutions to acquire en masse , due to a significant drop in price (Hodgson et al. 2015 ). As Olmos et al. ( 2018 ) remarks, the economic viability of VR has tackled one of the main entry barriers to adopting the technology. And secondly, academic research into the potential benefits of I-VR in education starts to expand, as well as its applied use in pedagogical settings (Hodgson et al. 2019 ). One of VR’s most important contributions to education is that it has allowed students to repeatedly practice complex and demanding tasks in a safe environment. This is particularly true of procedural tasks such as surgical operations or dental procedures that cannot be carried out for real until a certain level of competency has been achieved (Alaraj et al. 2011 ; Larsen et al. 2012 ). Additionally, VR has allowed for students to gain cognitive skills by way of experiential learning, such as exposing them to environments that would be too logistically problematic to visit in reality (Çalişkan 2011 ). For instance, by using a HMD, Bailenson et al. ( 2018 ) were able to expose students to an underwater environment to facilitate learning about climate change. VR has made an important contribution to education in that it has allowed for students to directly experience environments or situations that are difficult to replicate by using traditional teaching methods such as lectures, slideshows, or 2D videos.

A concise definition of VR’s key characteristics is challenging due to the ever-changing nature of the technology. However, Sherman and Craig ( 2003 ) proposed that there are a number of constituent elements that must underpin the VR experience, ultimately leading to the life-like perception of the virtual environment. These include the necessity for VR to be immersive, in that the participant’s own cognitive faculties produce a sense of being present and involved in the virtual space, often with reduced awareness of what is happening in the real-world around them. Additionally, the virtual space should offer a degree of interactivity, in that the user can manipulate the environment and test variables. This can include interacting with objects, virtual avatars, or even collaborating with other real-life users within the computer-generated space.

Definition of key terms

Due to the multidisciplinary nature of VR research and its pedagogical applications, it is important to define key terms used. VR can broadly be broken down into two main categories: desktop VR (D-VR), and immersive-VR (I-VR). D-VR is typically classified as non-immersive, in that a headset is not used, and the participant will be controlling and manipulating the virtual environment on a computer screen with traditional keyboard and mouse hardware (Lee et al. 2010 ). On the other hand, I-VR is typically multi-modal in nature by providing a sense of immersion in the environment through 360° visuals by aid of a HMD, auditory stimulation through the use of earphones, and increasingly the proprioception of limbs by way of controllers and tracking (Freina and Ott 2015 ; Howard-Jones et al. 2015 ; Murcia-López and Steed 2016 ). Although there are a range of HMDs on the market, from high-end hardware like the HTC Vive, to viable low-cost options like the Google Cardboard, they all utilise the same core principals of operation (Brown and Green 2016 ). Typically, a HMD will feature a set of embedded liquid crystal displays (LCD) which will present each eye an image from a slightly different angle. This mimics natural optic function by allowing the wearer to view a stereoscopic image complete with depth perception and a wide field of view. Mobile VR headsets can achieve the same effect using a single display by dividing the screen down the middle and presenting each half to the corresponding eye. Therefore, the current review defines a HMD as a device worn over the head, which provides a stereoscopic computer-generated or 360 ° video image to the user. This includes tethered (connected to a computer), stand-alone (no computer needed), or mobile VR headsets (mobile/cell phone connected to a HMD).

Previous literature and reviews

There have been a number of systematic reviews that have previously explored the relationship between VR and pedagogical attainment. Lee ( 1999 ) reviewed 19 studies from as far back as 1976 and found that 66% of students in simulation groups outperformed those in their respective control groups. However, this review did not focus exclusively on an educational level or age range, so featured both young kindergarten children, as well as higher education students. As a result, the generalisability of VR’s effectiveness as a pedagogical method is difficult to ascertain, with significant differences in age, task difficulty, and applications. Furthermore, all the studies are dated in terms of the technology utilised and feature early D-VR programmes and rudimental computer simulations. This early technology may be primitive when compared with the high-fidelity graphics and immersive components of contemporary technology. Nevertheless, these early studies do help exemplify that the use of technology in education is not a new concept, and computer-based simulations have long been employed as a way of facilitating learning.

A more recent analysis was undertaken by Merchant et al. ( 2014 ), and looked at three specific sub-categories of VR: games, simulations, and virtual worlds. Games give the actor autonomy and freedom to move around the virtual world, testing hypotheses, achieving goals, and eliciting motivation and learning through immersion (Gee 2004 ). Simulations attempt to recreate a real-world environment that can help facilitate learning by allowing for the testing of variables and resulting outcomes. Finally, virtual worlds can provide an immersive or non-immersive sense of presence in a three-dimensional (3D) world, and the ability to manipulate, interact, or construct objects. Furthermore, virtual worlds can give the opportunity for multiple users to interact with one another within the digital environment (Dickey 2005 ). The meta-analysis showed that although game-based VR produced the highest learning outcomes, simulations and virtual worlds were also effective at increasing educational attainment. Once again, the limitation of this review is that it did not restrict its analysis to exclusively one domain of education. Although higher education made up the greatest number of studies, research from elementary and middle school were also included in the analysis.

One of the most recent systematic reviews to look exclusively at I-VR through the utilisation of HMDs was carried out by Jensen and Konradsen ( 2018 ). In their comprehensive search of existing literature published between 2013 and 2017, the review identified 21 quantitative and qualitative papers that focused on both learning outcomes in I-VR, and subjective attitudes and experiences on the part of the user. The review found limited effectiveness of HMD in the acquisition of cognitive, psychomotor, and affective skills when compared with less immersive technologies. However, Jensen and Konradsen ( 2018 ) did highlight the relatively low quality of studies included as a concern, and this may impede the ability to draw firm conclusions about the educational utility of I-VR.

Rationale for review

There are several fundamental reasons that necessitate an updated assessment of the topic area, such as the increase in relevant published literature, as well as the narrow scope of previous reviews. The last major review looking at I-VR and HMDs as an educational tool was carried out by Jensen and Konradsen ( 2018 ), with the most recent studies featured in that paper being published in 2016. Since then, there has been a significant increase in relevant published literature, with > 70% of the papers included in the current review being published since 2017, and therefore not included in the previous systematic review. Additionally, unlike previous reviews, the current examination of I-VR’s pedagogical utility focuses exclusively on studies where I-VR is directly compared to a less immersive method of learning. As a result, the current paper is able to highlight not only whether I-VR is an effective medium, but also whether it is more effective when compared to alternative methods. Additionally, no other systematic review looking at I-VR and HMDs has had a particular focus on the experimental design, assessment measures, and intervention characteristics of the included studies. The review also addresses the underlying methodology of the included studies, to offer an understanding of how I-VR is being employed in experimental literature. Based upon the findings of previous studies as well as areas yet to be sufficiently explored, this paper has a number of core research questions:

To assess the subject area, discipline, and learning domain that I-VR has been employed in.

Understand where I-VR confers an educational benefit in terms of quantitative learning outcomes over non-immersive and traditional teaching methods.

To examine the experimental design of studies, focusing on how learning outcomes are assessed, and how the I-VR intervention is delivered.

To inform future experimental and applied practice in the field of pedagogical I-VR application.

Methodology

Search strategy.

The current systematic review included peer-reviewed journal articles and conference proceedings that passed all the inclusion criteria detailed. An initial scoping review identified seven databases that could be utilised in a comprehensive literature review, as well as associated keywords and search terms. These included Web of Science (Core Collection), Science Direct, Sage, IEEE Xplore, EBSCO, Taylor & Francis, and Google Scholar. These databases encompass a mixture of general, social science, and technological literature.

Each of the seven databases was searched using a series of keywords based on the following Boolean logic string:

("Virtual Reality" OR "Virtual-Reality" OR “Immersive Virtual Reality” OR “Head Mounted Display” OR “Immersive Simulation”) AND (Education OR Training OR Learning OR Teaching)

Due to the scope and parameters of the research objectives, only peer-reviewed literature published between January 2013 and December 2018 was included in the final review. Early access articles due to be published in 2019 were also included if these were found using the database searches. Date criteria was based upon an initial scoping review that found a substantial growth in relevant I-VR literature from 2013 onwards. A major contributing factor was the release of the Oculus Rift Development Kit 1 (DK-1) in early 2013, which is regarded as one of the first economically viable and high quality HMDs that could be used both within educational institutions, and at home (Lyne 2013 ).

The literature search across the databases yielded more than 12,000 references from a variety of sources. After the removal of duplicate records, 9,359 unique references were included for the title and abstract screening stage of the review.

Selection and screening

The open and general nature of the search string used led to a large number of references being returned for screening. As Jensen and Konradsen ( 2018 ) already alluded to in the last major review, VR research transcends various academic disciplines. The result is a lack of a clear taxonomy of definitions and terms. This means a wide net must be cast to ensure comprehensive capture of relevant material. This review defined I-VR as either a completely computer-generated environment, or the viewing of captured 360 ° video through the use of a HMD. Studies that utilised surgical or dental simulators and trainers such as the da Vinci Surgical System, were excluded as these represent a separate domain of both technological and pedagogical application. For example, surgical simulation based VR typically combines computer-generated visuals with simulated surgical tools, haptic feedback, and robotic components (Li et al. 2017 ). This type of technology would therefore not be applicable for general pedagogical application. Additionally, references were excluded if they: (1) focused on using I-VR as a rehabilitation or therapeutic tool; (2) were not in English; or (3) where the full-text was not available.

After title and abstract screening was performed, 197 references remained to be included in the full-text review. Each reference had to pass an inclusion flowchart based on each of the following criteria:

The population being sampled was from a high school, further or higher education establishment, or was an adult education student.

Population sampled did not have a developmental or neurological condition, nor could VR be used as a rehabilitation tool.

Paper described an experimental or quasi-experimental trial with at least one control group.

At least one group had to have undergone an educational HMD I-VR experience, and was compared with another group who underwent a non-immersive or traditional pedagogical method of education (e.g. Desktop VR, PowerPoint, traditional lecture).

A quantitative and objective learning outcome such as tests scores, completion time, or knowledge retention was used to assess effectiveness.

After full-text screening, 29 references passed all stages and were included in the systematic review. See Fig.  1 for a summary of the selection process by stage.

Stage-by-stage selection process

Inter-rater reliability checks were conducted at the title and abstract screening stage to assess the agreement of included studies. There were four individual evaluators that assessed the suitability of each reference based upon the inclusion criteria, which yielded an average agreement of 96%. Where any disagreement existed, the paper was discussed among all assessors until a unanimous decision was reached as to its suitability.

Quality assessment tool

To assess the quality of the studies, the Medical Education Research Study Quality Instrument (MERSQI) was used (Reed et al. 2007 ). Although this tool was primarily designed to examine the quality of studies in the field of medical education, it is in practice subject neutral. As the MERSQI assesses not only the quality of experimental design and outcomes measures, but also the assessment instrumentation used, it was viewed as a suitable and comprehensive tool for quality appraisal. In addition, the same instrument was used in a previously peer-reviewed systematic review examining VR, by Jensen and Konradsen ( 2018 ).

The MERSQI tool covers six quality assessment domains. These include: study design, sampling, type of data, validity of evaluation instrument, data analysis, and outcomes. Each domain is scored out of three, with a maximum overall score of 18. Unlike Jensen and Konradsen ( 2018 ), the current review gave full points in the study design category for experimental trials with participant randomisation, as well as appropriate pre-intervention measures. This decision was made as true randomised control trials featuring random sampling is unrealistic in I-VR pedagogical research, as the participant sample can only be drawn from an educational establishment.

Quality of studies

The first domain examined for quality was the study design of the papers. There were 20 studies (69%) that featured an experimental design with stated random allocation of participants between control and experimental group. The review featured nine studies (31%) that were quasi-experimental in nature, meaning there was non-random allocation of participants into experimental groups.

Only one of the studies featured participants being studied at more than one institution, with most of the studies included ( N  = 28) only sampling from a single establishment. All studies produced response rates of over 75%, which means they were given the highest score in that domain.

In terms of the type of data presented, all included studies featured an objective measure of learning outcomes such as test scores or completion times. No studies used self-assessment on the part of the participant to gauge learning outcomes.

The most pronounced weakness of the studies included in the review was the validity of the evaluation instrument used to assess learning outcomes. This domain pertained to the physical assessment instrumentation such as the quiz, test, or questionnaire that was given to the participant. Only six of the included studies (21%) reported the internal structure sufficiently through dimensionality, measurement invariance, or reliability using the criteria set down by Rios and Wells ( 2014 ). In addition, only 10 studies (34%) stated how the content was validated, with the majority ( N  = 19) not reporting this information. Only three studies (Kozhevnikov et al. 2013 ; Makransky et al. 2017 ; Molina-Carmona et al. 2018 ) appropriately outlined both the internal structure and validity of evaluation content. The majority of studies ( N  = 16) did not report either item.

Of the 29 studies in the current review, 26 scored full marks on the data analysis domain with both an appropriate and sufficiently complex analysis and reporting of the findings. Three studies scored lower than this due to reporting descriptive statistics only (Angulo and de Velasco 2013 ; Babu et al. 2018 ; Ray and Deb 2016 ).

Overall, the average quality score of a study in this systematic review was 12.7 with a range of 10.5–14.5 (SD = 1.0). This was 1.8 points higher than the review carried out by Jensen and Konradsen ( 2018 ), which could in part be due to differences in study design criteria which was previously outlined. A full summary of the MERSQI scores for each study can be found in Table 2 in the Appendix.

Subject areas and learning domains

Table 1 provides a summary of all 29 articles that were included in the review. Studies were first categorised by the population that was sampled. Most I-VR studies took place in a higher education establishment (college or university) using undergraduate or postgraduate students ( N  = 25). A smaller number of studies used high school pupils ( N  = 2), or adult education students ( N  = 2) such as those in vocational or work-based programmes.

Each of the included studies were then examined for the topic and subject area they pertained to. This was based upon the nature of the VR experience, participant pool, and intervention. In total, six main subject areas were identified. This included: medicine ( N  = 4), science (biology, chemistry, and physics) ( N  = 13), social science (human geography) ( N  = 1), computer science ( N  = 2), engineering and architecture ( N  = 7), and safety education ( N  = 1). One of the included studies (Molina-Carmona et al. 2018 ) did not neatly fit into one of the pre-defined categories as it utilised I-VR to teach abstract spatial concept abilities to multimedia engineering students. It was therefore categorised as ‘other’. Figure  2 shows the percentage of papers included by subject area.

Percentage of papers per subject area

In addition to the subject area, the learning outcomes were also categorised into three specific domains based upon the findings of previous systematic reviews, as well as the taxonomy of learning developed by Bloom et al. ( 1956 ). The first was cognitive which related to studies that intended to teach specific declarative information or knowledge. The second was procedural which intends to teach the user how to perform a specific task or learn psychomotor skills that pertain to a certain activity. Finally, the third learning outcome was affective skills which can be defined as a growth in areas relating to emotion and attitude. Most of the included studies ( N  = 24) concentrated on the cognitive domain, with two studies focusing on purely procedural and psychomotor skills. The remaining studies were a blend of two domains with Sankaranarayanan et al. ( 2018 ) and Smith et al. ( 2018 ) examining both cognitive and procedural skills, and Gutiérrez-Maldonado et al. ( 2015 ) utilising both cognitive knowledge and affective awareness in psychiatric diagnosis training. Figure  3 shows the percentage of studies included by learning domain.

Percentage of papers per learning domain

Experimental design

Outcome measures.

A thorough understanding of the role of I-VR as a pedagogical practice can only be fully appreciated when consideration is given to the assessment instrumentation and outcome measures used to assess its utility. As previously mentioned, when analysing the quality of the included studies, it was the evaluation instrumentation itself that was shown to have the most pronounced weakness.

To assess the evaluation instruments being employed, the measures were broken down into two broad domains: outcome measures, and assessment instrumentation. Outcome measures can broadly be defined as how learning outcomes were quantified (e.g. by comparing test scores). Assessment instrumentation pertains to the evaluative instrument itself that is used to measure the learning outcomes (e.g. multiple-choice questionnaire, exam style questions). Twenty-seven of the included studies (93%) used test scores to assess learning outcomes, with the majority using this as their sole method. There were four studies that used completion time as a metric of learning outcome, although only one study (Bharathi and Tucker 2015 ) used this method exclusively. There was one study (Sankaranarayanan et al. 2018 ) that used the correct order of operation in a procedural task as one of its main outcome measures. There were three papers that utilised other outcome measures that could not be easily categorised. For instance Greenwald et al. ( 2018 ) used counting the number of moves needed to complete a task, Webster ( 2016 ) used the performance on a virtual jigsaw puzzle, and Angulo and de Velasco ( 2013 ) used a mixture of scores and evaluations of an architectural space.

Assessment instrumentation

In terms of the direct assessment instrumentation used to examine outcome measures, there was a heavy reliance on the multiple-choice questionnaire (MCQ). There were eighteen (62%) studies that utilised this method of assessment, with the majority of those using it as their sole evaluation instrument. Only five studies used extended answer questions (long or short form) to probe for a deeper understanding of the educational content, which was usually done in combination MCQs. The studies that included the teaching of procedural skills used marking criteria and checklists to assess whether the correct order was being followed. For instance Yoganathan et al. ( 2018 ) had an expert assessor use marking criteria to assess the knot tying skills of students. Similarly, Smith et al. ( 2018 ) had evaluators observe students with a decontamination checklist which evaluated performance based upon certain key tasks that were performed.

There were a smaller number of studies that used more novel instrumentation and methods for evaluation, such as the utilisation of labelling and identifying parts of a 3D model (e.g. Babu et al. 2018 ; Moro et al. 2017 ; Stepan et al. 2017 ). Fogarty et al. ( 2017 ) probed spatial and conceptual understanding in their assessment instrument by having participants draw shapes based on their understanding of structural engineering principles. Additionally, Alhalabi ( 2016 ) used quizzes on both mathematical knowledge, and the appropriate understanding of graphics and charts as an assessment measure for engineering students.

There were three studies (Liou and Chang 2018 ; Madden et al. 2018 ; Ray and Deb 2016 ) where the nature of the assessment instrumentation could not be definitively ascertained from the description.

The majority of studies (62%) utilised the pretest–posttest design by comparing the test scores pre-intervention with those after the I-VR experience. The remainder of the studies tended to assess post-intervention scores only, usually by comparing the difference in learning outcome between I-VR and one or more control group. Less conventional means of post-intervention comparison was sometimes utilised, such as Johnston et al. ( 2018 ) comparing the average score on a specific exam question that pertained to an I-VR experience that some student did or did not undertake.

There were four studies that examined the short to medium term retention rate of information and learning through follow-up assessment. This ranged from as soon as 1 day after the initial I-VR experience (Babu et al. 2018 ), through to 6 months post-intervention (Smith et al. 2018 ). Olmos-Raya et al. ( 2018 ) and Stepan et al. ( 2017 ) had follow-up assessments at 1-week and 8-weeks, respectively.

Intervention characteristics

In addition to having appropriate assessment measures, it is also important to examine the nature of the I-VR intervention itself. The most popular HMDs used were the Oculus ( N  = 13) and HTC Vive ( N  = 7). There were seven studies that used a form of mobile VR headset such as the Google Cardboard or Samsung Gear VR. In one study (Yoganathan et al. 2018 ) the exact HMD system used could not be definitively ascertained. Figure  4 provides a breakdown of the HMDs used in the included studies.

HMDs used in studies

Most studies (72%) featured only a single intervention with the I-VR experience, meaning that the student was exposed to the technology just once. There were a few exceptions to this, with Ostrander et al. ( 2018 ) having seven individual I-VR experiences in their manufacturing lesson, as well as Ray and Deb ( 2016 ) utilising smartphone based I-VR over the course of 16 sessions. Other studies allowed a greater degree of freedom in the number of interventions or times that a student could use I-VR. This was usually a result of time being dedicated to the technology through scheduled classes or lab times (e.g. Akbulut et al. 2018 ; Alhalabi 2016 ; Molina-Carmona et al. 2018 ). Despite this, the I-VR intervention was usually a single and isolated one.

As well as most of the studies featuring a single intervention, the exposure duration was also typically short, ranging from 6 to 30 mins. Generally, the exception to this was when the I-VR exposure lasted as long as it took the participant to complete a specific task, assessment, or procedure within the immersive environment (e.g. Babu et al. 2018 ; Bharathi and Tucker 2015 ; Greenwald et al. 2018 ; Sankaranarayanan et al. 2018 ). Molina-Carmona et al. ( 2018 ) supplemented the limited intervention duration by allowing participants to take the HMD away with them, so they could access the educational content for 2 weeks outside the classroom. However, just as with the number of interventions, exposure duration tended to be short, lasting on average 13 mins for those I-VR experiences that had a set time limit.

Most of the studies (62%) utilised I-VR as the sole method of learning, and did not combine the technology with additional pedagogical practices or materials to encourage learning. Only a limited number of studies (38%) supplemented the I-VR lesson by providing additional aids that were designed to complement the educational experience. For example, Smith et al. ( 2018 ) and Stepan et al. ( 2017 ) both had participants use web-based modules and textbooks in addition to the I-VR experience before testing them on learning outcomes. A number of the included studies also utilised lecture based instruction or scheduled class time to operate in tandem with the I-VR environments (e.g. Akbulut et al. 2018 ; Fogarty et al. 2017 ; Johnston et al. 2018 ; Ray and Deb 2016 ; Sankaranarayanan et al. 2018 ).

Theoretical frameworks

A fundamental component of any educational tool or activity is to ground its use in learning theory or educational paradigms. Learning theories can broadly be broken down and defined by proposals regarding how student imbibe, process, and retain the information that they have learned (Pritchard 2017 ; Schunk 2011 ). When applied to educational I-VR, these theories should provide a pedagogical framework and foundation as how best to design interventions. Papers were examined for explicit statements regarding the theoretical basis for the study. Those papers that only mentioned theoretical approaches as part of the introduction or literature review were not deemed to have explicitly stated them. The majority of studies ( N  = 24) made no mention of a theoretical approach underpinning the intervention. There were two studies that applied a generative learning framework (Makransky et al. 2017 ; Parong and Mayer 2018 ). This can be defined as an approach where the learner will actively integrate new knowledge with information that is already stored in the brain (Osborne and Wittrock 1985 ). Webster ( 2016 ) employed Mayer's ( 2009 , 2014 ) Cognitive Theory of Multimedia Learning (CTML). CTML proposes a dual channel approach where visual and auditory information is actively processed, organised, and then stored in the brain. This is contingent on neither channel (visual or auditory) becoming overloaded with information. Smith et al. ( 2018 ) used the NLN Jeffries Simulation Theory as their theoretical basis. This theory, most commonly employed in nursing education, is where students learn information as part of a simulated experience (Jeffries et al. 2015 ). For the teaching of vocational skills, Babu et al. ( 2018 ) stated that their approach aligned with situated learning. Situated learning employs a constructivist approach in that students learns professional skills by actively participating in the experience (Huang et al. 2010 ).

Learning outcomes

For I-VR to gain wide-spread acceptance as a reliable pedagogical method, it must be shown to confer a tangible benefit in terms of learning outcomes over less immersive or traditional teaching methods.

Cognitive studies

There were twenty-four included studies that fell into the cognitive domain and aimed to teach specific declarative information or knowledge through the I-VR environment. The current review found that most studies demonstrated benefits in terms of learning outcomes when using I-VR compared to less immersive methods of learning. A smaller number of studies found no significant advantage regardless of the pedagogical method being utilised. The results of these cognitive studies have been broken down by subject area.

Science based cognitive studies

The review found that cognitive learning activities requiring a high degree of visualisation and experiential understanding may be best facilitated using immersive technologies. For instance, both Liou and Chang ( 2018 ) and Maresky et al. ( 2019 ) found that anatomical learning facilitated by complex 3D visualisations of the human body were more conducive to learning in I-VR compared to traditional learning or independent study. Similarly Lamb et al. ( 2018 ) used a virtual environment that allowed for the manipulation and movement of strands of DNA, which produced better learning outcomes in content tests than a lecture or a serious educational game. Greater attention and engagement with the I-VR environment as measured with infrared spectroscopy was one of the possible explanations given for the effectiveness of the technology. In a study by Johnston et al. ( 2018 ), participants volunteered to take part in a cell biology experience either because they were engaged with the subject matter itself, or wanted supplementary instruction. Johnston et al. ( 2018 ) compared the exam scores of those students who volunteered to take part with those who did not. The study found that participants who underwent the I-VR experience scored 5% higher on the related exam question compared to the rest of the assessment. Those who did not undergo the cell biology I-VR experienced scored on average 35% worse on the same question.

The increase in graphical fidelity afforded by I-VR has allowed not only for the creation of complex computer-generated environments, but also the viewing of high resolution 360° video. In one such study, Rupp et al. ( 2019 ) had participants watch a six minute 360° video about the International Space Station with either a HMD which created a sense of immersion and presence, or on a mobile screen. The research found that those participants in the HMD condition scored significantly higher in a learning outcome test (MCQ) than those who watched the video in the non-immersive condition.

Although I-VR has been shown to confer a benefit in science education, there is evidence to suggest that not all learning objectives can be learned equally well. For instance, in task devised by Allcoat and von Mühlenen ( 2018 ), the researchers found that I-VR conferred a benefit over video or textbook learning when questions required remembering , but not ones pertaining to understanding of the material. The authors suggest that unfamiliarity and the novelty of the I-VR environments could have contributed to the lack of an obvious benefit in the latter domain. Another study that examined specific question types to understand I-VR’s effectiveness was undertaken by Kozhevnikov et al. ( 2013 ). In this study, participants learned more conceptual and abstract relative motion concepts using either I-VR or D-VR. The study demonstrated that those in the I-VR condition performed significantly better in the two-dimensional problems than their D-VR counterparts, although there was no significant difference between groups in problems featuring only one spatial dimension.

There were several studies in the domain of science that showed no obvious benefits to using I-VR over traditional pedagogical methods. Two studies (Greenwald et al. 2018 ; Moro et al. 2017 ) compared science learning in I-VR with desktop based VR and 2D videos. Results showed no clear benefit of I-VR based instruction when comparing the difference and significance of learning outcomes between mediums. Similarly, Stepan et al. ( 2017 ) found that I-VR was no more effective than online textbooks for the teaching of neuroanatomy. Interestingly, the same study found no difference in information retention rates when the participants were reassessed 8-weeks later. Madden et al. ( 2018 ) used I-VR, D-VR, and the traditional ball and stick method to teach astronomy principles pertaining to phases of the moon. The study found that I-VR and D-VR produced comparable test score results, with no significant differences in attainment. However, the authors commented on the encouraging finding that despite being a novel technology to most participants, I-VR still facilitated comparable learning outcomes to more traditional methods.

Despite the majority of studies demonstrating that I-VR learning is more effective or at least on par with traditional pedagogical methods, some studies have shown a detrimental effect of I-VR. Makransky et al. ( 2017 ) used a combination of assessment and EEG to find that an I-VR lab simulation produced significantly poorer test scores than a non-immersive alternative. Similarly, during another science experiment, Parong and Mayer ( 2018 ) found that students who used I-VR during a biology lesson scored significantly poorer than those who learned using a PowerPoint. Both of these studies cited Mayer's ( 2009 , 2014 ) Cognitive Theory of Multimedia Learning as a possible explanation for the poorer performance for I-VR. The researchers postulate that the high-fidelity graphics and animations could have significantly increased cognitive load, which would have detracted from the learning task at hand. It was therefore proposed that a less immersive, yet well designed PowerPoint presentation would facilitate better learning outcomes than a graphically rich I-VR experience.

Engineering and architectural based cognitive studies

I-VR was effective in engineering and architectural education as a tool to visualise key concepts within the discipline. For example, Fogarty et al. ( 2017 ) allowed students to volunteer for an I-VR experience who struggled with the comprehension of spatial arrangements in structural engineering. Before the intervention, those students who volunteered to take part scored significantly poorer than their non-intervention counterparts. At post-test, not only did those who underwent the I-VR experience score significantly higher than they did at pre-test, but they also eliminated the significant difference with the non-intervention group. This would suggest that I-VR could serve an important function in supplementing or assisting learning in those students who are struggling to grasp complex problems relating to their discipline. Interestingly, Angulo and de Velasco ( 2013 ) used many of these same spatial and visualisation principles in a more applied setting. Their study split students into groups who were tasked with designing an architectural space (a health clinic waiting room), either with the assistance of an I-VR design tool (experimental group) or a physical model (control group). The study found the space that gained the most positive affect was designed by the I-VR group.

Webster ( 2016 ) created a graphically rich immersive environment which combined active and passive media with elements of gamification and interactivity to teach corrosion concepts to US army personnel. The study found that although both the I-VR environment and a traditional lecture were effective pedagogical methods for teaching these principles, it was the I-VR condition that produced the highest gain in knowledge acquisition.

There was also some evidence to suggest that I-VR interventions could assist in short-term retention of information in engineering related activities. Babu et al. ( 2018 ) found that although participants performed similarly in a mechanical labelling task using either I-VR or a tablet computer immediately post-intervention, the I-VR group had better retention of knowledge when the test was re-administered 1 day later. Furthermore, those participants in the I-VR group were also less likely to wrongly recall information compared to the non-immersive group on the retention test.

Interestingly, Ostrander et al. ( 2018 ) examined cognitive learning outcomes over seven separate manufacturing tasks utilising I-VR in one group, and a traditional class-based environment in the other. The study found that in six out of seven tasks, I-VR was no more effective than a traditional class where students could interact with the instructor or the physical models that they were accustomed to.

Medical based cognitive studies

Although papers featuring surgical simulators did not form part of this review, there were several applications of I-VR in the field of general medical education. Harrington et al. ( 2018 ) had medical students watch a ten-minute 360° video with slides containing surgical information superimposed over it. This was viewed either on a large television screen, or through a Gear VR headset. The study found no significant differences in knowledge retention scores between those who viewed the information through a HMD, or a traditional television screen. Despite not showing a distinct advantage in cognitive learning outcomes, the authors did suggest that the 360° surgical experience may facilitate a better understanding of how teamwork and interaction takes place within an operating theatre. This type of learning may be more difficult to measure using assessment instrumentation such as the MCQ, but nevertheless it could be that the experiential nature of I-VR may facilitate an understanding of interactions and communications. Smith et al. ( 2018 ) used either I-VR or D-VR on a computer to teach students about decontamination protocols. The research found that I-VR was no more effective than D-VR in a MCQ immediately post-intervention, or at 6-weeks follow-up.

Computer science based cognitive studies

Two studies demonstrated a significant advantage in using I-VR to teach computer science information. For instance, Akbulut et al. ( 2018 ) found that students who underwent an I-VR experience that focused on software engineering principles scored 12% higher than students who did not undergo I-VR learning. Interestingly, in a study by Ray and Deb ( 2016 ) that ran over 16 sessions on microcontrollers in computing, the I-VR group performance lagged behind that of the control group who used slideshows for the first four sessions. It was only on session number five that the I-VR group outperformed the control group, and this performance enhancement remained relatively stable in the majority of the remaining 11 sessions. In effect, it took the I-VR group some time to catch up with the control group, but once they did, they tended to outperform them in the remaining lessons. The authors propose that this may have been due to the novelty of the I-VR equipment which participants took time to become comfortable and competent with.

Other cognitive studies

I-VR was also used by Molina-Carmona et al. ( 2018 ) as a means of spatial ability acquisition and visualisation. The study showed that learning outcomes as assessed by a spatial visualisation test were higher among those who undertook the task in an immersive, compared to a non-immersive environment. There was only one study in the field of social science that used I-VR to teach cognitive information. Olmos-Raya et al. ( 2018 ) used either I-VR or a tablet-based system to teach high school students about human geography. The research found that I-VR produced higher learning gains on a MCQ than the tablet-based system. Further, those who used I-VR performed better than the non-immersive group on a knowledge retention quiz when administered 1-week later.

Procedural studies

Three of the four studies that attempted to utilise I-VR as a means of teaching procedural skills showed a distinct advantage over less immersive methods. Bharathi and Tucker ( 2015 ) found that engineering students were faster in assembling a household appliance in a virtual functional analysis activity in I-VR compared to D-VR. Yoganathan et al. ( 2018 ) also found that medical students were more accurate in knot tying practice when using I-VR as a training tool as opposed to a control group who used a standard video. Medical and surgical residents were also studied by Sankaranarayanan et al. ( 2018 ) who used I-VR as a teaching tool for emergency fire response in an operating theatre environment. This study found that 70% of those who utilised the I-VR training were able to perform the correct procedure in the correct order. This was 50% higher than the control group who were exposed to a presentation and reading material only and did not experience I-VR.

One of the studies found no significant advantage to using I-VR as a learning tool. Smith et al. ( 2018 ) split nursing students into an I-VR group, a D-VR group (desktop PC based), or a written instruction group to learn about appropriate protocols for decontamination. The study found that there was no significant difference in performance between the groups as measured by a decontamination checklist, or the time taken to complete the task. Furthermore, reassessment 6 months later showed that I-VR conferred no advantage in procedural knowledge retention (accuracy and speed) compared to less immersive methods.

Affective studies

Only one of the studies attempted to use I-VR as a pedagogical tool to teach applied behavioural and affective skills. Gutiérrez-Maldonado et al. ( 2015 ) used I-VR in the field of diagnostic psychiatry in an attempt to improve interview skills when assessing patients for an eating disorder. Participants were exposed to a series of virtual patient avatars in either the I-VR condition, or a D-VR condition using stereoscopic glasses. Analysis showed that both conditions were equally as effective, and no significant differences were shown in the acquisition of skills between the two groups. Nevertheless, this was a novel study as it traversed the boundaries between traditional cognitive skill acquisition and applied behavioural and affective change.

Discussion and implications

The purpose of this review was to investigate I-VR’s effectiveness as a pedagogical method in education, as well as examining the experimental design and characteristics of the included studies. In particular, the review found that the utilisation of I-VR is typically restricted to a small number of subject areas such as science and engineering. Furthermore, a heavy reliance has been placed on the MCQ and test score measures to assess learning outcomes. In addition, I-VR interventions were typically short and isolated, and were not complemented with additional or supplementary learning material. Despite this, most studies did find a significant advantage of using I-VR over less immersive methods of learning. This was the case particularly when the subject area was highly abstract or conceptual, or focused on procedural skills or tasks.

Is the utilisation of I-VR within education restrictive?

The findings of the review suggest a relatively homogenous application of I-VR in terms of both the subject areas represented, as well as the learning domain being taught. Almost 70% of the studies were from the field of science or engineering, with other subjects being marginally represented. It is worth noting, however, that although medical disciplines made up a small proportion of the studies included (14%), this was because most medical applications of I-VR feature surgical simulators and therefore were not part of the current review’s inclusion criteria. Most studies utilised I-VR as a way of teaching cognitive skills, with only a handful examining the procedural or affective applications.

The findings of the review raise several issues when trying to assess the general effectiveness of I-VR in education. Similar to the findings of others (e.g. Jensen and Konradsen 2018 ; Radianti et al. 2020 ), the arts, humanities, and social sciences were underrepresented in in the current review. This makes generalisable conclusions as to the cognitive benefit of the technology in these subjects challenging. One major reason for this under representation may be the lack of I-VR learning content, experiences, and teaching tools. Jensen and Konradsen ( 2018 ) highlighted that instructors are restricted to the material published and produced by VR designers, and this may not necessarily meet the individual needs of the teacher, or the learning outcome trying to be achieved. The skillset needed to produce and create wholly virtual environments that can be rendered and displayed in a HMD is still demanding, despite the release of affordable VR creation suites. Therefore, the bespoke I-VR experiences required to teach social science lessons (or indeed any subject) is completely dependent on an appropriate I-VR tool already existing or having the technical proficiency to create one. A potential solution to the lack of bespoke material could be the examination of the pedagogical effectiveness of HMD 360° video in the classroom, as opposed to computer-generated environments. This content is comparatively easier to create using appropriate video equipment and can be tailored to the individual needs of the instructor or student group. Widespread research that examines the potential of I-VR in a multitude of diverse disciplines and learning domains will continue to be constrained by the availability of the requisite material. That is until such a time where bespoke and individually tailored I-VR experiences become more accessible.

Implications of outcome measures and assessment instrumentation

One of the most striking characteristics of the assessment instrumentation used in the studies was the reliance on the MCQ to assess learning outcomes. Although there have been many debates on the respective advantages and disadvantages of utilising the MCQ, it has generally been considered that it is most appropriate for testing large amounts of surface knowledge over the course of an entire module or syllabus (Excell 2000 ). As O’Dwyer ( 2012 ) points out, the assessment instrumentation encourages comprehensive learning of the entirety of the taught material, as opposed to just specific components. However, since most of the studies featured single interventions of between 6 and 30 mins, doubts are cast on whether MCQs are the most appropriate way to assess learning. Since the MCQ was most commonly administered immediately after the I-VR experience, much of the information may still be stored in short-term memory, and this may not give an accurate reflection of more comprehensive learning or long-term retention.

A second disadvantage associated with the heavy reliance on the MCQ is the limited breadth of knowledge that can be assessed. In Jensen and Konradsen’s ( 2018 ) systematic review, the researchers found that none of the cognitive studies went beyond teaching lower level cognitive skills as defined by Bloom’s taxonomy (Bloom et al. 1956 ). Similar results were found in the current review, with most studies requiring only a knowledge of previously learned material to successfully achieve the desired learning goal. Previous research on pedagogical assessment material (e.g. Ozuru et al. 2013 ) has suggested that the MCQ cannot assess higher levels of cognitive understanding or conceptual knowledge. Therefore, it may not only be the nature of the I-VR experience itself that restricts the learning of higher level cognitive skills, but also the restrictive nature of the assessment instrumentation that may impede an appropriate demonstration of learning outcomes. The utilisation of short or long form answers could be able to provide a more appropriate measure of the depth of learning achieved, giving the student an opportunity to demonstrate their conceptual knowledge of a given subject. Furthermore, I-VR research could benefit by expanding the very definition of what constitutes a learning outcome. This could be achieved by not relying exclusively on test score comparisons, but rather examine how I-VR could be used to foster deeper conceptual understanding through experiential learning and subsequent classroom discussions with peers or instructors.

Implication of intervention characteristics for learning outcomes

The current review examined how I-VR is being utilised in experimental and applied settings, and the implications this has for assessing its pedagogical suitability. In most studies, the participant took part in a single I-VR experience that was also short in duration. This presents several key challenges. Most importantly, the novelty of the I-VR technology itself may have impeded the learning experience of the user, especially if they had never used the technology before or were unfamiliar with it. This seemed to be demonstrated by Ray and Deb ( 2016 ) who found that in the initial sessions of I-VR learning, performance was on average poorer than those who underwent traditional teaching methods. It was only after the participants began to become familiar with the technology (on session number five) that learning surpassed the control group. Similarly, studies that allowed for extended exposure to I-VR (e.g. Akbulut et al. 2018 ; Alhalabi 2016 ; Molina-Carmona et al. 2018 ), either through free navigation, repeated sessions, or scheduled class time, tended to show an advantage of using I-VR over non-immersive or traditional methods. It is therefore important to address the potentially negative influence that I-VR’s novelty as a learning tool may have, especially when outcomes are directly compared to another medium or method. Scepticism for media comparison studies was highlighted in the 1980s by Clark ( 1983 ), and then later re-addressed by Parong and Mayer ( 2018 ). As Parong and Mayer ( 2018 ) put it, the side-by-side comparison of two learning methods is an “apples-to-oranges type of comparison” (p. 788). This “apples-to-oranges” comparison is made starker when considering that I-VR is an unfamiliar technology to most in an educational capacity, and its pedagogical outcomes are being directly compared with familiar methods such as textbooks or lectures. It is important to consider that the novelty of HMDs and I-VR may hinder learning outcomes and classroom application, and it is therefore prudent to ensure that the degree of familiarity with I-VR technology is factored into any direct comparison with other methods. In practice, this could mean that participants require extended familiarisation trials or free navigation before the start of experimental studies as a means of mitigating against potential problems caused by technological novelty.

In addition to the short intervention and exposure time, most studies did not complement I-VR with an additional method of teaching or self-learning. The limited number of studies that did tended to utilise web-based textbooks or modules, as well as lectures and scheduled class time. Encouragingly, those studies that combined or supplemented traditional class-based learning with I-VR (e.g. Akbulut et al. 2018 ; Fogarty et al. 2017 ; Johnston et al. 2018 ; Sankaranarayanan et al. 2018 ; Yoganathan et al. 2018 ) tended to show a learning advantage. This suggests that I-VR may be best employed as form of blended or multi-modal learning to supplement and complement class-based instruction (Garrison and Kanuka 2004 ). An area for investigation would be to examine I-VR’s application longitudinally in a natural classroom environment. The current review contained only a limited number of studies that employed this approach, however, by implementing and studying how I-VR can be adopted and integrated into a module or syllabus, a clearer picture of its capabilities can emerge.

Learning theories ultimately provide a theoretical framework and foundation as how best to design educational interventions (Pritchard 2017 ; Schunk 2011 ). However, the review found that few papers explicitly state that any predetermined learning theory was used to advise the characteristics or methods of the study. Similar findings were reported in a systematic review by Radianti et al. ( 2020 ) examining I-VR use in higher education exclusively. Radianti et al.’s ( 2020 ) review found that in around 70% of the 38 studies included, no learning theory was mentioned as forming the foundation of the VR activity. Several studies have shown that educators regard clear pre-defined intervention characteristics and objectives as essential components of I-VR teaching (Fransson et al. 2020 ; Lee and Shea 2020 ). It is therefore essential that future experimental and applied research is based on a sound theoretical basis that can advise how the technology can be appropriately utilised and assessed.

Learning outcomes in I-VR

The current review examined learning outcomes across three domains: cognitive, procedural, and affective. By far the most popular domain was the teaching of cognitive skills and knowledge which made up 83% of the studies in the current review. Around half of those demonstrated a positive effect on learning when using I-VR over less immersive pedagogical methods. Most of the remaining studies showed no significant effect either way, with only a small number of papers exhibiting detrimental results. Researchers have suggested that the increased levels of immersive content that stimulate multisensory engagement can ultimately lead to more effective learning outcomes (Webster 2016 ). When this is implemented in cognitive learning activities that require a high degree of spatial understanding and visualisation (e.g. Maresky et al. 2019 ), I-VR can allow the user to gain insights that are difficult to reproduce in reality. This review has already identified scientific subjects such as biology and physics as promising avenues for educational I-VR implementation. However, other scientific disciplines that require abstract or conceptual understanding (e.g. chemistry, mathematics) could also benefit from the visualisation afforded by I-VR.

Studies that utilised I-VR for the teaching of procedural skills and knowledge produced encouraging results, with three of the four studies finding a significantly positive increase in learning (Bharathi and Tucker 2015 ; Sankaranarayanan et al. 2018 ; Yoganathan et al. 2018 ). Interestingly, two of the studies featured a transfer component by having the user first practice the procedure in I-VR, and then use this form of experiential learning to complete a task in the real world. Yoganathan et al. ( 2018 ) had students practice how to tie a surgical knot in I-VR and then complete this task for real in-front of an expert. Sankaranarayanan et al. ( 2018 ) had medical students learn how to deal with an operating theatre fire by first practicing the procedure in I-VR, and then applying this knowledge to a mock emergency in a real operating room. Both studies found a positive effect of using I-VR as the training method by demonstrating improved results when performed in a real environment. These are encouraging findings for I-VR’s effectiveness in psychomotor and procedural education, as there has been a degree of scepticism over whether I-VR simply produces a “getting good at the game” effect. For instance, Jensen and Konradsen ( 2018 ) point out that the honing of procedural skills within I-VR may simply lead to the participant becoming proficient when performing the task virtually, and this may not necessarily transfer to the real world. The current review has identified that the two procedural studies that implemented a transfer task did indeed demonstrate a significant benefit to using I-VR as an initial education method. This demonstrates that virtual training can be a successful precursor to implementation in the real world. This suggests that I-VR could be useful in educating students in dangerous vocational subjects such as electrical engineering without risk to themselves or others. However, this view is based on a small number of studies, and it is therefore important that future procedural tasks utilise a transfer activity to understand the potential scope and parameters surrounding I-VR training and real-world application.

Only one of the studies had a firm focus on the training of affective skills, namely by using I-VR as a way of teaching diagnostic interview techniques in a psychiatric setting (Gutiérrez-Maldonado et al. 2015 ). Although this study found no clear advantage to using I-VR, other research out with the domain of education has demonstrated promising results in utilising the technology for affective and behavioural change. This included applying the technology successfully in areas such as exposure therapy, anxiety disorder treatment, and empathy elicitation (Botella et al. 2017 ; Maples-Keller et al. 2017a , b ; Schutte and Stilinović 2017 ). As a result of the strong non-educational body of literature suggesting I-VR can facilitate affective and behavioural change, future research should examine how this can be applied in an educational context, and then transferred to real-world scenarios. For instance, in their psychiatric interview experience, Gutiérrez-Maldonado et al. ( 2015 ) had users interact solely with virtual avatars, and did not have the participants demonstrate their learning with a real actor or patient. Therefore, just like with procedural skill acquisition, affective I-VR experiences should seek to understand how virtual learning can then be applied to real situations.

Implications and future practice

The current review has been able to identify a body of experimental and applied research that show the potential benefits of using I-VR in education. It has already been noted that I-VR has traditionally been used to teach low level or fundamental skills and knowledge, and has not necessarily been used to facilitate what Bloom et al. ( 1956 ) would consider higher level learning. This would include analysing and evaluating experience. By expanding the definition of learning outcomes to encompass potential benefits such as an increased depth of understanding or the ability to identify complex themes, pedagogical practice can take advantage of the inherent strength of the medium. These should be comprehensively analysed to investigate learning outcomes that go beyond simple test scores.

The review has also been able to identify areas for improvement in future studies, which would address confounding variables and expand the scope of research. Firstly, as Allcoat and von Mühlenen ( 2018 ) suggest, the novelty of I-VR could hamper learning outcomes due to unfamiliarity with the technology. Therefore, it is important to factor in an extended familiarisation or free navigation period that would help alleviate this concern. Additionally, follow-up qualitative analysis such as interviews or focus groups could help explore the phenomenology or direct experience of using I-VR, and highlight concerns relating to unfamiliarity or technological anxiety. The biggest concern relating to the assessment instrumentation was the over reliance on the MCQ (62% of studies used it as the sold method of assessment). Although this method is deemed appropriate for assessing large amounts of surface knowledge, it may not reveal more nuanced forms of learning that extend beyond mere recall of information. Therefore, long form essay questions, oral examinations, or group discussions could be used to facilitate students’ ability to present their in-depth understanding and applied knowledge. Future research must base the nature of these interventions on a sound theoretical framework. This would assist in identifying specific learning objectives and methods of assessments. An explicit theoretical approach was commonly lacking in the included studies.

I-VR has already been demonstrated to be an effective tool in non-pedagogical behaviour change, such as treating phobias, mental health conditions, or as a tool for rehabilitation (Botella et al. 2017 ; Maples-Keller et al. 2018; Ravi et al. 2017 ). Research should therefore concentrate on I-VR’s potential as an acquisition tool for affective skills. There is already a strong body of evidence suggesting I-VR experiences can elicit high levels of empathetic response and perspective taking, and this should be explored within an educational context (Herrera et al. 2018 ; Shin 2018 ). For example, Dyer et al. ( 2018 ) used I-VR to allow health care students to take the perspective of an older patient with age-related medical conditions, which led to increased empathy. Future studies should investigate whether this perspective taking ability can lead to higher domains of learning, such as evaluating one’s actions, applying problem solving skills, or creating new solutions as a direct result of the insights they received from I-VR. This will require researchers and instructors to carefully consider their tools for evaluation and assessment, perhaps incorporating mixed-methods to give a more holistic overview of learning achieved.

Conclusions

The current review found that I-VR conferred a learning benefit in around half of cognitive studies, especially where highly complex or conceptual problems required spatial understanding and visualisation. Although many studies found no significant benefit of using I-VR over less immersive technology, only a small number resulted in detrimental effects on learning outcomes. However, the homogenous nature of assessment instrumentation, such as an over reliance on the MCQ may have stifled the ability for participants to demonstrate learning outcomes beyond low level cognitive knowledge. Short exposure times and isolated interventions could also pose a problem as the novel nature of the technology could negatively impact the amount of learning able to be imbibed. Encouragingly, most procedural tasks did show a benefit to utilising I-VR, and furthermore, there was evidence that virtual skill acquisition could be transferred successfully to real world problems and scenarios. The ability to repeatedly practice a procedure in a safe environment whilst expending little resources could be one of the most advantageous and intrinsic benefits of I-VR technology. Although affective behavioural change has been widely studied in non-educational applications of I-VR, the domain was underrepresented in the current review, and is an important area for future investigation.

Over the coming years, technological advancement, an increase in creative content, and the possibilities for instructors to create bespoke I-VR experiences will all contribute to I-VR’s potential as a teaching tool. It is essential therefore that the implementation of such technology is based on sound theoretical and experimental evidence in order to ensure that the I-VR is utilised correctly, and to its full potential.

Akbulut, A., Catal, C., & Yıldız, B. (2018). On the effectiveness of virtual reality in the education of software engineering. Computer Applications in Engineering Education, 26 , 918–927.

Google Scholar  

Alaraj, A., Lemole, M., Finkle, J., Yudkowsky, R., Wallace, A., Luciano, C., … Charbel, F. (2011). Virtual reality training in neurosurgery: Review of current status and future applications. Surgical Neurology International , 2 , 52

Alhalabi, W. S. (2016). Virtual reality systems enhance students’ achievements in engineering education. Behaviour & Information Technology, 35 , 919–925.

Allcoat, D., & von Mühlenen, A. (2018). Learning in virtual reality: Effects on performance, emotion and engagement. Research in Learning Technology, 26 , 2140.

Angulo, A., & de Velasco, G. V. (2013). Immersive simulation of architectural spatial experiences. Blucher Design Proceedings, 1 , 495–499.

Babu, S., Krishna, S., Unnikrishnan, R., & Bhavani, R. (2018). Virtual reality learning environments for vocational education: A comparison study with conventional instructional media on knowledge retention. In 2018 IEEE 18th International Conference on Advanced Learning Technologies (ICALT) (pp. 385–389). IEEE.

Bailenson, J. N., Markowitz, D. M., Pea, R. D., Perone, B. P., & Laha, R. (2018). Immersive virtual reality field trips facilitate learning about climate change. Frontiers in Psychology, 9 , 2364.

Bharathi, A. K. B. G., & Tucker, C. S. (2015). Investigating the impact of interactive immersive virtual reality environments in enhancing task performance in online engineering design activities. In ASME 2015 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference . ASME.

Bloom, B. S., Englehard, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. (1956). Taxonomy of educational objectives, handbook I: The cognitive domain . New York, NY: Longmans.

Botella, C., Fernández-Álvarez, J., Guillén, V., García-Palacios, A., & Baños, R. (2017). Recent progress in virtual reality exposure therapy for phobias: A systematic review. Current Psychiatry Reports, 19 , 42.

Brown, A., & Green, T. (2016). Virtual reality: Low-cost tools and resources for the classroom. TechTrends, 60 , 517–519.

Çalişkan, O. (2011). Virtual field trips in education of earth and environmental sciences. Procedia-Social and Behavioral Sciences, 15 , 3239–3243.

Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53 , 445–459.

Dickey, M. D. (2005). Three-dimensional virtual worlds and distance learning: Two case studies of Active Worlds as a medium for distance education. British Journal of Educational Technology, 36 , 439–451.

Dyer, E., Swartzlander, B. J., & Gugliucci, M. R. (2018). Using virtual reality in medical education to teach empathy. Journal of the Medical Library Association, 106 , 498–500.

Ellinger, R. S., & Frankland, P. (1976). Computer-assisted and lecture instruction: A comparative experiment. Journal of Geography, 75 , 109–120.

Excell, P. S. (2000). Experiments in the use of multiple-choice examinations for electromagnetics-related topics. IEEE Transactions on Education, 43 , 250–256.

Fogarty, J., McCormick, J., & El-Tawil, S. (2017). Improving student understanding of complex spatial arrangements with virtual reality. Journal of Professional Issues in Engineering Education and Practice, 144 , 04017013.

Fransson, G., Holmberg, J., & Westelius, C. (2020). The challenges of using head mounted virtual reality in K-12 schools from a teacher perspective. Education and Information Technologies., 2 , 20–22.

Freina, L., & Ott, M. (2015). A literature review on immersive virtual reality in education: State of the art and perspectives. In The 11th International Scientific Conference ELearning and Software for Education (pp. 133–141).

Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. Internet and Higher Education, 7 , 95–105.

Gee, J. P. (2004). What video games have to teach us about learning and literacy. Education Training, 46 , 20.

Greenwald, S. W., Corning, W., Funk, M., & Maes, P. (2018). Comparing learning in virtual reality with learning on a 2D screen using electrostatics activities. Journal of Universal Computer Science, 24 , 220–245.

Gutiérrez-Maldonado, J., Ferrer-García, M., Pla-Sanjuanelo, J., Andrés-Pueyo, A., & Talarn-Caparrós, A. (2015). Virtual reality to train diagnostic skills in eating disorders. Comparison of two low cost systems. Studies in Health Technology and Informatics, 219 , 75–81.

Häfner, P., Dücker, J., Schlatt, C., & Ovtcharova, J. (2018). Decision support method for using virtual reality in education based on a cost-benefit-analysis. In 4th International Conference of the Virtual and Augmented Reality in Education, VARE 2018 , 103–112.

Harrington, C. M., Kavanagh, D. O., Wright Ballester, G., Wright Ballester, A., Dicker, P., Traynor, O., … Tierney, S. (2018). 360° operative videos: A randomised cross-over study evaluating attentiveness and information retention. Journal of Surgical Education , 75 , 993–1000

Herrera, F., Bailenson, J., Weisz, E., Ogle, E., & Zak, J. (2018). Building long-term empathy: A large-scale comparison of traditional and virtual reality perspective-taking. PLoS ONE, 13 , e0204494.

Hodgson, E., Bachmann, E. R., Vincent, D., Zmuda, M., Waller, D., & Calusdian, J. (2015). WeaVR: A self-contained and wearable immersive virtual environment simulation system. Behavior Research Methods., 47 (1), 296–307.

Hodgson, P., Lee, V. W. Y., Chan, J. C. S., Fong, A., Tang, C. S. Y., Chan, L., et al. (2019). Immersive virtual reality (IVR) in higher education: Development and implementation. In M. Dieck & T. Jung (Eds.), Augmented reality and virtual reality (pp. 161–173). New York: Springer.

Howard-Jones, P., Ott, M., van Leeuwen, T., & De Smedt, B. (2015). The potential relevance of cognitive neuroscience for the development and use of technology-enhanced learning. Learning, Media and Technology, 40 , 131–151.

Huang, H. M., Rauch, U., & Liaw, S. S. (2010). Investigating learners’ attitudes toward virtual reality learning environments: Based on a constructivist approach. Computers and Education, 55 , 1171–1182.

Jeffries, P. R., Rodgers, B., & Adamson, K. (2015). NLN Jeffries simulation theory: Brief narrative description. Nursing Education Perspectives, 36 , 292–293.

Jensen, L., & Konradsen, F. (2018). A review of the use of virtual reality head-mounted displays in education and training. Education and Information Technologies, 23 , 1515–1529.

Johnston, A. P. R., Rae, J., Ariotti, N., Bailey, B., Lilja, A., Webb, R. I., … Parton, R. G. (2018). Journey to the centre of the cell: Virtual reality immersion into scientific data. Traffic , 19 , 105–110

Kozhevnikov, M., Gurlitt, J., & Kozhevnikov, M. (2013). Learning relative motion concepts in immersive and non-immersive virtual environments. Journal of Science Education and Technology, 22 , 952–962.

Lamb, R., Antonenko, P., Etopio, E., & Seccia, A. (2018). Comparison of virtual reality and hands on activities in science education via functional near infrared spectroscopy. Computers and Education, 124 , 14–26.

Larsen, C. R., Oestergaard, J., Ottesen, B. S., & Soerensen, J. L. (2012). The efficacy of virtual reality simulation training in laparoscopy: A systematic review of randomized trials. Acta Obstetricia et Gynecologica Scandinavica, 91 , 1015–1028.

Lee, C., & Shea, M. (2020). Exploring the use of virtual reality by pre-service elementary teachers for teaching science in the elementary classroom. Journal of Research on Technology in Education, 52 , 163–177.

Lee, E. A. L., Wong, K. W., & Fung, C. C. (2010). How does desktop virtual reality enhance learning outcomes? A structural equation modeling approach. Computers & Education, 55 , 1424–1442.

Lee, J. (1999). Effectiveness of computer-based instructional simulation: A meta analysis. International Journal of Instructional Media, 26 , 71–86.

Li, L., Yu, F., Shi, D., Shi, J., Tian, Z., Yang, J., … Jiang, Q. (2017). Application of virtual reality technology in clinical medicine. American Journal of Translational Research , 9 , 3867–3880

Liou, W., & Chang, C. (2018). Virtual reality classroom applied to science education. In 2018 23rd International Scientific-Professional Conference on Information Technology (IT) (pp. 1–4).

Lyne, D. S. (2013). Development of virtual reality applications for the construction industry using the Oculus Rift head mounted display. In Proceedings of the 13th International Conference on Construction Applications of Virtual Reality (pp. 30–31).

Madden, J. H., Won, A. S., Schuldt, J. P., Kim, B., Pandita, S., Sun, Y., … Holmes, N. G. (2018). Virtual reality as a teaching tool for moon phases and beyond. In 2018 Physics Education Research Conference .

Makransky, G., Terkildsen, T. S., & Mayer, R. E. (2017). Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learning and Instruction, 60 , 225–236.

Maples-Keller, J. L., Bunnell, B. E., Kim, S.-J., & Barbara, R. (2017a). The use of virtual reality technology in the treatment of anxiety and other psychiatric disorders. Harvard Review of Psychiatry, 25 , 103–113.

Maples-Keller, J. L., Yasinski, C., Manjin, N., & Rothbaum, B. O. (2017b). Virtual reality-enhanced extinction of phobias and post-traumatic stress. Neurotherapeutics, 14 , 554–563.

Maresky, H. S., Oikonomou, A., Ali, I., Ditkofsky, N., Pakkal, M., & Ballyk, B. (2019). Virtual reality and cardiac anatomy: Exploring immersive three-dimensional cardiac imaging, a pilot study in undergraduate medical anatomy education. Clinical Anatomy, 32 , 238–243.

Mayer, R. E. (2009). Multimedia learning (2nd ed.). Cambridge: Cambridge University Press.

Mayer, R. E. (2014). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed.). Cambridge: Cambridge University Press.

Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers and Education, 70 , 29–40.

Molina-Carmona, R., Pertegal-Felices, M., Jimeno-Morenilla, A., & Mora-Mora, H. (2018). Virtual reality learning activities for multimedia students to enhance spatial ability. Sustainability, 10 , 1074.

Moro, C., Štromberga, Z., Raikos, A., & Stirling, A. (2017). The effectiveness of virtual and augmented reality in health sciences and medical anatomy. Anatomical Sciences Education, 10 , 549–559.

Murcia-López, M., & Steed, A. (2016). The effect of environmental features, self-avatar, and immersion on object location memory in virtual environments. Frontiers in ICT, 3 , 1–10.

O’Dwyer, A. (2012). A teaching practice review of the use of multiple-choice questions for formative and summative assessment of student work on advanced undergraduate and postgraduate modules in engineering. The All Ireland Journal of Teaching and Learning in Higher Education, 4 , 1–12.

Olmos, E., Cavalcanti, J. F., Soler, J.-L., Contero, M., & Alcañiz, M. (2018). Mobile virtual reality: A promising technology to change the way we learn and teach. In S. M. Bell (Ed.), Mobile and ubiquitous learning: An international handbook (pp. 95–106). Singapore: Springer.

Olmos-Raya, E., Ferreira-Cavalcanti, J., Contero, M., Castellanos, M. C., Giglioli, I. A. C., & Alcañiz, M. (2018). Mobile virtual reality as an educational platform: A pilot study on the impact of immersion and positive emotion induction in the learning process. Eurasia Journal of Mathematics, Science and Technology Education, 14 , 2045–2057.

Osborne, R., & Wittrock, M. (1985). The generative learning model and its implications for science education. Studies in Science Education., 12 , 59–87.

Ostrander, J. K., Tucker, C. S., Simpson, T. W., & Meisel, N. A. (2018). Evaluating the effectiveness of virtual reality as an interactive educational resource for additive manufacturing. In Volume 3: 20th International Conference on Advanced Vehicle Technologies; 15th International Conference on Design Education , V003T04A018. ASME.

Ozuru, Y., Briner, S., Kurby, C. A., & McNamara, D. S. (2013). Comparing comprehension measured by multiple-choice and open-ended questions. Canadian Journal of Experimental Psychology, 67 , 215–227.

Parong, J., & Mayer, R. E. (2018). Learning science in immersive virtual reality. Journal of Educational Psychology, 110 , 785–797.

Pritchard, A. (2017). Ways of learning: Learning theories for the classroom (4th ed.). London: Routledge.

Radianti, J., Majchrzak, T. A., Fromm, J., & Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers and Education, 147 , 103778.

Ravi, D. K., Kumar, N., & Singhi, P. (2017). Effectiveness of virtual reality rehabilitation for children and adolescents with cerebral palsy: An updated evidence-based systematic review. Physiotherapy, 103 , 245–258.

Ray, A. B., & Deb, S. (2016). Smartphone based virtual reality systems in classroom teaching - A study on the effects of learning outcome. In 2016 IEEE Eighth International Conference on Technology for Education (T4E) (pp. 68–71).

Reed, D. A., Cook, D. A., Beckman, T. J., Levine, R. B., Kern, D. E., & Wright, S. M. (2007). Association between funding and quality of published medical education research. JAMA, 298 , 1002.

Rios, J., & Wells, C. (2014). Validity evidence based on internal structure. Psicothema, 26 , 108–116.

Rupp, M. A., Odette, K. L., Kozachuk, J., Michaelis, J. R., Smither, J. A., & McConnell, D. S. (2019). Investigating learning outcomes and subjective experiences in 360-degree videos. Computers & Education, 128 , 256–268.

Sankaranarayanan, G., Wooley, L., Hogg, D., Dorozhkin, D., Olasky, J., Chauhan, S., … Jones, D. B. (2018). Immersive virtual reality-based training improves response in a simulated operating room fire scenario. Surgical Endoscopy and Other Interventional Techniques , 32 , 3439–3449

Schunk, D. H. (2011). Learning theories: An educational perspective (6th ed.). London: Pearson.

Schutte, N. S., & Stilinović, E. J. (2017). Facilitating empathy through virtual reality. Motivation and Emotion, 41 , 708–712.

Sherman, W. R., & Craig, A. B. (2003). Understanding virtual reality: Interface, application, and design. In Understanding Virtual Reality: Interface, Application, and Design .

Shin, D. (2018). Empathy and embodied experience in virtual environment: To what extent can virtual reality stimulate empathy and embodied experience? Computers in Human Behavior., 78 , 64–73.

Slater, M. (2018). Immersion and the illusion of presence in virtual reality. British Journal of Psychology, 109 , 431–433.

Smith, S. J., Farra, S. L., Ulrich, D. L., Hodgson, E., Nicely, S., & Mickle, A. (2018). Effectiveness of two varying levels of virtual reality simulation. Nursing Education Perspectives, 39 , 10–15.

Stepan, K., Zeiger, J., Hanchuk, S., Del Signore, A., Shrivastava, R., Govindaraj, S., et al. (2017). Immersive virtual reality as a teaching tool for neuroanatomy. International Forum of Allergy and Rhinology, 7 , 1006–1013.

UKAuthority. (2019). VR and AR attract education sector interest. Retrieved December 16, 2019, from https://www.ukauthority.com/articles/vr-and-ar-attract-education-sector-interest/

Webster, R. (2016). Declarative knowledge acquisition in immersive virtual learning environments. Interactive Learning Environments, 24 , 1319–1333.

Yoganathan, S., Finch, D. A., Parkin, E., & Pollard, J. (2018). 360° virtual reality video for the acquisition of knot tying skills: A randomised controlled trial. International Journal of Surgery, 54 , 24–27.

Download references

Author information

Authors and affiliations.

Division of Psychology, School of Education & Social Sciences, University of the West of Scotland, Paisley, UK

D. Hamilton, J. McKechnie, E. Edgerton & C. Wilson

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to D. Hamilton .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

See Table 2 .

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Hamilton, D., McKechnie, J., Edgerton, E. et al. Immersive virtual reality as a pedagogical tool in education: a systematic literature review of quantitative learning outcomes and experimental design. J. Comput. Educ. 8 , 1–32 (2021). https://doi.org/10.1007/s40692-020-00169-2

Download citation

Received : 21 October 2019

Revised : 16 June 2020

Accepted : 02 July 2020

Published : 11 July 2020

Issue Date : March 2021

DOI : https://doi.org/10.1007/s40692-020-00169-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Virtual reality
• Head mounted displays
• Simulations
• Find a journal
• Publish with us
• Track your research

Study and Analysis of Virtual Reality and its Impact on the Current Era

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.