INTRODUCTION
As nursing is a practicing profession, clinical experience is a central pedagogy in the learning of nursing students (Wagner et al., 2009). Recently, the provision of clinical experience has posed challenges linked to the changing dynamics within health care systems and the workforce; such as increased patient acuity, heightened workload for nurses, and shortage of nursing workforce (Lavoie et al., 2018). Additionally, the escalation in COVID-19 cases during the pandemic period resulted in the reduction of clinical placement availability for nursing students in Aotearoa New Zealand. This situation resonated with the challenges faced by nursing academics and students globally. A meeting held by nursing academics and leaders in Washington (USA) in 2020 proposed virtual reality simulation learning as a substitute for clinical placement hours until students could be safely returned to clinical placements (Wild et al., 2020). The challenge of placing students in clinical placement areas during COVID-19 was a watershed moment for nursing academics to explore and develop teaching methodologies that effectively engage students in practical learning experiences.
Although simulation pedagogy has always been used in nursing education, virtual reality simulation is a relatively new option for more interactive and realistic experiences for nursing students (Fealy et al., 2019). The world of virtual reality simulation has been expanding with the inclusion of the latest modalities, such as HoloLens, a mixed-reality wearable tool, used in this study (Kim et al., 2021). However, research-based evidence leveraging these latest modalities in nursing education warrants a greater understanding of the experiences of nursing academics and students to seek answers as to what constitutes the best teaching-learning practices.
The spectrum of virtual reality simulation
Recent advancements in computer technology have expanded the world of simulation by providing additional tools with a high grade of realism and immersion (Fealy et al., 2019). The degree to which the learner’s senses can be engaged in the virtual environment is known as the ‘level of immersion’. Based on the ‘level of immersion’, the virtual reality simulation can be classified on a spectrum ranging from non-immersive to fully immersive virtual reality (Hauze et al., 2019). Further advancement in virtual reality technology has added newer versions of augmented reality and mixed-reality simulation.
Non-immersive virtual reality is an older version involving depicting a virtual scenario on a computer screen (De Gagne et al., 2013; Foronda et al., 2013). The learner interacts with the virtual simulation via keyboard or mouse and controls the movements of virtual ‘avatars’ while remaining fully aware of the natural surroundings (Foronda et al., 2013). Conversely, ‘immersive virtual reality’ refers to an advanced version of virtual reality technology with a higher level of immersion involving the visualisation of an interactive 3D virtual environment through a headset or head-mounted display (Fealy et al., 2019). The interactive virtual environment also responds to multiple sensory inputs and feedback from the user. The user is completely immersed in the virtual environment, with the natural environment obscured from the user’s view (Hauze et al., 2019). A further development is augmented reality technology where 3D graphic content is overlayed onto a natural environment, so the user can see the enriched digital content as well as the real environment (Brigham, 2017; Carlson & Gagnon, 2016). This can be viewed using mobile phone screens.
Mixed reality is the latest technology hybrid of immersive and augmented reality technologies, allowing the user to view the real world while interacting with a 3D virtual environment with enhanced depth and perspective (Hauze et al., 2019; Ramachandiran et al., 2019). Mixed reality employs head-mounted display or headsets to visualise the virtual scenario within the natural environment (Brigham, 2017). This implies the learner can interact with the virtual patient while also seeing the real surroundings, other students, and educators. The terms ‘mixed reality’ and ‘augmented reality’ have been used interchangeably in the literature, but the meaning is not the same. There is a need to define and describe the technology being researched or studied using precise nomenclature, including the details of the simulation procedure.
Theoretical framework underpinning the simulation tool
The situated cognition theory postulates that learning does not take place in silos but is embedded in the social context or community of practice within which it happens (Paige & Daley, 2009). The learning is generated through interaction with other learners, the learning environment, and associated activities (Paige & Daley, 2009). Assumptions cannot be made that the knowledge learned from one situation is transferable to a new environment or clinical experience, unless the learner has gained the knowledge in a situation similar to the context in which it is meant to be used in the future (Onda, 2012). The mixed reality device used in this study, presents 3D holograms representing patients with different clinical manifestations, including fluctuations in vital signs to resemble the real clinical situation. The authentic visualisation allows students to construct new meaningful learning in the context of simulation experience (Paige & Daley, 2009). Furthermore, the nursing students engage in collaborative learning in interaction with other learners, nursing educators, and the virtual environment. The immersive and interactive simulation connects theoretical knowledge with practical application, easing the transition to professional practice and fostering social integration (Paige & Daley, 2009).
Aim
This study sought to explore the perceptions and experiences of nursing faculty and students when using a mixed reality simulation tool as a wearable headset and in remote mobile device mode. Additionally, the study sought to provide recommendations for the best teaching and learning practices to leverage mixed reality simulation learning in the future.
METHODOLOGY AND METHODS
Study design
This study used a constructivist research paradigm. Constructivism was chosen in light of the aim of the study to understand the subjective perceptions and experiences as uniquely constructed by participants (nursing lecturers and students) (Creswell & Creswell, 2017). By applying the lens of constructivism to this research project, nursing students and lecturers were anticipated to have varied and unique simulation experiences with the mixed reality device. There is no single reality but multiple meaningful realities for participants, constructed in their unique individual and social contexts (Creswell & Creswell, 2017). Aligned to the chosen research paradigm, the qualitative methodological approach was employed to understand the multiple realities and subjective interpretations associated with the unique constructs of human experiences (Creswell & Creswell, 2017). Furthermore, the descriptive qualitative design used for this study allowed the interpretation of perceptions and experiences by staying closer to the participants’ words without drawing abstract concepts from data (Bradshaw et al., 2017; Sandelowski, 2010).
The mixed-reality simulation tool
HoloLens (Microsoft), a mixed reality wearable holographic device was used for this study. The device is capable of projecting a three-dimensional graphic overlay (called a Hologram) in the users’ visual space through the headset (Leonard & Fitzgerald, 2018). The wearable headset is fitted with stereoscopic glasses and infrared sensors that map physical ambiance and detect the user’s hand and finger movements (called air tapping) (Karthika et al., 2017; Tepper et al., 2017). For example, the user can see a virtual hologram of a patient experiencing an anaphylactic reaction and their vital signs screen through the headset. The user can walk around the virtual hologram to make observation from different angles while seeing and hearing the patient passing through the different stages of an anaphylactic reaction (Frost et al., 2020). This application of the technology (HoloPatient) depicts patient scenarios with various clinical conditions, such as myocardial infarction and burns injuries, to facilitate learning of clinical assessment and reasoning skills (Hauze et al., 2019). A second application (HoloHuman) allows students to explore the different organs and systems of the human body through visualisation of the virtual holograms (Hauze et al., 2019).
Participants and settings
Convenience sampling was used to recruit semester two nursing students enrolled in the three-year Bachelor of Nursing degree programme at one of the tertiary educational organisations in Aotearoa New Zealand (n=6). In addition, nurse educators were recruited (n=4). Nursing students and educators were verbally introduced to the research project in the classroom and staff meeting, respectively.
The age of student participants ranged from 19 to 39 years across the sample. Mostly, the participants did not have any clinical experience except one student who recently finished the clinical placement of five weeks in an acute inpatient setting. However, most of the participants had prior gaming experience with virtual reality platforms other than HoloLens. The total average teaching experience across the educators’ sample ranged from 4 to 18 years. Most of the nursing educators had no to minimal previous experience with using HoloLens.
Data collection
The nursing educators group experienced both HoloPatient and HoloHuman modes through the wearable headset for about two hours before participating in the focus group to share their perceptions and experiences. Nursing students experienced remote simulation (HoloPatient only) using the pre-downloaded HoloPatient app on their mobile devices. Table 1 shows the simulation process. The remote simulation experience was live streamed by the nurse educator using the HoloLens headset. Nursing students were able to see the virtual patient and vital signs screen via their mobile device camera synchronously to the educator while engaging in pre-briefing, simulation activity, and debriefing questions via the online Zoom meeting platform. The 2012 National League for Nursing (NLN) Jeffries Simulation Theory postulates that the simulation participants’ outcomes can be assessed or measured in the form of reaction (learner satisfaction, confidence) and learning gains (change in knowledge, skills, and behaviour) (Jeffries et al., 2015). These domains of participants’ assessment underpinned the focus group questionnaire for nursing educators and students.
Data analysis
Thematic analysis was guided by the Braun and Clarke’s method of thematic analysis (2006) to include steps: 1) familiarisation with the data 2) coding; 3) generating initial themes; 4) developing and reviewing themes; 5) refining, defining and naming themes; and 6) writing up. The transcripts were read and compared with the audio recordings. This step ensured the accuracy of transcripts and allowed for deeper immersion into data. After re-reading, coding was done manually using the Microsoft Word software functionality to highlight and link specific segments of data extracts to the codes. Codes were collated into potential themes using a thematic map. Different codes were colour coded, and pattern and meaning were analysed. The themes and sub-themes emerged during this phase, and the relationships between these became explicit; some of the codes were grouped under one theme and on the other hand, some of the codes emerged as themes. Finally, the themes were clearly defined, described, and refined in relation to the research questions. Several measures had been used by the researcher such as maintaining reflexivity throughout the research process to ensure the trustworthiness of research findings.
Ethical approval
Ethics approval was gained from the University of Auckland Human Participants Ethics Committee (No: 3404) and the tertiary educational institute’s ethics committee. Participants were given a Participant Information Sheet and informed that their participation was voluntary and they could withdraw anytime without requiring any explanation. The focus groups were transcribed by an independent transcriber, who had signed confidential agreement prior to transcribing. Codes, such as Nursing student 1 (NS1), nursing educator 3 (NE3), were used to maintain anonymity. Data was stored and managed as per the University of Auckland research data management policy.
Impact of the COVID-19 pandemic on the research project
This research study took place when Aotearoa New Zealand was grappling with the unprecedented challenges posed by the COVID-19 pandemic (2020-2021). Consequently, the recruitment process underwent challenges with rapidly evolving dynamics of multiple timetabling changes and restrictions on face-to-face learning. The nursing educator group experienced the HoloLens simulation with the technology specialist (USA based) during the pandemic associated social distancing rules. Consequently, only four lecturers experienced the simulation and participated in the focus group to share their perceptions and experiences. The in-person simulation was modified to remote simulation mode for the students due to COVID-19 related restrictions on face-to-face contact.
FINDINGS
Overall, three key themes emerged for the nursing educators and students: 1) benefits of mixed reality simulation learning; 2) challenges associated with the learning; and 3) recommendations for further improvement.
Benefits of mixed reality simulation learning
Visually engaging learning
Nursing educators unanimously commended the feature of the mixed reality device (the HoloLens) to provide an engaging experience through elaborate 3D visualisation. Also, the students placed a value on this feature to visualise the signs and symptoms of the virtual patient, such as itchy skin, change in breathing, level of agitation, and fluctuation of vital signs. This was highlighted in the statement of the Participant NS1 who described how it enabled: “Putting my powers of observation, looking at his vitals, comparing it to his reaction with each stage of the observation…”
Realistic learning
The ‘realism’ of the virtual patient was considered equivalent to the actual patient. Besides, an opportunity to exercise observational skills similar to what students are expected to do in the clinical area provided for a more realistic learning experience, as supported by a nursing educator in the statement:
It will help realistically the students to see what they’re going to see out there. (NE5)
Furthermore, the nursing student reinforced:
With the wounds in front [of you] and how [seeing] his reaction and how he scratches, is like a real person in your head. [NE5]
Potential usefulness as a complementary tool
Despite the clear benefits of using the HoloLens simulation, the nursing educators collectively agreed that it should be leveraged in conjunction with other teaching methods and tools for maximum benefit to nursing students. Participant NE1 explained the best use of mixed reality simulation was by:
Using it in conjunction with the textbook learning, [which] boosts the understanding of theoretical concepts by providing visualisation coupled with an opportunity to experience the concepts practically. [NE1]
Less stressful
The mixed reality simulation provided a less stressful and safer environment for nursing students without any fear that their decisions can harm a real patient:
Being able to detach yourself from reality, knowing that what you’ve got in front of you is nothing but a virtual patient, meaning you can take your time, make as many observations and mistakes as you want, or just start it all over again to improve on your initial assessment. [NS2]
Besides, not having the lecturer present within the same physical space in the online simulation mode, the learning experience was verbalised as being:
Less intimidating with space to focus on learning and ability to make observations. [NS2]
Benefits of remote experience
The remote mode of the mixed reality simulation was adopted in response to the COVID-19 restrictions on face-to-face learning at the time the research study was conducted. Nursing students were able to view the live online simulation from their home through the pre-downloaded mobile app. The ability to see the Hologram of a virtual patient and the vital sign screen using a mobile phone app was positively received in revising and consolidating the learning:
Recalling all our past info and making sense of what we have learnt, even in a small environment like a bedroom is very, very useful. [NS4].
Opportunity to clinically reason
The mixed reality simulation provided an opportunity for the nursing students to observe, process the information, link it to the previous learning, work out what could be wrong clinically with the patient:
Being able to analyse his vitals and think to ourselves okay, what is wrong here, what do I have to be concerned about. [NS3]
Besides, the realistic simulation scenario fostered the introspection of feelings and reactions:
To see a big wound on the back of a virtual patient and how these thoughts relate to the future nursing practice. [NS1]
Challenges associated with the mixed reality simulation learning
Challenge to learn technology
Learning the new and unique HoloLens technology has been acknowledged as a challenge by both nursing educators and students, as reflected in statements from the participants:
There’s no other piece of technology that works the same way as this… even if you’re super tech-savvy, this is a new skill. [NE1]
Another (NS3) stated they were “feeling a bit lost” as they had never used it before.
Limitations to the scope of technology
Nursing students have revealed some limitations to the current scope of technology, such as limited virtual patient-student interaction and the lack of opportunity to practice hands-on skills:
We are using only visual senses at this point, cannot insert intravenously, or the actual interventions. [NS2]
Recommendations for future
Need of prior training
Robust training to use the technology has been suggested as a way forward to deal with the challenge of possible technical difficulty during simulation:
If we can’t get good training [for the students], that’s when they don’t appreciate [it] and use the HoloLens to its full potential. [NE4]
Not feeling “well prepared” or getting “mixed up” was described as feelings by some of the nursing students, which could have been avoided if properly trained prior to the simulation experience as recommended by the nursing students.
Local implementation plan
Apart from the initial training, the availability of expert teaching staff and information technology support was suggested to be a crucial factor for smoother uptake of the mixed reality device programme at the school of nursing. Additionally, development of standardised lesson plans outlining a clear plan for the delivery of content material was recommended for its effective integration into the curriculum.
Refining the technology
Nursing educators emphasised the need to expand the content of mixed reality device’s scenarios, for example:
We’re not seeing the heartbeat, but that’s potential. [NE3]
Additionally, modifying and adapting the simulation scenarios to the Aotearoa New Zealand context – including cultural safety – has been suggested as a potential area of advancement for programme developers. In the future, the technology would benefit from further refinement with the addition of more interactive features, such as the addition of call bells and the ability to practice tasks, such as sewing up wounds and inserting intravenous drips.
DISCUSSION
This small study sought to explore the perceptions and experiences of nursing faculty and students of a mixed reality simulation. The study took place during the COVID-19 pandemic when face-to-face contact was limited and as such, the simulation required the nursing students to engage remotely. This novel approach demonstrated the feasibility of a nurse educator being on site and wearing the headset, while the students connected into the simulation experience through their mobile phones and engaged in pre and post briefing via Zoom. Developing simulation in a remote mode holds the potential to increase access to learning for nursing students.
Nursing educators in our study perceived the rich visual simulation as a beneficial learning experience in engaging nursing students. Congruent to the findings of the current study, the virtual reality simulation platforms have been claimed helpful in enhancing undergraduate nursing students’ learning by providing elaborate 3D visualisation and replicating reality with authenticity (Jenson & Forsyth, 2012; Samosorn et al., 2020; Vottero, 2014). In a mixed methods study, Aebersold et al. (2018) explored the value of the augmented reality tool for nursing students (n=69) to visualise the nasogastric tract. The experimental group experienced the augmented reality simulation training module about tube insertion skill on a tablet (Aebersold et al., 2018). The augmented reality group performed better in the skills of NGT insertion than the control group with statistically significant results (p=0.011) (Aebersold et al., 2018). For simulation to be effective, a high degree of realism is necessary (Bowen-Withington et al., 2020). The provision of a high-fidelity simulation experience for novice nursing students without any prior clinical experience may support them to envision the reality of the clinical world.
In our study, the nursing students positively appraised the mixed reality simulation, which fostered their thinking like a nurse and clinical reasoning. High fidelity simulation has been seen as an effective tool in fostering the development of clinical judgment and reasoning skills for nursing students as well as promoting confidence (Cant & Cooper, 2010; Lei et al., 2022). However, the nurse educators suggested the importance of combining learning approaches. There is limited available evidence on the solo use of virtual reality simulation compared to being used in conjunction with other teaching methods. However, a randomised controlled trial by Smith and Hamilton (2015) concluded that using virtual reality simulation as a supplemental tool for clinical tasks, such as the insertion of an in-dwelling catheter, supported nursing students’ clinical learning. For simulation scenarios to be effective, there is a need to ensure they reflect and support the development of not only technical tasks but also soft skills, such as communication and teamwork (Bowen-Withington et al., 2020).
Our study was particularly interested in the technical application of the mixed-reality simulation. Simulation provides the opportunity for a safer and less stressful environment without posing any risk to the actual patient is an essential feature of the simulation pedagogy (Cant & Cooper, 2010; Ulrich et al., 2014; Verkuyl et al., 2017). On the other hand, technological difficulties and first-time exposure of nursing students to the new virtual reality simulation tool may create high anxiety levels (Cobbett & Snelgrove-Clarke, 2016). Using a new technology can be stressful for students and there is a need for good education to ensure students can fully participate in the learning experience.
Literature elsewhere has identified the importance of attending to technical glitches and issues, which impair the functioning of the virtual reality tools (Saunder & Berridge, 2015; Ulrich et al., 2014). The nurse educators in our study identified the necessity of information technology support. Likewise, the Aotearoa New Zealand study by Rummel et al. (2023) reinforced the need of additional training to tackle the issue of technical difficulties when experiencing mixed reality simulation. There is a need for key stakeholders, including nurse educators, nursing students, and clinicians to collaborate with mixed reality developers to expand the scope and value of the technology for effective use in nursing education. One such area of potential development is to enhance the simulation scenarios to prepare the nursing students for the provision of culturally safe care that forms an integral part of the competency standards for nurses in Aotearoa New Zealand (Nursing Council of New Zealand, 2012).
The nurse educators identified the need to develop a clear lesson plan, in line with best simulation practice guidelines provided by the 2012 NLN Jeffries Simulation Theory (Jeffries et al., 2015). The simulation learning should clearly outline the written learning objectives, learning outcomes, cues for nursing students, briefing and debriefing activity (Jeffries et al., 2015). For educational institutions and educators, it is necessary to recognise that the development of simulated learning experiences for nursing students requires considerable preparatory time (Bowen-Withington et al., 2020). Adequate time and well facilitated debriefing is considered one of the most important activities in simulation (Macdiarmid et al., 2020). High fidelity simulation using mixed reality devices has the potential to provide excellent learning experiences for nursing students with appropriate planning and support.
Limitations
The findings of this research study have limited generalisability as the sample size was small. The study occurred during the COVID-19 pandemic, which impacted the recruitment process and increased attrition in the sample. At the time nurse educators and nursing students were under considerable stress. This study warrants further research with a larger sample size of nurse educators and nursing students. The study explored the mixed reality simulation used remotely leaving aspects of face-to-face use of the simulation unexplored.
CONCLUSION
The study set out to provide insight into the perceptions and experiences of nursing students and educators with a mixed reality simulation device. Overall, nurse educators and undergraduate nursing students identified the value of this simulation mode to provide visually engaging, realistic, and safer learning experiences. Additionally, the opportunity to learn the skills of clinical reasoning and ease of access from home settings was commended by the students. However, the technological complexity associated with using a mixed reality device and headset were emphasised, identifying the need for robust training before the simulation experience. Refinement to the scope and content of the technology was recommended, including using the technology to develop more hands-on and soft skills, as well as scenarios which reflect the Aotearoa New Zealand context and enable learning to support the Nursing Council of New Zealand competency requirements for cultural safety. Mixed reality simulation has the potential to be effectively implemented to supplement undergraduate nursing education.
Acknowledgments
The author would like to thank Michael Crossan, Professional Teaching Fellow, School of Nursing, University of Auckland for his vital input in the initial stages of development of the research project. Thanks to the participants, nurse educators, and students for sharing their valuable perceptions and experiences during the unprecedented challenging times of pandemic.
Funding
None
Conflict of interest
None