AI or VR? Matching emerging tech to real-world learning

Defining the problem can determine the best tech to deliver realistic training environments. Here is how one media lab demystifies technology-enabled learning experiences and empowers educators to develop their own

 

The gap between what we teach and what employers need is widening rapidly. This gap is not only in knowing how to use emerging technology but also in effective education in problem-solving, collaboration and the ability to adapt to change.

At the University of Sydney, our Faculty of Medicine and Health (FMH) has made great strides in how we educate our health and medical students. The recent announcement of A$10 million (£5.7 million) for a “futuristic fitout” at the Westmead Innovation Centre is just part of a wider push to bring our teaching and our students’ experiences into the 21st century – embracing the power of technologies such as artificial intelligence (AI) and virtual reality (VR).

At the FMH Media Lab, we test emerging tech to see if it can deliver more effective learning experiences than current practice. Defining the problem determines the choice of tech.

We have had success with a range of solutions – from complex VR simulations to AI-driven interactions with conversational agents (which are dialogue systems that conduct natural language processing and respond automatically using human language) to requiring students to curate exhibitions in a 3D digital museum. We create digital copies of real-world objects such as anatomy models and use real estate tour software to create guided tours of our simulation training labs to orientate students, as well as creating CGI, videos, podcasts and other multimedia.

• The library and the metaverse: a match made in heaven
• Using immersive tech to enrich healthcare education
• How to deliver healthcare education in a hybrid world

The FMH Media Lab began life at the University of Sydney’s Westmead campus, which is embedded within Westmead Hospital in Western Sydney, Australia. This generated close collaborations with front-line clinical educators delivering in-service training.

Our approach was to develop training environments that were as close as possible to the practice environment because learning complex skills such as managing medical emergencies is time based and embodied. By developing VR simulations of an advanced life support (ALS) scenario, we are able to curate learning experiences that are more indelible than the existing approach, which is to study the ALS handbook. These tools are already being integrated into nursing and medical education at the Westmead Innovation Centre.

Image

 

Training for the team leader in an advanced life support response
Photo: courtesy of the University of Sydney

 

From educators’ teaching problems to prototype

We like it when an educator comes to us with a problem. We often present the solutions we have developed so educators can get a sense of how they might apply them in their situation. We start by getting to know the learning activities, the desired learning outcomes and the logistics: how many participants, how many hours. We encourage the educators to try the technology: to put on a VR headset or experience a digital tour of a remote or inaccessible location so they get a sense of the experiential nature of the activity.

If the scale of the learning situation justifies building a digital asset, we like to work up a prototype and improve it iteratively, following the educators’ feedback. Some solutions, such as putting together a digital tour that can act as an orientation exercise or assessment, are easy to develop, taking maybe only a couple of hours. Other solutions, such as an AI-driven chatbot or a VR app, require more resources, so the context must warrant the effort.

We also like to co-design the experiences with both undergraduate and postgraduate students whenever we can. If the new learning experience is successful, we work with the students and educators to finesse how it can best be integrated into the curriculum design and lesson plans.

Making tech transparent and accessible

We see our role as demystifying technology-enabled learning experiences and empowering educators to develop their own. We like to operate within a research framework, constantly seeking to pin down exactly what measurable outcomes we anticipate. We find that emerging technology can provide scalable complex experiences that are difficult or impossible to routinely reproduce for every student in the class, such as rehearsing verbal de-escalation skills in a hospital setting, taking a patient history or performing a patient handover between health professionals.

Image

Code Black training for de-escalation of violence in the emergency department
Photo: courtesy of the University of Sydney

 

In assignments such as the curation of a medical exhibition in a 3D digital museum, students are actively engaged in object-based learning pedagogy and are required to demonstrate a coherent “narrative” through time and space. Their assignments are viewable on flatscreen but can also be experienced in VR, meaning students and their educators (and their parents and future employers) can walk around in their assignment. Acquiring practical skills in a new digital platform, a valuable workplace skill, occurs as a by-product of the assignment. We believe this assignment creates more indelible, more readily retrievable memories, and we are researching this.

How to work with educators to develop tech solutions

We suggest this approach:

• Read up on emerging tech.
• Buy a few and make some things.
• Let your colleagues know you’re looking for use cases.
• Scope the opportunity: current practice, learning outcomes, scale.
• Make a prototype and test it with colleagues and students.
• Make iterative improvements and test.
• Don’t be afraid to try: it’s only technology.

So, the FMH Media Lab operates as resource for educators, making emerging technology-enhanced learning experiences available in class in a way that integrates with and supports current practice and provides more experiential learning experiences.

Martin Brown is co-director of the Faculty of Medicine and Health Media Lab; Phil Poronnik is director of education innovation in the education innovation research theme in the School of Medical Sciences; Claudio Corvalan-Diaz is technical officer in the education innovation research theme in the School of Medical Science; and William Havellas is technical and virtual assets officer in the education innovation research theme in the School of Medical Science. All are at the University of Sydney.

If you found this interesting and want advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.