Session: Technology enhanced learning in orbit: inside ESA's mixed reality trainer

• Event: ISE 2025
• From session: Reality-Embedded Learning
• Speaker: Fridolin Wild, Professor, Institute of Educational Technology; leads the Performance Augmentation Lab, The Open University
• Estimated read time: 8 minutes
• Watch the full session

Quick read summary

• Technology enhanced learning is leaving the screen: a European Space Agency trainer on Apple Vision Pro teaches astronauts inside a digital twin.
• Fridolin Wild of The Open University argues the skills gap makes this the greatest opportunity ever for learning technology.
• Remote trainers join as avatars, and AI instructors answer from the manufacturer’s procedure document, not from a general language model.
• In an October 2024 pilot, 12 of about 20 European astronaut trainers saw potential to prevent errors and cut travel.
• The hardest problem is still ahead: tracking hands and fingers inside the massive gloves of a space suit.

Partway through his talk at ISE 2025, Fridolin Wild showed a clip of a virtual instructor beamed into a living room. She is asked how to open the airlock on the International Space Station. The sound failed, so he paraphrased her reply: a job for astronauts, then a confident account of how it might work, improvised by a general language model. For technology enhanced learning, that confident guess is the problem to solve.

Wild is a Professor at The Open University, where he leads the Performance Augmentation Lab. He was showing a training system his team built with the European Space Agency (ESA). It runs on Apple Vision Pro, puts trainers and trainees in a shared copy of a station module, and holds its virtual instructors to the manual. ISE covered the session in an earlier report on Wild’s talk.

Is the skills gap the biggest chance technology enhanced learning has had?

Wild thinks it is. He cited an International Monetary Fund (IMF) prediction that almost 40 per cent of global employment may be affected by AI. A survey The Open University runs with the Chamber of Commerce found that 73 per cent of organisations had recruitment problems in 2023. Some 42 per cent could not fill jobs.

That gap, he said, "certainly is the greatest opportunity ever for technology enhanced learning."

Longer missions break today’s astronaut training

Astronaut training has to change because missions are becoming too long and too distant for the current model. Training takes about two to three years today, Wild said, and the full theoretical training is estimated at six to seven years. Front-loading all of it stops being realistic once the destination is the Moon again, and ultimately Mars. His answer is training that is more autonomous and more flexible, delivered where the knowledge is needed, or during transit.

The test case was a filter change that keeps the crew alive

The team chose the cabin filter change in Columbus, the European module of the International Space Station, as its test procedure. A HEPA filter cleans the module’s air. Wild was blunt about the stakes: "if it’s not changed regularly, they die." The procedure runs to about 30 precise steps and is, Wild admitted, boring. That makes it a good test: errors are costly and the steps are fixed.

Why does the place matter to educational technology?

The place matters because context helps people remember, so where training happens shapes what learners can recall later. Wild places the ESA project in the fourth of four ages of educational technology. Digitisation came first, then learning management systems such as Moodle, Canvas and Brightspace, then personalised apps such as Duolingo. The fourth is what he calls embedded learning: information placed where the action happens, users freed from small screens and reality used as the canvas.

That is the case for training where a task will be done, or in a faithful copy of it. Wild was candid that routine use is still some way off.

Which devices can carry training into the real world?

Almost any display that blends the real and the simulated can carry it, from handheld devices to magic mirrors and spatially aligned projectors. Wild sorts the options along two axes: fully virtual to augmented reality, and shared to personal.

For the ESA build, his team chose Apple Vision Pro, which he called best in class among mixed reality devices.

How does the mixed reality training system work?

The system puts a trainer, a trainee and the procedure in one shared space, wherever each person is. A remote trainer or trainee beams in and interacts by voice and body movement, seeing the physical environment through a digital twin of the Columbus module.

The twin can run as full virtual reality, or as a light turquoise wireframe that shows where people and objects are while keeping attention on the training content. Content is anchored to the physical location, so a person in the module and a remote user see the same things.

The avatars took real work, from interpolating foot movement to latency and lip sync, so they talk rather than just stand around.

How does retrieval augmented generation keep a virtual instructor honest?

Retrieval augmented generation (RAG) keeps an instructor honest by taking the content of each answer from an approved knowledge base and using the language model only to phrase it. That was Wild’s fix for the airlock problem. Here is the flow for one question about the filter change:

1. The trainee looks at one of the virtual instructors and asks about the next step.
2. An orchestration layer turns the speech into text and uses gaze to decide which instructor answers.
3. The knowledge base holds the manufacturer’s servicing document, split into chunks and turned into vector embeddings that a computer can search by meaning.
4. The system finds the chunk that holds the answer and passes it to a large language model with a short background prompt, such as "keep it short".
5. The model phrases the reply, but the content comes from the document, which stops the instructor hallucinating.
6. The avatar replies with lip sync and gestures, walks to the right spot and demonstrates.

The trainers saw a way to prevent errors

The trainers saw strong potential, above all for preventing errors. In October 2024 the team evaluated the system with 12 European astronaut trainers. Wild noted that sounds like a lot until you know there are only about 20. Astronauts interviewed beforehand had pinned their hopes on 3D simulation and on practising procedures. For emergency response, Wild said, the ISS already has excellent virtual reality training.

Participants valued a trainer that adapts to each user, is available at any hour and cuts travel. The hardest problem, Wild said, is further off: delivering all this inside a space suit.

FAQ

• Does technology enhanced learning only apply to classrooms? No. Fridolin Wild of The Open University argues that technology enhanced learning is moving into real workplaces, delivered through XR and AI where the task happens. His team’s work with the European Space Agency trains astronaut procedures inside a digital twin of the Columbus module.
• How is mixed reality training different from a fully virtual simulation? A fully virtual simulation replaces the user’s surroundings, while mixed reality combines the real environment with digital content anchored to it. The European Space Agency system can do both, switching between full virtual reality and a light wireframe view.
• Can an AI instructor be trusted with a safety-critical procedure? An AI instructor can be trusted only when its answers come from approved documents rather than a general language model. Fridolin Wild’s team splits the manufacturer’s procedure into searchable chunks and uses the model only to phrase the reply.

The suit is the real classroom

The striking thing about Wild’s project is the order it works in. Training usually brings people to a classroom or a simulator. This system brings the training to the work, and then finds that the work sets the hardest limits. A 30-step filter change is easy to model. Guiding someone through it in a pressurised suit, when the headset cannot track fingers inside a glove, is not solved yet.

Any training provider can use the parts that already work: a spatially accurate twin, a shared view for remote experts and an instructor that refuses to guess. The airlock moment showed why the last one matters most. A confident wrong answer is worse than none when the procedure is the difference between clean air and no air.

Wild waits every year for the devices to disappear into the surroundings, and admits it has not happened yet. Still, if grounded guidance can be trusted with a task that keeps a crew breathing, the case for a training room on the ground is already made.

About the speaker

Fridolin Wild
Professor, Institute of Educational Technology; leads the Performance Augmentation Lab, The Open University
Wild researches how XR and AI can deliver learning in the real world, and works with the European Astronaut Centre of the European Space Agency on mixed reality systems for astronaut training.

More on learning technology at ISE

Immersive learning is part of a much wider edtech story at ISE.

• Collaboration and education tech at ISE: classroom tech, the Connected Classroom and EdTech Congress.
• Emerging technologies at ISE: XR and mixed reality beyond the classroom.
• ISE’s content programme: where to find the sessions that matter to your role.
• New to ISE? Get your bearings with the first-time visitor guide.