Together with Lufthansa Aviation Training and RotorSky, RSA FG is continuing its work to further improve training for pilots. The use of mixed reality and eye tracking does not only record the behavior of trainees, but also provide them with direct feedback during training.
Studio PCA is involved in another project to improve training for pilots. Following the completion of Airtention, it is now working on “EmPACT”: Empowering Pilots via Adaptive Competency-based Training. The aim is to develop training towards competence-based training and assessment (CBTA). In addition to RSA FG and LAT, three Austrian companies – SOMAREALITY, RotorSky and Norden.aero – are contributing their expertise to EmPACT at the Zurich site.
In the Airtention project, RSA FG developed a system together with Lufthansa Aviation Training Switzerland AG Zurich (LAT) to record visual attention markers and thus evaluate the technical skills of the trainees. The use of eye tracking made it possible to measure, among other things, gaze behavior on the instruments, general attention and the handling of disruptive factors.
However, optimized training also requires individual support, not just pure monitoring. This active part of the training can be achieved on the one hand by instructors adapting the training content and providing them with real-time information in a dashboard. On the other hand, this can also be achieved by specifically drawing the trainees’ attention to information that has been overlooked, for example through overlays. Mixed reality (MR) technology is ideal for implementing the project: Here, the monitoring technologies developed in Airtention are combined with another level through eye tracking: the active and immersive provision of information to the trainees using MR technology.
However, optimized training also requires individual support, not just pure monitoring. This active part of the training can be achieved on the one hand by instructors adapting the training content and providing them with real-time information in a dashboard. On the other hand, this can also be achieved by specifically drawing the trainees’ attention to information that has been overlooked, for example through overlays. Mixed reality (MR) technology is ideal for implementing the project: Here, the monitoring technologies developed in Airtention are combined with another level through eye tracking: the active and immersive provision of information to the trainees using MR technology.
The vision of the EmPACT project is to develop the next level of CBTA technologies to support training by complementing the current purely passive monitoring technologies with active feedback and adaptive training. The project therefore aims to improve pilot training with the help of technology.
Three different training scenarios were implemented for this purpose:
- VR training at home: consumer headsets for home use as a platform for conducting introductory and repetition training in order to make the best use of expensive simulator time.
- MR technologies in standard simulators: MR headsets as an extension of standard static simulators, such as the helicopter simulators at RotorSky
- MR technologies in MR simulators: High Level Mixed Reality Simulator at LAT in Zurich.
Attention Aware Systems (AAS) can record the cognitive states of users using sensors, model them, then document them and intervene to control them. The aim of the “Airtention” project was to develop such an AAS for pilot training with the intention of modeling human attention in order to assess the learning progress of trainees in various key competencies.
In addition to the studies in simulation training environments, one of the objectives of the Airtention project was to investigate the transferability of the developed technologies to real-life application and use in real aircraft. The primary aim was to test the integrated hardware under uncontrolled environmental conditions (e.g. bright sun, rapidly changing light conditions, etc.), to research the applicability of the sensor technology, to apply analysis algorithms and to collect study data from real flights.
