Project Information
Runtime:
- Mai 2024- 31. Dezember 2024
with
The aim is to design and implement an AI-driven receptive music-based intervention (RMbI) for dementia patients. Since musical preferences and the effect of music are highly individualized, XR technologies (XR headsets, embedded eye-tracking, embedded hand-tracking) will be built upon for (i) an automated approach for musical anamnesis (identify, which music elicits engagement and activation, positive affectivity in individuals), (ii) an RMbI design (creation of playlist and interactive gamification elements) and (iii) an evaluation of RMbI impact by analyzing cognitive and affective functions in the interactive scenarios.
The following components are realized in the project:
-
- XR gamification prototype: simple games are realized in an XR environment (Varjo headset) and combined with music to enable an exact and repeatable study process
- Sensor integration: the various sensors (eye tracking in the Varjo headset, external cameras, empatica wristband for skin conductance, heart rate monitor, etc.) must be integrated and connected to the extent that simultaneous analysis of the various sensors is possible
- Data framework: Data from both the gamification prototype and the sensor technology are stored in a common data framework, where they can be further processed and analytically evaluated
- Study implementation: the effect of the music in the game on the various indicators is systematically studied and documented
The implementation of a cognitive musical anamnesis system would represent significant added value both in medical applications and in scientific utilization. The XR gamification prototype has a wide range of potential applications: it can serve as a basic technology in future research projects, but can also be used as an acquisition asset and demonstrator to directly demonstrate crucial specialist knowledge and expertise and thus strengthen acquisition activities and external image.
