User Modeling from Behavioral Time Series in Virtual Reality Environments

Doktorand: Luis Quintero, DSV

Expertgranskare: Jefrey Lijffijt, Universitetet i Gent och Robert Ramberg, DSV

Huvudhandledare: Uno Fors, DSV

Handledare: Panagiotis Papapetrou och Jaakko Hollmén, DSV


Several digital services used in our daily offer personalization powered by machine learning (ML). The combination of ML and virtual reality (VR) environments could drive the creation of systems that support personalized healthcare or adaptive learning in immersive environments, like the metaverse. The main challenge is that personalization heavily depends on the user, the context, and the system's purpose. In this midterm seminar, we present four papers exploring how ML can leverage behavioral data to detect human cognitive factors in real-time and create personalized VR applications. Specifically, we propose methods to process physiological time series from heart and brain wearables, motion trajectories from VR trackers, and gameplay interactions embedded in immersive systems to detect skill level and emotional states. We also outline possible future studies harnessing methods from explainable ML and reinforcement learning to examine the performance of adaptive VR systems in real-world tasks.