User Modeling from Behavioral Time Series in Virtual Reality Environments

PhD student: Luis Quintero, DSV

Expert reviewers: Jefrey Lijffijt, University of Ghent and Robert Ramberg, DSV

Main supervisor: Uno Fors, DSV

Supervisors: Panagiotis Papapetrou and Jaakko Hollmén, DSV


Several digital services used in our daily offer personalization powered by machine learning (ML). The combination of ML and virtual reality (VR) environments could drive the creation of systems that support personalized healthcare or adaptive learning in immersive environments, like the metaverse. The main challenge is that personalization heavily depends on the user, the context, and the system's purpose. In this midterm seminar, we present four papers exploring how ML can leverage behavioral data to detect human cognitive factors in real-time and create personalized VR applications. Specifically, we propose methods to process physiological time series from heart and brain wearables, motion trajectories from VR trackers, and gameplay interactions embedded in immersive systems to detect skill level and emotional states. We also outline possible future studies harnessing methods from explainable ML and reinforcement learning to examine the performance of adaptive VR systems in real-world tasks.