Most people who attempt to get fit using VR drop out when their workouts gets tough or dull – could programs that use sensors to tune into a user’s emotional state improve adherence?
Virtual reality (VR) video games that combine screen time with exercise are a great way to get fit, but game designers face a major challenge – like with regular exercise, adherence to ‘exergames’ is low, with most users dropping out once they start to feel uncomfortable or bored.
Computer scientists at the University of Bath believe they’ve found a solution: create exergames that use sensors to continuously measure a person’s emotional state while they exercise, then tweak the game – for instance, making it easier or harder – to keep the user engaged.
Dr Dominic Potts, lead author of a new study into harnessing cutting-edge sensor technology to keep exercisers motivated, said: “When it comes to physical exercise in all forms, motivation and exercise adherence are huge problems. With exergaming, we can address this issue and maximise a person’s enjoyment and performance by adapting the challenge level to match a user’s abilities and mood.
“Exercise games that are completely adaptive will sense a person’s emotions and give them more ‘rewards’ when they’re struggling and more obstacles when they’re ready for a new challenge.”
Game designers have long aspired to develop more personalised exergames – i.e. programs that tune in to a person as they work out, adapting to their struggles and ambitions of the game unfolds – however, finding a trustworthy method for measuring a user’s evolving emotional state has proven elusive.
The Bath team has made a breakthrough by employing a novel range of sensors – which could be embedded in VR headsets and wearable devices such as smartwatches – to track physical changes experienced by an exerciser as they work out.
The team’s findings were published recently at the CHI Conference on Human Factors in Computing Systems – the premier international conference of Human-Computer Interaction (HCI), where the paper received an honourable mention award.
The researchers hope their findings will be adopted by game designers to create immersive programs that can keep a person peddling, running or weightlifting long after they’d normally choose to quit.
Background noise
Until now, sensors – which can be extremely effective at tracking emotional states when a person is sedentary – have proven unreliable at measuring the emotional landscape of a person involved in physical activity, making it hard to recognise if a person is experiencing happiness, stress of boredom.
“Traditionally, these sensing devices have been put into VR headsets to track blinks and pupil dilation, but generally they are highly susceptible to physiological and background noise,” explained Dr Potts.
Unwanted ‘noise’ is generated in two ways: by a person moving in unpredictable ways – as they often do when exercising – and by the exerciser responding to the virtual environment (VE) they’re immersed in. So, for instance, a sensor that shows a person’s pupils dilating as they exercise might reflect luminosity changes in the VE rather than the user’s evolving emotional state.
Reliable tracking
In the new study, 72 participants were involved in a VR static bike race while the Bath scientists used a specific combination of sensors to measure pupil size, facial expressions, heart rate, levels of sweating, skin inflammation and electrodermal activity (which measures the skin’s ability to conduct electricity, reflecting stress levels).
Data was collected through the sensors while racers worked out in four distinct VEs, where each environment was designed to induce a specific emotion (happiness, sadness, stress, and calmness). Participants moved through these VEs at three different exercise intensities (low, medium, and high).
For each workout, the researchers were able to paint an accurate picture of a user’s emotional state, matching the game’s level of difficulty and the nature of the VE with the physiological changes experienced by the user, as picked up by their sensors. Drawing from this research, eight guidelines have been formulated for creators of VR exergame, aimed at enhancing the emotional engagement of users. Among these recommendations are the following:
- Pupil-detecting sensors should be designed to correct for luminosity changes in the virtual environment.
- Preexisting sweat levels of a user should be taken into account to predict stress and arousal of the nervous system.
- Sensor data should be cleaned before and during an exergame (i.e. removing signals unrelated to emotional changes) to measure interpersonal differences and enable the game to be tailored to each user.
- Multiple physiological sensors should be used to improve predictions of a user’s emotional state.
Dr Christof Lutteroth, who leads exergame research as director of the REVEAL research centre and is coinvestigator at CAMERA, both based at the University of Bath, said: “In the long run, our objective is to make VR exercise emotionally intelligent.
“We fully expect VR physical activity to explode in popularity in the years ahead – school children are already using them as part of their exercise programmes and they are also being used in rehab and sports science – so it’s important to focus on making technology that’s emotionally intelligent and adaptive to differences between users.”
The University of Bath research team for this study included: Dr Dominic Potts, Masters student Zoe Broad, Psychology undergraduate Tarini Sehgal, Joseph Hartley, Professor Eamonn O’Neill, Dr Crescent Jicol, Dr Christopher Clarke and Dr Christof Lutteroth.
This work is supported by the European Union’s Horizon Europe research and innovation program and Innovate UK under grant agreement No 101070533, project EMIL (The European Media and Immersion Lab): https://emil-xr.eu/.
