Assessing the design and potentials of an Adaptive and Interactive Music System for Biosensor-Based Serious Games
The serious (applied) games market is forecast to raise considerably by 2025 , with its popularity within educational, medical, training, simulation and advertisement applications , . Serious games are created to fulfil an learning objective, rather than for pure entertainment . Furthermore, Ravyse et al (2017) define them to have an intended impact on cognition, behavior or motor skills and reviewed what are the success factors that enhance learning .
The adoption of neuroscience, clinical psychology and meditation research within serious games has created a new genre known as ‘neurofeedback’. This has created a market for personal EEG and biosensor devices, as an affordable, high-resolution alternative to professional EEG equipment, along with applications to be used both in the home and medical environment , , , .
The designs of these applications are starting to incorporate machine learning algorithms, that have been proved by EEG research studies to have successfully identified arousal and emotion at a relatively high-resolution , , . Research in combining EEG with physio sensors to enhance the resolution and to investigate other neuro-signatures is still in the early stages , . Predominantly, neurofeedback research is neuroscience based and is commonly aimed towards clinical psychology practitioners in non-pharmacological treatment for ADHD and other psychiatric disorders . To date, only a small set of computer science studies tackle neurofeedback game design concepts for both entertainment and applied games , . Equally limited is audio specific research, with only one reviewing the design of an adaptive audio system for a EEG-based horror game, another in the design of an audio-biofeedback system for patients with parkinson’s disease and a meditation mobile application designed around audio-feedback for the Interaxon Muse Headset , , . Moreover, there are no neurofeedback games that explore music creation and expression to create an immersive musical experience during gameplay, other than by individual artist installations , , . However, extensive research has proved the potential of game audio for wellbeing, audio as gameplay and interactive music video games for musical development , , , , .
Predominantly, the design of serious games use pre-existing popular games such as minecraft and real-world situations , . Currently, there seems to be a lack of studies identifying the effects of applied games and a lack of novel designs within the consumer serious game market. However, M.Nazry and Romano (2017) created two navigation-based serious games and successfully proved that the relationship of enhanced mood and improved learning efficiency existed, with the findings showing the games did enhance mood post-experience, there was an overall trend of improved learning in-game and time spent was longer and more engaged when happier . Another area to explore is the role empathy plays in developing engagement. There have been extensive case studies and VR experiences that have been used as an effective tool to shift individuals into pro-social emotional states , . In particular, it has become popular amongst applied games designed to teach primary/secondary school children .
From this apparent gap between the relationships of game audio, EEG-based game research and the effects of serious games, I propose to examine the potentials of using adaptive audio as a form of musical expression for biosensor-based serious games to aid in improving people’s mood, wellbeing and creative performance.
“Global Serious Game Market to Reach 9840 Million US by 2025, from 2770 Million US in 2018,” 16-Jan-2019. [Online]. Available: http://www.wboc.com/story/39804156/global-serious-game-market-to-reach-9840-million-us-by-2025-from-2770-million-us-in-2018. [Accessed: 25-Jan-2019].
“EEG (Electroencephalography) Systems Market | Industry Report, 2025.” [Online]. Available: https://www.grandviewresearch.com/industry-analysis/electroencephalography-eeg-systems-devices-market. [Accessed: 25-Jan-2019].
P. Frye, “Global Consumer EEG Device Market Report 2018: Neurosky & More,” Positive Source24, 07-Sep-2018. [Online]. Available: https://positivesource24.com/global-consumer-eeg-device-market/. [Accessed: 25-Jan-2019].
D. Wortley, “The Future of Immersive Technologies and Serious Games,” in Subconscious Learning via Games and Social Media, O. Sourina, D. Wortley, and S. Kim, Eds. Singapore: Springer Singapore, 2015, pp. 45–56.
M. A. Hogervorst, A.-M. Brouwer, and J. B. F. van Erp, “Combining and comparing EEG, peripheral physiology and eye-related measures for the assessment of mental workload,” Front. Neurosci., vol. 8, p. 322, Oct. 2014.
K. Katahira, Y. Yamazaki, C. Yamaoka, H. Ozaki, S. Nakagawa, and N. Nagata, “EEG Correlates of the Flow State: A Combination of Increased Frontal Theta and Moderate Frontocentral Alpha Rhythm in the Mental Arithmetic Task,” Front. Psychol., vol. 9, p. 300, Mar. 2018.
X. Y. Lee, E. Koukouna, C. G. Lim, C. Guan, T. S. Lee, and D. S. S. Fung, “Can We Play with ADHD? An Alternative Game-Based Treatment for Inattentive Symptoms in Attention-Deficit/Hyperactivity Disorder,” in Subconscious Learning via Games and Social Media, O. Sourina, D. Wortley, and S. Kim, Eds. Singapore: Springer Singapore, 2015, pp. 69–86.
“On Your Wavelength, Marcus Lyall in collaboration with Rob Thomas and Alex Anpilogov,” Merge Festival. [Online]. Available: http://mergefestival.co.uk/merge-events-2015/2015/8/18/on-your-wavelength-marcus-lyall-in-collaboration-with-rob-thomas. [Accessed: 25-Jan-2019].
G. F. Tondello, R. Orji, K. Vella, D. Johnson, M. M. M. van Dooren, and L. E. Nacke, “Positive Gaming: Workshop on Gamification and Games for Wellbeing,” in Extended Abstracts Publication of the Annual Symposium on Computer-Human Interaction in Play, 2017, pp. 657–660.
“Learning Games - 26 Serious Games to Change the World,” Mission to Learn - Lifelong Learning Blog, 18-Apr-2008. [Online]. Available: https://www.missiontolearn.com/learning-games-for-change/. [Accessed: 30-Jan-2019].
S. Hill, “Your Brain on VR Empathy – Virtual Reality Pop,” Virtual Reality Pop, 05-Jan-2017. [Online]. Available: https://virtualrealitypop.com/your-brain-on-vr-empathy-fc7a3743b2e5. [Accessed: 30-Jan-2019].
“Top Games That Teach Empathy | Common Sense Education,” Common Sense Education, 24-Sep-2013. [Online]. Available: https://www.commonsense.org/education/top-picks/top-games-that-teach-empathy. [Accessed: 30-Jan-2019].
Home institution: Queen Mary
Supervisor: Dr Laurissa Tokarchuk
Ready to apply?
Once you have identified your potential supervisor, we would encourage you to contact them to discuss your research proposal.Learn More