Affective gaming in real-time emotion detection and music emotion recognition: Implementation approach with electroencephalogram

Abstract

Affective Gaming can be considered as the concept of detecting the real-time emotional state of a player during various stages in gameplay and then enhancing the user interactivity accordingly to the emotional state. Based on this conception, this paper presents the research phase of the development of an Affective Car Racing computer game. The designs were created based on the theory of “Affective Loop” in games. Affective Loop consists of Emotion Elicitation, Emotion Detection/Modelling and finally Emotion Expression by Game Engine. This paper considers the second and third subphases of this loop. Designs are done for these two phases based on technologies that are still not been utilized by many game developers when designing a game. Emotion Detection/Modelling phase is introduced with a technique of capturing Electroencephalography (EEG) signals for predicting the real-time emotion of the player while interacting with the game engine. Emotion Expression phase considers the concept of Music Emotion Recognition (MER), which is a novel concept for the Gaming Industry. The authors had trained SVM models for emotion modeling via EEG Signals that will be captured by the Emotiv Epoc 14 channel device. The authors had classified the Rock and Electronic genre of music via Multi-Label RAKEL classification (Precision score of 75%) to play music excerpts based on the effect of the gamer during gameplay.