Neuronal correlates of emotions in human-machine interaction

Introduction and method

Previous neurophysiological studies of emotions have focused on the affective response in the emotional valence of a situation in which the reaction is rooted in perception or memories [1]. Furthermore emotions have been investigated with regard to the trait of a subject, e.g. anger-out vs. anger control [2] and regarding motivational direction, e.g. approach vs. withdrawal [3]. Aiming at an enhancement of human-computer interaction by incorporating the emotional state of the user, a novel type of investigation is required. Neuronal correlates of emotional reactions related to interaction (e.g. annoyance due to one's own failure or an error of the machine; joy of success) have to be analyzed and methods for their detection in real-time need to be developed. In the present study we have acquired multi-channel EEG in four subjects while they were interacting with computer applications that have been specifically designed in order to provoke – in alternating phases – neural, positive or negative (stress, annoyance) emotions. In particular, a two-player variant of a two-alternative forced-choice task had to be performed while in alternating periods either one or the other player was given "unfair" preferential treatment by providing the task stimulus slightly in advance. This bias could not be noticed by the players.

Results and discussion

The behavioral data are consistent between subjects and indicate that the participants had a temporary feeling of inferiority and adapted their strategy (accepting higher error rates in order to achieve faster reaction times to cope with the competitor). As neuronal correlates, intra-individual significant differences between periods of negative and positive emotions were found in the theta-, alpha-, or beta-band with widely distributed and spatially coherent topographies, see Figure 1 for the result from one subject. Notably, the frequency band as well as its spatial focus varied between subjects. The variety of EEG correlates found among the four subjects already demonstrates the need for adaptive methods in order to enhance human-machine interfaces by emotional decoding. Experimental studies with a larger number of subjects will show whether some pattern clusters could be identified that potentially correlate with individual attitudes towards the particular emotion-provoking environmental situation.

Figure 1

Topographical maps of alpha band-power (9–13 Hz) for the emotional states "stress" and "annoyance" (minus baseline) in one subject. For comparison a map of alpha band-power is shown for the condition "eyes closed" which has a focus clearly different from the other states.