Towards a cure for BCI illiteracy: machine learning based co-adaptive learning

Brain-Computer Interfaces (BCIs) allow a user to control a computer application by brain activity as acquired, e.g., by EEG. One of the biggest challenges in BCI research is to understand and solve the problem of "BCI Illiteracy," which is that BCI control does not work for a non-negligible portion of subjects (estimated 15% to 30%). In a screening study, N = 80 subjects performed motor imagery first in a calibration (i.e. without feedback) measurement and then in a feedback measurement in which they could control a 1D cursor application. Coarsely, we observed three categories of subjects: subjects for whom (I) a classifier could be successfully trained and who performed feedback with good accuracy; (II) a classifier could be successfully trained, but feedback did not work well; (III) no classifier with acceptable accuracy could be trained. While subjects of Category II had obviously difficulties with the transition from offline on online operation, subjects of Category III did not show the expected modulation of sensorimotor rhythms (SMRs): either no SMR idle rhythm was observed over motor areas, or this idle rhythm was not attenuated during motor imagery. Here we present preliminary results of a pilot study in which it was investigated, whether co-adaptive learning using machine-learning techniques could help subjects of Categories II and III to achieve successful feedback. In this setup, the session immediately started with BCI feedback. In the first three runs, a pretrained subject-independent classifier on simple features (band-power in alpha (8–15 Hz) and beta (16–32 Hz) frequency range in 3 Laplacian channels at C3, Cz, C4) was used and adapted (covariance matrix and pooled mean [1]) to the subject after each trial. For the subsequent three runs, a classifier was trained on a more complex composed band-power feature in a subject-specific narrow band from optimized Common Spatial Pattern (CSP [2]) filters and in six Laplacian channels. While CSP filters were static, the position of the Laplacians was updated based on a statistical criterion and the classifier was retrained on the combined CSP plus Laplacians feature to provide flexibility with respect to spatial location of modulated brain activity. Finally for the last two runs, a classifier was trained on CSP features, which have been calculated on runs 4–6. The pooled mean of the linear classifier was adapted after each trial [1]. Initially, we verified the novel experimental design with 6 subjects of Category I. Here, very good feedback performance was obtained within the first run after 20 to 40 trials of adaptation and further increased in subsequent runs. In the present pilot study, further two subjects of Category II and three of Category III took part. All these five subjects did not have control in the first three runs but they were able to gain it when the machine learning based techniques came into play in runs 4–6 (a jump from run 3 to run 4 in Cat. II, and a continuous increase in runs 4 to 6 in Cat. III; see Figure 1). For most subjects a further improvement could be obtained in runs 7 and 8. Summarizing, it was demonstrated that subjects categorized as BCI illiterates before could gain BCI control within one session. In particular, one subject who had no SMR idle rhythm in the beginning of the measurement could develop such with our feedback training; see Figure 1. This important finding gives rise to the development of neurofeedback training procedures that might help to cure BCI illiteracy.

Figure 1

Left : Grand average of feedback performance within each run (horizontal bars and dots for each group of 20 trials) for subjects of Category I ( N = 6), Category II ( N = 2) and Category III ( N = 3). An accuracy of 70% is assumed to be the threshold required for BCI applications. Right: For one subject of Category III spectra in channel CP3 and scalp topographies of band-power differences (signed r²-values) between the motor imagery conditions are compared between the beginning (runs 1+2) and the end (runs 7+8) of the experiment.