How we make BP work in simple terms.

Interacting with BP

BP relies on a very simple paradigm that detect attention by comparing brain responses triggered during visual stimulation. For this, Brain Painting records the cerebral activity from scalp electrodes and provides control and feedback over an interface. This system is referred to as a brain-computer Interface (BCI). It only requires two monitors, an electroencephalogram (EEG) and the user's attention to use Brain Painting.

BrainPainting Setup

Simplified setup of Brain Painting. A: the end-user with the EEG cap. B: EEG amplifier. C: stimulation monitor. D: canvas monitor

Brain Painting matrices

This specific setup was initially used to spell letters to form words and sentences. In BP, we replaced letters by actions on a virtual canvas that result in paintings. By concentrating for a few seconds on the target, the user can send selections to the canvas. Meanwhile the lines and columns of the matrix are flashing (5 stimulations per second)

Brain Painting matrix for V1 with faces

BP version 1 matrix with face flashes (text within the matrix is in german).

Matrix Flashing

A standard spelling matrix during flashing

Event related potentials

Flash stimulations elicit distinctive brain responses called event-related potentials (ERPs), that are discriminated using EEG. To elicit stronger responses, we instead use a face overlay to trigger additional face-evoked potentials (Kaufmann et al., 2011).

plot of P300 evoked potential

Plotted difference between target and non-target averaged ERPs. A positivity in the amplitude is visible at about 300ms, peaking at 400ms.

The human cortex show different ERPs whether the event (the flash) occurs on the chosen target, or not. The so-called "P300 ERP" is a positive activation observable between 200ms to 500ms.

EEG acquisition

10-20 model of the EEG cap with 8 electrodes

Electrodes locations for BP, according to the "10-20" system representation.

We use a g.usbamp (gtec.at) with g.gammabox and 8 active electrodes, but most amplifiers would do the job as long as their electrodes are properly positionned. The most important ones are placed on the Frontal-Central-Parietal and Occipital midline on which the P300 is most prominent. Plus some more electrodes covering the parieto-occipital area, related to vision.

EEG signals

Five seconds of filtered EEG signals acquired while using BP at home

Signal processing and classification

During signal processing, we acquire a time resolution between 2000Hz to 256Hz. After removing high (>60Hz), and low (.01Hz) frequencies from the EEG (and the dreadly 50Hz electromagnetic noise that our power lines emit) the signal is both human and machine readable.

Two dimentional linear classifier

Projection of a linear classifier for two classes (i.e. target and non-target). Unsurprisingy, there is an overlap between those classes, leading to a chance of classification errors.

During calibration, the EEG signals over time are cut into small pieces, called features. They are needed to train a classifier using stepwise linear discriminant analysis (SWLDA). The most helpful electrodes and timings are kept and weighted, returning a linear classifier.

As soon as the linear classifier is trained and returns a good performance, you can start using BP. In the last version we included an fast calibration process that allow to calibrate or check the performance within 10 minutes. The calibration process is integrated in the interface, so that any operator can easily do it within a few clicks.

Disclaimer: This section contains technical information about how BP works. Those were made readable for a broad audience. If you look for exactness, please refer to the publication list. If you cannot access one them, don't hesitate to contact us to request an author's version. Alternatively, the wikipedia page for BCIs might give you a larger picture of the topic.