Tweet
Tweet

Slide 1

Slide 1 text

Brain-Computer Interfaces Science Fiction or Reality? Piotr Wittchen wittchen.io

Slide 2

Slide 2 text

Evolution of communication tools 2

Slide 3

Slide 3 text

3

Slide 4

Slide 4 text

4

Slide 5

Slide 5 text

5

Slide 6

Slide 6 text

6

Slide 7

Slide 7 text

7

Slide 8

Slide 8 text

Leap motion 8

Slide 9

Slide 9 text

What’s next? 9

Slide 10

Slide 10 text

10

Slide 11

Slide 11 text

A Brain-Computer Interface (BCI) is a direct communication pathway between the brain and an external device. 11

Slide 12

Slide 12 text

12

Slide 13

Slide 13 text

In 1924 Hans Berger was the first to record human brain activity. He was able to identify oscillatory activity in the brain, such as the alpha wave (8–12 Hz). 13

Slide 14

Slide 14 text

What can we do with BCI? 14

Slide 15

Slide 15 text

Help people with Locked-In Syndrome (LIS) 15

Slide 16

Slide 16 text

Create tools for rehabilitation 16

Slide 17

Slide 17 text

Train meditation 17

Slide 18

Slide 18 text

18 Gathering secure information

Slide 19

Slide 19 text

Develop new ways of communication 19

Slide 20

Slide 20 text

20

Slide 21

Slide 21 text

21

Slide 22

Slide 22 text

MindFlix https://www.youtube.com/watch?v=cyMqFEJSI_U 22

Slide 23

Slide 23 text

Art and Music Eunoia https://vimeo.com/65175792 23

Slide 24

Slide 24 text

Does anyone care about BCI today? 24

Slide 25

Slide 25 text

People talk about BCI at the conferences 25

Slide 26

Slide 26 text

Facebook is doing research regarding BCI 26

Slide 27

Slide 27 text

27

Slide 28

Slide 28 text

28

Slide 29

Slide 29 text

Similar technologies 29

Slide 30

Slide 30 text

BCI vs. Neuroprosthetics Neuroprosthetics connect the nervous system to a device, whereas BCIs connect the brain with a computer system 30

Slide 31

Slide 31 text

Electromyography (EMG) Technique for evaluating and recording the electrical activity produced by skeletal muscles. 31

Slide 32

Slide 32 text

Brain imaging techniques 32

Slide 33

Slide 33 text

Magnetoencephalography MEG is the technique for mapping brain activity by recording magnetic fields produced by electrical currents occurring naturally in the brain, using very sensitive magnetometers. 33

Slide 34

Slide 34 text

Positron Emission Tomography PET is the nuclear medical imaging technique that produces a 3D image or picture of functional processes in the body. 34

Slide 35

Slide 35 text

Single-Photon Emission Computed Tomography SPECT is the nuclear medicine tomographic imaging technique using gamma rays. 35

Slide 36

Slide 36 text

Functional Magnetic Resonance Imaging FMRI is the Magnetic Resonance Imaging procedure that measures brain activity by detecting associated changes in blood flow. 36

Slide 37

Slide 37 text

Functional Near Infrared Using FNIR, brain activity is measured through hemodynamic responses (blood movement) associated with neuron behavior. Mentioned responses can be detected with Infrared (IR) radiation. 37

Slide 38

Slide 38 text

Electroencephalography EEG is recording of the electrical activity of the brain along the scalp 38

Slide 39

Slide 39 text

How does EEG work? After placing electrodes on the skull, we can register changes of electric potential on the surface of the skin. These changes are caused by activity of neurons of the cortex. 39

Slide 40

Slide 40 text

40

Slide 41

Slide 41 text

41 Source: Nunez P., Srinivasan R.: Electroencephalogram, Scholarpedia, 2007

Slide 42

Slide 42 text

42 EEG Voltage recorded from the electrode electrode amplitude frequency phase Frequency component time Source: Nunez P., Srinivasan R.: Electroencephalogram, Scholarpedia, 2007

Slide 43

Slide 43 text

EEG rhythmic activity frequency bands ● Delta (up to 4 Hz) - in babies or slow-wave sleep by adults ● Theta (4-8 Hz) - young children or drowsiness by older children and adults, idling ● Alpha (8-13 Hz) - relaxed/reflecting, closing eyes, pathologically: coma ● Beta (>13-30 Hz) - active, busy, concentration or anxious thinking, pathologically: Benzodiazepines ● Gamma (30-100+ Hz) - perception that combines different senses (e.g. sound and sight), short-term memory matching of recognized objects ● Mu (8-13 Hz) - shows rest-state motor neurons 43

Slide 44

Slide 44 text

Selected affordable EEG hardware available today Muse Emotiv NeuroSky 44

Slide 45

Slide 45 text

NeuroSky MindWave Mobile ● Bluetooth v. 3.0 class 2 (10 meters range) ● Wireless pairing ● Static headset ID for pairing purposes ● 8-hours battery runtime (uses one AAA battery - 1.5 V) ● iOS and Android support ● Measures raw brainwaves ● Processing and output of EEG power spectrum (Alpha, Beta, etc.) ● Processing and output proprietary eSense meter for mediation, attention and other future meters ● Blink detection ● Signal quality analysis 45

Slide 46

Slide 46 text

Applications 46

Slide 47

Slide 47 text

47 EEG Analyzer Google Play: https://play.google.com/store/apps/details?id=com.pwittchen.eeganalyzer

Slide 48

Slide 48 text

EEG Controller 48

Slide 49

Slide 49 text

BrainWave Visualiser https://play.google.com/store/apps/details?id=com.neurosky.unitythinkgear 49

Slide 50

Slide 50 text

Brain Bits 50 https://github.com/dashersw/brain-bits

Slide 51

Slide 51 text

Muse 51 https://play.google.com/store/apps/details?id=com.interaxon.muse

Slide 52

Slide 52 text

How to write software for the NeuroSky? 52

Slide 53

Slide 53 text

Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 53

Slide 54

Slide 54 text

Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 54

Slide 55

Slide 55 text

Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 55

Slide 56

Slide 56 text

Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 56

Slide 57

Slide 57 text

Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 57

Slide 58

Slide 58 text

Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 58

Slide 59

Slide 59 text

Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 59

Slide 60

Slide 60 text

Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 60

Slide 61

Slide 61 text

Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 61

Slide 62

Slide 62 text

Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 62

Slide 63

Slide 63 text

Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 63

Slide 64

Slide 64 text

Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 64

Slide 65

Slide 65 text

Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 65

Slide 66

Slide 66 text

Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 66

Slide 67

Slide 67 text

Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 67

Slide 68

Slide 68 text

Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 68

Slide 69

Slide 69 text

Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 69

Slide 70

Slide 70 text

Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 70

Slide 71

Slide 71 text

Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 71

Slide 72

Slide 72 text

Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 72

Slide 73

Slide 73 text

Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 73

Slide 74

Slide 74 text

Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 74

Slide 75

Slide 75 text

Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 75

Slide 76

Slide 76 text

Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 76

Slide 77

Slide 77 text

Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 77

Slide 78

Slide 78 text

78

Slide 79

Slide 79 text

Let’s see the live demo! What can go wrong? 79

Slide 80

Slide 80 text

Source code, examples and links https://github.com/pwittchen/neurosky-android-sdk (brand new project) https://github.com/pwittchen/EEGReader (old project using ThinkGear only) http://developer.neurosky.com/ (official website of the NeuroSky) http://developer.choosemuse.com/sdk (official Muse developer website) http://wittchen.io/tags/bci/ (a few articles about BCI on my blog) 80

Slide 81

Slide 81 text

Summary ● Direct communication between human brain and the computer is no longer science-fiction ● We have affordable devices using EEG technology today ● We are able to write Brain-Computer Interfaces without having scientific knowledge about it ● We can develop new ways of communication with BCI ● We can create data acquisition tools for doctors and scientists ● We can develop tools supporting meditation ● We can develop tools for rehabilitation and improving concentration ● We can create communication tools for people with illnesses like LIS ● BCI can be improved with Machine Learning techniques 81

Slide 82

Slide 82 text

Brain-Computer Interfaces Science Fiction or Reality? Piotr Wittchen wittchen.io get presentation slides at: wittchen.io/talks Thank you for attention! Questions?