Brain-Computer Interfaces - Science Fiction or Reality?

Brain-Computer Interface (BCI) is a direct communication pathway between the brain and an external device. It seems to be new technology, but it has its origins in 1924! Today, we have technologies, which lie on the border between biology and computer science like neuroprosthetics, electromyography and of course BCI. During the talk, we will start the journey through different brain imaging techniques, but in the end, we will focus on electroencephalography (EEG). BCI is no longer expensive technology available only in laboratories. Anyone can have an access to it. Such technology gives us the possibility to create human-computer interaction of the future. Additionally, we can develop communication interfaces for people with illnesses like LIS and it may be the only hope for them. During this talk, we will discuss not only a theory but also practical examples, available software and working pieces of code for the Android platform. If you care about transhumanism, you cannot neglect BCI.

This presentation was shown during the Øredev 2018 conference in Malmö, Sweden on 19th of November 2018.

Transcript

1. Brain-Computer Interfaces Science Fiction or Reality? Piotr Wittchen wittchen.io
2. Evolution of communication tools 2
8. Leap motion 8
9. What’s next? 9
11. A Brain-Computer Interface (BCI) is a direct communication pathway between the brain and an external device. 11
13. In 1924 Hans Berger was the first to record human brain activity. He was able to identify oscillatory activity in the brain, such as the alpha wave (8–12 Hz). 13
14. What can we do with BCI? 14
15. Help people with Locked-In Syndrome (LIS) 15
16. Create tools for rehabilitation 16
17. Train meditation 17
18. 18 Gathering secure information
19. Develop new ways of communication 19
22. MindFlix https://www.youtube.com/watch?v=cyMqFEJSI_U 22
23. Art and Music Eunoia https://vimeo.com/65175792 23
24. Does anyone care about BCI today? 24
25. People talk about BCI at the conferences 25
26. Facebook is doing research regarding BCI 26
29. Similar technologies 29
30. BCI vs. Neuroprosthetics Neuroprosthetics connect the nervous system to a device, whereas BCIs connect the brain with a computer system 30
31. Electromyography (EMG) Technique for evaluating and recording the electrical activity produced by skeletal muscles. 31
32. Brain imaging techniques 32
33. Magnetoencephalography MEG is the technique for mapping brain activity by recording magnetic fields produced by electrical currents occurring naturally in the brain, using very sensitive magnetometers. 33
34. Positron Emission Tomography PET is the nuclear medical imaging technique that produces a 3D image or picture of functional processes in the body. 34
35. Single-Photon Emission Computed Tomography SPECT is the nuclear medicine tomographic imaging technique using gamma rays. 35
36. Functional Magnetic Resonance Imaging FMRI is the Magnetic Resonance Imaging procedure that measures brain activity by detecting associated changes in blood flow. 36
37. Functional Near Infrared Using FNIR, brain activity is measured through hemodynamic responses (blood movement) associated with neuron behavior. Mentioned responses can be detected with Infrared (IR) radiation. 37
38. Electroencephalography EEG is recording of the electrical activity of the brain along the scalp 38
39. How does EEG work? After placing electrodes on the skull, we can register changes of electric potential on the surface of the skin. These changes are caused by activity of neurons of the cortex. 39
41. 41 Source: Nunez P., Srinivasan R.: Electroencephalogram, Scholarpedia, 2007
42. 42 EEG Voltage recorded from the electrode electrode amplitude frequency phase Frequency component time Source: Nunez P., Srinivasan R.: Electroencephalogram, Scholarpedia, 2007
43. EEG rhythmic activity frequency bands • Delta (up to 4 Hz) - in babies or slow-wave sleep by adults • Theta (4-8 Hz) - young children or drowsiness by older children and adults, idling • Alpha (8-13 Hz) - relaxed/reflecting, closing eyes, pathologically: coma • Beta (>13-30 Hz) - active, busy, concentration or anxious thinking, pathologically: Benzodiazepines • Gamma (30-100+ Hz) - perception that combines different senses (e.g. sound and sight), short-term memory matching of recognized objects • Mu (8-13 Hz) - shows rest-state motor neurons 43
44. Selected affordable EEG hardware available today Muse Emotiv NeuroSky 44
45. NeuroSky MindWave Mobile • Bluetooth v. 3.0 class 2 (10 meters range) • Wireless pairing • Static headset ID for pairing purposes • 8-hours battery runtime (uses one AAA battery - 1.5 V) • iOS and Android support • Measures raw brainwaves • Processing and output of EEG power spectrum (Alpha, Beta, etc.) • Processing and output proprietary eSense meter for mediation, attention and other future meters • Blink detection • Signal quality analysis 45
46. Applications 46
47. 47 EEG Analyzer Google Play: https://play.google.com/store/apps/details?id=com.pwittchen.eeganalyzer
48. EEG Controller 48
49. BrainWave Visualiser https://play.google.com/store/apps/details?id=com.neurosky.unitythinkgear 49
50. Brain Bits 50 https://github.com/dashersw/brain-bits
51. Muse 51 https://play.google.com/store/apps/details?id=com.interaxon.muse
52. How to write software for the NeuroSky? 52
53. Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 53
54. Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 54
55. Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 55
56. Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 56
57. Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 57
58. Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 58
59. Gathering data from NeuroSky with ThinkGear TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 59
60. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 60
61. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 61
62. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 62
63. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 63
64. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 64
65. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 65
66. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 66
67. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 67
68. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 68
69. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 69
70. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 70
71. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 71
72. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 72
73. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 73
74. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 74
75. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 75
76. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 76
77. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 77
79. Let’s see the live demo! What can go wrong? 79
80. Source code, examples and links https://github.com/pwittchen/neurosky-android-sdk (brand new project) https://github.com/pwittchen/EEGReader (old project using ThinkGear only) http://developer.neurosky.com/ (official website of the NeuroSky) http://developer.choosemuse.com/sdk (official Muse developer website) http://wittchen.io/tags/bci/ (a few articles about BCI on my blog) 80
81. Summary • Direct communication between human brain and the computer is no longer science-fiction • We have affordable devices using EEG technology today • We are able to write Brain-Computer Interfaces without having scientific knowledge about it • We can develop new ways of communication with BCI • We can create data acquisition tools for doctors and scientists • We can develop tools supporting meditation • We can develop tools for rehabilitation and improving concentration • We can create communication tools for people with illnesses like LIS • BCI can be improved with Machine Learning techniques 81
82. Brain-Computer Interfaces Science Fiction or Reality? Piotr Wittchen wittchen.io get presentation slides at: wittchen.io/talks Thank you for attention! Questions?