Brain-Computer Interfaces - Science Fiction or Reality?

E8bb35e5d8c868e7dffcd9859248d6f1?s=47 Piotr Wittchen
November 20, 2018

Brain-Computer Interfaces - Science Fiction or Reality?

Brain-Computer Interface (BCI) is a direct communication pathway between the brain and an external device. It seems to be new technology, but it has its origins in 1924! Today, we have technologies, which lie on the border between biology and computer science like neuroprosthetics, electromyography and of course BCI. During the talk, we will start the journey through different brain imaging techniques, but in the end, we will focus on electroencephalography (EEG). BCI is no longer expensive technology available only in laboratories. Anyone can have an access to it. Such technology gives us the possibility to create human-computer interaction of the future. Additionally, we can develop communication interfaces for people with illnesses like LIS and it may be the only hope for them. During this talk, we will discuss not only a theory but also practical examples, available software and working pieces of code for the Android platform. If you care about transhumanism, you cannot neglect BCI.

This presentation was shown during the Øredev 2018 conference in Malmö, Sweden on 19th of November 2018.

E8bb35e5d8c868e7dffcd9859248d6f1?s=128

Piotr Wittchen

November 20, 2018
Tweet

Transcript

  1. Brain-Computer Interfaces Science Fiction or Reality? Piotr Wittchen wittchen.io

  2. Evolution of communication tools 2

  3. 3

  4. 4

  5. 5

  6. 6

  7. 7

  8. Leap motion 8

  9. What’s next? 9

  10. 10

  11. A Brain-Computer Interface (BCI) is a direct communication pathway between

    the brain and an external device. 11
  12. 12

  13. In 1924 Hans Berger was the first to record human

    brain activity. He was able to identify oscillatory activity in the brain, such as the alpha wave (8–12 Hz). 13
  14. What can we do with BCI? 14

  15. Help people with Locked-In Syndrome (LIS) 15

  16. Create tools for rehabilitation 16

  17. Train meditation 17

  18. 18 Gathering secure information

  19. Develop new ways of communication 19

  20. 20

  21. 21

  22. MindFlix https://www.youtube.com/watch?v=cyMqFEJSI_U 22

  23. Art and Music Eunoia https://vimeo.com/65175792 23

  24. Does anyone care about BCI today? 24

  25. People talk about BCI at the conferences 25

  26. Facebook is doing research regarding BCI 26

  27. 27

  28. 28

  29. Similar technologies 29

  30. BCI vs. Neuroprosthetics Neuroprosthetics connect the nervous system to a

    device, whereas BCIs connect the brain with a computer system 30
  31. Electromyography (EMG) Technique for evaluating and recording the electrical activity

    produced by skeletal muscles. 31
  32. Brain imaging techniques 32

  33. Magnetoencephalography MEG is the technique for mapping brain activity by

    recording magnetic fields produced by electrical currents occurring naturally in the brain, using very sensitive magnetometers. 33
  34. Positron Emission Tomography PET is the nuclear medical imaging technique

    that produces a 3D image or picture of functional processes in the body. 34
  35. Single-Photon Emission Computed Tomography SPECT is the nuclear medicine tomographic

    imaging technique using gamma rays. 35
  36. Functional Magnetic Resonance Imaging FMRI is the Magnetic Resonance Imaging

    procedure that measures brain activity by detecting associated changes in blood flow. 36
  37. Functional Near Infrared Using FNIR, brain activity is measured through

    hemodynamic responses (blood movement) associated with neuron behavior. Mentioned responses can be detected with Infrared (IR) radiation. 37
  38. Electroencephalography EEG is recording of the electrical activity of the

    brain along the scalp 38
  39. How does EEG work? After placing electrodes on the skull,

    we can register changes of electric potential on the surface of the skin. These changes are caused by activity of neurons of the cortex. 39
  40. 40

  41. 41 Source: Nunez P., Srinivasan R.: Electroencephalogram, Scholarpedia, 2007

  42. 42 EEG Voltage recorded from the electrode electrode amplitude frequency

    phase Frequency component time Source: Nunez P., Srinivasan R.: Electroencephalogram, Scholarpedia, 2007
  43. EEG rhythmic activity frequency bands • Delta (up to 4

    Hz) - in babies or slow-wave sleep by adults • Theta (4-8 Hz) - young children or drowsiness by older children and adults, idling • Alpha (8-13 Hz) - relaxed/reflecting, closing eyes, pathologically: coma • Beta (>13-30 Hz) - active, busy, concentration or anxious thinking, pathologically: Benzodiazepines • Gamma (30-100+ Hz) - perception that combines different senses (e.g. sound and sight), short-term memory matching of recognized objects • Mu (8-13 Hz) - shows rest-state motor neurons 43
  44. Selected affordable EEG hardware available today Muse Emotiv NeuroSky 44

  45. NeuroSky MindWave Mobile • Bluetooth v. 3.0 class 2 (10

    meters range) • Wireless pairing • Static headset ID for pairing purposes • 8-hours battery runtime (uses one AAA battery - 1.5 V) • iOS and Android support • Measures raw brainwaves • Processing and output of EEG power spectrum (Alpha, Beta, etc.) • Processing and output proprietary eSense meter for mediation, attention and other future meters • Blink detection • Signal quality analysis 45
  46. Applications 46

  47. 47 EEG Analyzer Google Play: https://play.google.com/store/apps/details?id=com.pwittchen.eeganalyzer

  48. EEG Controller 48

  49. BrainWave Visualiser https://play.google.com/store/apps/details?id=com.neurosky.unitythinkgear 49

  50. Brain Bits 50 https://github.com/dashersw/brain-bits

  51. Muse 51 https://play.google.com/store/apps/details?id=com.interaxon.muse

  52. How to write software for the NeuroSky? 52

  53. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 53
  54. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 54
  55. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 55
  56. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 56
  57. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 57
  58. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 58
  59. Gathering data from NeuroSky with ThinkGear TGDevice device = new

    TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() { @Override public void handleMessage(Message msg) { super.handleMessage(msg); if(msg == TGDevice.MSG_STATE_CHANGE) { if(msg.arg1 == TGDevice.STATE_CONNECTED) { device.start(); } // handle rest of the messages here... } } }); } device.connect(); 59
  60. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener()

    { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 60
  61. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener()

    { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 61
  62. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener()

    { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 62
  63. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener()

    { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 63
  64. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener()

    { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 64
  65. Using NeuroSky Android SDK NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener()

    { @Override public void onStateChange(State state) { // handle state change... } @Override public void onSignalChange(Signal signal) { // handle signal change... } @Override public void onBrainWavesChange(Set<BrainWave> brainWaves) { // handle brain waves change... } }); neuroSky.connect(); 65
  66. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 66
  67. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 67
  68. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 68
  69. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 69
  70. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 70
  71. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 71
  72. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 72
  73. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 73
  74. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 74
  75. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 75
  76. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 76
  77. Using NeuroSky Android SDK with RxJava RxNeuroSky neuroSky = new

    RxNeuroSky(); neuroSky .stream() .subscribeOn(Schedulers.computation()) .observeOn(AndroidSchedulers.mainThread()) .subscribe(brainEvent -> { handleStateChange(brainEvent.state()); handleSignalChange(brainEvent.signal()); handleBrainWavesChange(brainEvent.brainWaves()); }); neuroSky .connect() .subscribeOn(Schedulers.io()) .observeOn(AndroidSchedulers.mainThread()) .subscribe( () -> showMessage("connecting..."), throwable -> { /* handle an error... */ } ); 77
  78. 78

  79. Let’s see the live demo! What can go wrong? 79

  80. Source code, examples and links https://github.com/pwittchen/neurosky-android-sdk (brand new project) https://github.com/pwittchen/EEGReader

    (old project using ThinkGear only) http://developer.neurosky.com/ (official website of the NeuroSky) http://developer.choosemuse.com/sdk (official Muse developer website) http://wittchen.io/tags/bci/ (a few articles about BCI on my blog) 80
  81. Summary • Direct communication between human brain and the computer

    is no longer science-fiction • We have affordable devices using EEG technology today • We are able to write Brain-Computer Interfaces without having scientific knowledge about it • We can develop new ways of communication with BCI • We can create data acquisition tools for doctors and scientists • We can develop tools supporting meditation • We can develop tools for rehabilitation and improving concentration • We can create communication tools for people with illnesses like LIS • BCI can be improved with Machine Learning techniques 81
  82. Brain-Computer Interfaces Science Fiction or Reality? Piotr Wittchen wittchen.io get

    presentation slides at: wittchen.io/talks Thank you for attention! Questions?