$30 off During Our Annual Pro Sale. View Details »

Brain-Computer Interfaces - Science Fiction or Reality?

Piotr Wittchen
November 20, 2018

Brain-Computer Interfaces - Science Fiction or Reality?

Brain-Computer Interface (BCI) is a direct communication pathway between the brain and an external device. It seems to be new technology, but it has its origins in 1924! Today, we have technologies, which lie on the border between biology and computer science like neuroprosthetics, electromyography and of course BCI. During the talk, we will start the journey through different brain imaging techniques, but in the end, we will focus on electroencephalography (EEG). BCI is no longer expensive technology available only in laboratories. Anyone can have an access to it. Such technology gives us the possibility to create human-computer interaction of the future. Additionally, we can develop communication interfaces for people with illnesses like LIS and it may be the only hope for them. During this talk, we will discuss not only a theory but also practical examples, available software and working pieces of code for the Android platform. If you care about transhumanism, you cannot neglect BCI.

This presentation was shown during the Øredev 2018 conference in Malmö, Sweden on 19th of November 2018.

Piotr Wittchen

November 20, 2018
Tweet

More Decks by Piotr Wittchen

Other Decks in Research

Transcript

  1. Brain-Computer Interfaces
    Science Fiction or Reality?
    Piotr Wittchen
    wittchen.io

    View Slide

  2. Evolution of communication tools
    2

    View Slide

  3. 3

    View Slide

  4. 4

    View Slide

  5. 5

    View Slide

  6. 6

    View Slide

  7. 7

    View Slide

  8. Leap motion
    8

    View Slide

  9. What’s next?
    9

    View Slide

  10. 10

    View Slide

  11. A Brain-Computer Interface (BCI) is a direct
    communication pathway between the brain and
    an external device.
    11

    View Slide

  12. 12

    View Slide

  13. In 1924 Hans Berger was the first
    to record human brain activity. He
    was able to identify oscillatory
    activity in the brain, such as the
    alpha wave (8–12 Hz).
    13

    View Slide

  14. What can we do with BCI?
    14

    View Slide

  15. Help people with Locked-In Syndrome (LIS)
    15

    View Slide

  16. Create tools for rehabilitation
    16

    View Slide

  17. Train meditation
    17

    View Slide

  18. 18
    Gathering secure information

    View Slide

  19. Develop new ways of communication
    19

    View Slide

  20. 20

    View Slide

  21. 21

    View Slide

  22. MindFlix
    https://www.youtube.com/watch?v=cyMqFEJSI_U
    22

    View Slide

  23. Art and Music
    Eunoia
    https://vimeo.com/65175792
    23

    View Slide

  24. Does anyone care about BCI today?
    24

    View Slide

  25. People talk about BCI at the conferences
    25

    View Slide

  26. Facebook is doing research regarding BCI
    26

    View Slide

  27. 27

    View Slide

  28. 28

    View Slide

  29. Similar technologies
    29

    View Slide

  30. BCI vs. Neuroprosthetics
    Neuroprosthetics connect
    the nervous system to a
    device, whereas BCIs
    connect the brain with a
    computer system
    30

    View Slide

  31. Electromyography (EMG)
    Technique for evaluating and recording
    the electrical activity produced by
    skeletal muscles.
    31

    View Slide

  32. Brain imaging techniques
    32

    View Slide

  33. Magnetoencephalography
    MEG is the technique for mapping brain activity by
    recording magnetic fields produced by electrical
    currents occurring naturally in the brain, using very
    sensitive magnetometers.
    33

    View Slide

  34. Positron Emission Tomography
    PET is the nuclear medical imaging
    technique that produces a 3D image
    or picture of functional processes in
    the body.
    34

    View Slide

  35. Single-Photon Emission Computed Tomography
    SPECT is the nuclear medicine tomographic
    imaging technique using gamma rays.
    35

    View Slide

  36. Functional Magnetic Resonance Imaging
    FMRI is the Magnetic Resonance Imaging procedure that
    measures brain activity by detecting associated changes in
    blood flow.
    36

    View Slide

  37. Functional Near Infrared
    Using FNIR, brain activity is measured through hemodynamic
    responses (blood movement) associated with neuron
    behavior. Mentioned responses can be detected with Infrared
    (IR) radiation.
    37

    View Slide

  38. Electroencephalography
    EEG is recording of the electrical activity
    of the brain along the scalp
    38

    View Slide

  39. How does EEG work?
    After placing electrodes on the skull, we can register changes
    of electric potential on the surface of the skin. These changes
    are caused by activity of neurons of the cortex.
    39

    View Slide

  40. 40

    View Slide

  41. 41
    Source: Nunez P., Srinivasan R.: Electroencephalogram, Scholarpedia, 2007

    View Slide

  42. 42
    EEG Voltage recorded
    from the electrode
    electrode
    amplitude
    frequency
    phase
    Frequency component
    time
    Source: Nunez P., Srinivasan R.: Electroencephalogram, Scholarpedia, 2007

    View Slide

  43. EEG rhythmic activity frequency bands
    ● Delta (up to 4 Hz) - in babies or slow-wave sleep by adults
    ● Theta (4-8 Hz) - young children or drowsiness by older children and adults,
    idling
    ● Alpha (8-13 Hz) - relaxed/reflecting, closing eyes, pathologically: coma
    ● Beta (>13-30 Hz) - active, busy, concentration or anxious thinking,
    pathologically: Benzodiazepines
    ● Gamma (30-100+ Hz) - perception that combines different senses (e.g. sound
    and sight), short-term memory matching of recognized objects
    ● Mu (8-13 Hz) - shows rest-state motor neurons
    43

    View Slide

  44. Selected affordable EEG hardware available today
    Muse Emotiv NeuroSky
    44

    View Slide

  45. NeuroSky MindWave Mobile
    ● Bluetooth v. 3.0 class 2 (10 meters range)
    ● Wireless pairing
    ● Static headset ID for pairing purposes
    ● 8-hours battery runtime (uses one AAA battery -
    1.5 V)
    ● iOS and Android support
    ● Measures raw brainwaves
    ● Processing and output of EEG power spectrum
    (Alpha, Beta, etc.)
    ● Processing and output proprietary eSense
    meter for mediation, attention and other future
    meters
    ● Blink detection
    ● Signal quality analysis
    45

    View Slide

  46. Applications
    46

    View Slide

  47. 47
    EEG Analyzer
    Google Play: https://play.google.com/store/apps/details?id=com.pwittchen.eeganalyzer

    View Slide

  48. EEG Controller
    48

    View Slide

  49. BrainWave Visualiser
    https://play.google.com/store/apps/details?id=com.neurosky.unitythinkgear
    49

    View Slide

  50. Brain Bits
    50
    https://github.com/dashersw/brain-bits

    View Slide

  51. Muse
    51
    https://play.google.com/store/apps/details?id=com.interaxon.muse

    View Slide

  52. How to write software for the NeuroSky?
    52

    View Slide

  53. Gathering data from NeuroSky with ThinkGear
    TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() {
    @Override public void handleMessage(Message msg) {
    super.handleMessage(msg);
    if(msg == TGDevice.MSG_STATE_CHANGE) {
    if(msg.arg1 == TGDevice.STATE_CONNECTED) {
    device.start();
    }
    // handle rest of the messages here...
    }
    }
    });
    }
    device.connect();
    53

    View Slide

  54. Gathering data from NeuroSky with ThinkGear
    TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() {
    @Override public void handleMessage(Message msg) {
    super.handleMessage(msg);
    if(msg == TGDevice.MSG_STATE_CHANGE) {
    if(msg.arg1 == TGDevice.STATE_CONNECTED) {
    device.start();
    }
    // handle rest of the messages here...
    }
    }
    });
    }
    device.connect();
    54

    View Slide

  55. Gathering data from NeuroSky with ThinkGear
    TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() {
    @Override public void handleMessage(Message msg) {
    super.handleMessage(msg);
    if(msg == TGDevice.MSG_STATE_CHANGE) {
    if(msg.arg1 == TGDevice.STATE_CONNECTED) {
    device.start();
    }
    // handle rest of the messages here...
    }
    }
    });
    }
    device.connect();
    55

    View Slide

  56. Gathering data from NeuroSky with ThinkGear
    TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() {
    @Override public void handleMessage(Message msg) {
    super.handleMessage(msg);
    if(msg == TGDevice.MSG_STATE_CHANGE) {
    if(msg.arg1 == TGDevice.STATE_CONNECTED) {
    device.start();
    }
    // handle rest of the messages here...
    }
    }
    });
    }
    device.connect();
    56

    View Slide

  57. Gathering data from NeuroSky with ThinkGear
    TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() {
    @Override public void handleMessage(Message msg) {
    super.handleMessage(msg);
    if(msg == TGDevice.MSG_STATE_CHANGE) {
    if(msg.arg1 == TGDevice.STATE_CONNECTED) {
    device.start();
    }
    // handle rest of the messages here...
    }
    }
    });
    }
    device.connect();
    57

    View Slide

  58. Gathering data from NeuroSky with ThinkGear
    TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() {
    @Override public void handleMessage(Message msg) {
    super.handleMessage(msg);
    if(msg == TGDevice.MSG_STATE_CHANGE) {
    if(msg.arg1 == TGDevice.STATE_CONNECTED) {
    device.start();
    }
    // handle rest of the messages here...
    }
    }
    });
    }
    device.connect();
    58

    View Slide

  59. Gathering data from NeuroSky with ThinkGear
    TGDevice device = new TGDevice(BluetoothAdapter.getDefaultAdapter(), new Handler() {
    @Override public void handleMessage(Message msg) {
    super.handleMessage(msg);
    if(msg == TGDevice.MSG_STATE_CHANGE) {
    if(msg.arg1 == TGDevice.STATE_CONNECTED) {
    device.start();
    }
    // handle rest of the messages here...
    }
    }
    });
    }
    device.connect();
    59

    View Slide

  60. Using NeuroSky Android SDK
    NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() {
    @Override public void onStateChange(State state) {
    // handle state change...
    }
    @Override public void onSignalChange(Signal signal) {
    // handle signal change...
    }
    @Override public void onBrainWavesChange(Set brainWaves) {
    // handle brain waves change...
    }
    });
    neuroSky.connect();
    60

    View Slide

  61. Using NeuroSky Android SDK
    NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() {
    @Override public void onStateChange(State state) {
    // handle state change...
    }
    @Override public void onSignalChange(Signal signal) {
    // handle signal change...
    }
    @Override public void onBrainWavesChange(Set brainWaves) {
    // handle brain waves change...
    }
    });
    neuroSky.connect();
    61

    View Slide

  62. Using NeuroSky Android SDK
    NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() {
    @Override public void onStateChange(State state) {
    // handle state change...
    }
    @Override public void onSignalChange(Signal signal) {
    // handle signal change...
    }
    @Override public void onBrainWavesChange(Set brainWaves) {
    // handle brain waves change...
    }
    });
    neuroSky.connect();
    62

    View Slide

  63. Using NeuroSky Android SDK
    NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() {
    @Override public void onStateChange(State state) {
    // handle state change...
    }
    @Override public void onSignalChange(Signal signal) {
    // handle signal change...
    }
    @Override public void onBrainWavesChange(Set brainWaves) {
    // handle brain waves change...
    }
    });
    neuroSky.connect();
    63

    View Slide

  64. Using NeuroSky Android SDK
    NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() {
    @Override public void onStateChange(State state) {
    // handle state change...
    }
    @Override public void onSignalChange(Signal signal) {
    // handle signal change...
    }
    @Override public void onBrainWavesChange(Set brainWaves) {
    // handle brain waves change...
    }
    });
    neuroSky.connect();
    64

    View Slide

  65. Using NeuroSky Android SDK
    NeuroSky neuroSky = new NeuroSky(new ExtendedDeviceMessageListener() {
    @Override public void onStateChange(State state) {
    // handle state change...
    }
    @Override public void onSignalChange(Signal signal) {
    // handle signal change...
    }
    @Override public void onBrainWavesChange(Set brainWaves) {
    // handle brain waves change...
    }
    });
    neuroSky.connect();
    65

    View Slide

  66. Using NeuroSky Android SDK with RxJava
    RxNeuroSky neuroSky = new RxNeuroSky();
    neuroSky
    .stream()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(brainEvent -> {
    handleStateChange(brainEvent.state());
    handleSignalChange(brainEvent.signal());
    handleBrainWavesChange(brainEvent.brainWaves());
    });
    neuroSky
    .connect()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(
    () -> showMessage("connecting..."),
    throwable -> { /* handle an error... */ }
    );
    66

    View Slide

  67. Using NeuroSky Android SDK with RxJava
    RxNeuroSky neuroSky = new RxNeuroSky();
    neuroSky
    .stream()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(brainEvent -> {
    handleStateChange(brainEvent.state());
    handleSignalChange(brainEvent.signal());
    handleBrainWavesChange(brainEvent.brainWaves());
    });
    neuroSky
    .connect()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(
    () -> showMessage("connecting..."),
    throwable -> { /* handle an error... */ }
    );
    67

    View Slide

  68. Using NeuroSky Android SDK with RxJava
    RxNeuroSky neuroSky = new RxNeuroSky();
    neuroSky
    .stream()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(brainEvent -> {
    handleStateChange(brainEvent.state());
    handleSignalChange(brainEvent.signal());
    handleBrainWavesChange(brainEvent.brainWaves());
    });
    neuroSky
    .connect()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(
    () -> showMessage("connecting..."),
    throwable -> { /* handle an error... */ }
    );
    68

    View Slide

  69. Using NeuroSky Android SDK with RxJava
    RxNeuroSky neuroSky = new RxNeuroSky();
    neuroSky
    .stream()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(brainEvent -> {
    handleStateChange(brainEvent.state());
    handleSignalChange(brainEvent.signal());
    handleBrainWavesChange(brainEvent.brainWaves());
    });
    neuroSky
    .connect()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(
    () -> showMessage("connecting..."),
    throwable -> { /* handle an error... */ }
    );
    69

    View Slide

  70. Using NeuroSky Android SDK with RxJava
    RxNeuroSky neuroSky = new RxNeuroSky();
    neuroSky
    .stream()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(brainEvent -> {
    handleStateChange(brainEvent.state());
    handleSignalChange(brainEvent.signal());
    handleBrainWavesChange(brainEvent.brainWaves());
    });
    neuroSky
    .connect()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(
    () -> showMessage("connecting..."),
    throwable -> { /* handle an error... */ }
    );
    70

    View Slide

  71. Using NeuroSky Android SDK with RxJava
    RxNeuroSky neuroSky = new RxNeuroSky();
    neuroSky
    .stream()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(brainEvent -> {
    handleStateChange(brainEvent.state());
    handleSignalChange(brainEvent.signal());
    handleBrainWavesChange(brainEvent.brainWaves());
    });
    neuroSky
    .connect()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(
    () -> showMessage("connecting..."),
    throwable -> { /* handle an error... */ }
    );
    71

    View Slide

  72. Using NeuroSky Android SDK with RxJava
    RxNeuroSky neuroSky = new RxNeuroSky();
    neuroSky
    .stream()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(brainEvent -> {
    handleStateChange(brainEvent.state());
    handleSignalChange(brainEvent.signal());
    handleBrainWavesChange(brainEvent.brainWaves());
    });
    neuroSky
    .connect()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(
    () -> showMessage("connecting..."),
    throwable -> { /* handle an error... */ }
    );
    72

    View Slide

  73. Using NeuroSky Android SDK with RxJava
    RxNeuroSky neuroSky = new RxNeuroSky();
    neuroSky
    .stream()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(brainEvent -> {
    handleStateChange(brainEvent.state());
    handleSignalChange(brainEvent.signal());
    handleBrainWavesChange(brainEvent.brainWaves());
    });
    neuroSky
    .connect()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(
    () -> showMessage("connecting..."),
    throwable -> { /* handle an error... */ }
    );
    73

    View Slide

  74. Using NeuroSky Android SDK with RxJava
    RxNeuroSky neuroSky = new RxNeuroSky();
    neuroSky
    .stream()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(brainEvent -> {
    handleStateChange(brainEvent.state());
    handleSignalChange(brainEvent.signal());
    handleBrainWavesChange(brainEvent.brainWaves());
    });
    neuroSky
    .connect()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(
    () -> showMessage("connecting..."),
    throwable -> { /* handle an error... */ }
    );
    74

    View Slide

  75. Using NeuroSky Android SDK with RxJava
    RxNeuroSky neuroSky = new RxNeuroSky();
    neuroSky
    .stream()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(brainEvent -> {
    handleStateChange(brainEvent.state());
    handleSignalChange(brainEvent.signal());
    handleBrainWavesChange(brainEvent.brainWaves());
    });
    neuroSky
    .connect()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(
    () -> showMessage("connecting..."),
    throwable -> { /* handle an error... */ }
    );
    75

    View Slide

  76. Using NeuroSky Android SDK with RxJava
    RxNeuroSky neuroSky = new RxNeuroSky();
    neuroSky
    .stream()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(brainEvent -> {
    handleStateChange(brainEvent.state());
    handleSignalChange(brainEvent.signal());
    handleBrainWavesChange(brainEvent.brainWaves());
    });
    neuroSky
    .connect()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(
    () -> showMessage("connecting..."),
    throwable -> { /* handle an error... */ }
    );
    76

    View Slide

  77. Using NeuroSky Android SDK with RxJava
    RxNeuroSky neuroSky = new RxNeuroSky();
    neuroSky
    .stream()
    .subscribeOn(Schedulers.computation())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(brainEvent -> {
    handleStateChange(brainEvent.state());
    handleSignalChange(brainEvent.signal());
    handleBrainWavesChange(brainEvent.brainWaves());
    });
    neuroSky
    .connect()
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe(
    () -> showMessage("connecting..."),
    throwable -> { /* handle an error... */ }
    );
    77

    View Slide

  78. 78

    View Slide

  79. Let’s see the live demo!
    What can go wrong?
    79

    View Slide

  80. Source code, examples and links
    https://github.com/pwittchen/neurosky-android-sdk (brand new project)
    https://github.com/pwittchen/EEGReader (old project using ThinkGear only)
    http://developer.neurosky.com/ (official website of the NeuroSky)
    http://developer.choosemuse.com/sdk (official Muse developer website)
    http://wittchen.io/tags/bci/ (a few articles about BCI on my blog)
    80

    View Slide

  81. Summary
    ● Direct communication between human brain and the computer is no longer
    science-fiction
    ● We have affordable devices using EEG technology today
    ● We are able to write Brain-Computer Interfaces without having scientific
    knowledge about it
    ● We can develop new ways of communication with BCI
    ● We can create data acquisition tools for doctors and scientists
    ● We can develop tools supporting meditation
    ● We can develop tools for rehabilitation and improving concentration
    ● We can create communication tools for people with illnesses like LIS
    ● BCI can be improved with Machine Learning techniques
    81

    View Slide

  82. Brain-Computer Interfaces
    Science Fiction or Reality?
    Piotr Wittchen
    wittchen.io
    get presentation slides at:
    wittchen.io/talks
    Thank you for attention! Questions?

    View Slide