Save 37% off PRO during our Black Friday Sale! »

Accessible Software with Brain Computer Interface and Open Data

Accessible Software with Brain Computer Interface and Open Data

This slide deck was used for Mozilla Festival 2018 for the following session-


Bhoomika Agarwal

October 27, 2018


  1. Bhoomika Agarwal Abhiram Ravikumar Mozilla Festival 2018 | October 27,

    2018 Accessible Software with Brain Computer Interface and Open Data Licence: CC BY-SA 4.0 International. You are free to share and adapt.
  2. • Bhoomika Agarwal (@myriadbhoom) ◦ Mozillian ◦ Data Science Enthusiast

    ◦ LinkedIn Learning Instructor ◦ Poet ◦ Developer at SAP Labs, India • Abhiram Ravikumar (@abhi12ravi) ◦ Mozilla Tech Speaker ◦ Data Science enthusiast ◦ LinkedIn Learning Instructor ◦ Developer at SAP Labs, India About Us 2
  3. Tweet at Us! #bci #DiversityandInclusion #neurotech @MozillaFestival @mozilla Any questions/feedback?

    Tweet at us: @myriadbhoom @abhi12ravi 3
  4. Objective 4 Understanding key narratives • What is disability? •

    Tools to overcome disability • A sneak peek into the life of a person with disability Brain Computer Interface • What is BCI • Tech behind BCI • Our solution and the impact it will create How can you contribute? • Open source project (BCI Media Player) • Issues on GitHub
  5. Credits: iStockPhoto 5 Activity 1. Put away your devices 2.

    Close your eyes 3. Imagine you cannot move your hands or legs or open your eyes 4. Stay still for 2 minutes until you hear a beep
  6. What is disability? • A physical or mental condition that

    limits a person's movements, senses, or activities. • A disadvantage or handicap, especially one imposed or recognized by the law. 6
  7. Cannot fly? 7 What is disability? Use an aeroplane! Credits:

    Shutterstock photos Credits: VectorStock photos
  8. What is disability? Cannot see in the dark? Use night-vision

    goggles! 8 Credits: Credits: NightVisionGuys channel on YouTube
  9. None
  10. Persons with Disabilities • International Symbol of Access • What’s

    an impairment? ◦ Cognitive ◦ Intellectual ◦ Mental ◦ Physical ◦ Sensory ◦ Combination of the above • Problems they face ◦ Access to technology (limited) ◦ Access to web (cut off) ◦ Limited forums to express ◦ Many more.. 10 Credits:
  11. Why is accessibility and inclusion important? Accessibility means having a

    place, environment, or event that is set up from the start to be accessible to all individuals Why inclusion matters? • Narratives are diverse and unique • Different perspectives, diversity and thought processes • Demonstrate social responsibility 11 Credits: I Love You in American Sign Language
  12. What is Brain Computer Interface (BCI)?

  13. Who’s this? Courtesy:

  14. What does a PWD’s day look like? 14 Credits: MRPF

    Channel on YouTube
  15. Ultracortex Mark IV EEG headsets (Credits:

  16. What is BCI? 16 • A Brain Computer Interface (BCI)

    is a medium through which a person uses his/her brain to control the machine- which can be a computer, wheelchair, robot, assistive or an alternative communication device • BCIs strive to decode brain signals into control commands for severely handicapped people with no means of muscular control
  17. BCI devices 17 Credits: Neurosky Mindwave Credits: Human Media Interaction-

  18. Types of Brain waves 18 Delta waves 0.1 - 4

    Hz Sleep and empathy Theta waves 4 - 8 Hz Daydreaming, meditation, creative ideas Alpha waves 8 - 12Hz Alertness with eyes closed, relaxed positive feelings Beta waves 12 - 30 Hz Alertness with eyes open Gamma waves >30 Hz Cognitive and motor functions
  19. General architecture of BCI 19 Data acquisition system Pre-processing Feature

    Extraction Feature Classification Device Commands Robotic arm Virtual Reality Wheelchair Gaming
  20. BCI media player

  21. 21 Motivation behind the BCI media player • Design a

    system that allows people with severe motor disability to communicate with computers in real time. • This system should be able to scale up to multiple dimensions effectively. • The developed system must be cost effective and easy to use. • The system should not require much training of the user before usage
  22. How? 22 • Motor imagery with Mu waves (8-12Hz in

    the sensorimotor cortex) • Establish a combination of binary digits • Left hand movement maps to a ‘0’ and right hand movement maps to a ‘1’ The motor homunculus on the precentral gyrus. The motor cortex can be seen in the lateral view of the cerebral cortex. (From Netter, 1974, p. 68)
  23. 23 Architecture of the BCI media player BCI headset Data

    acquisition Data Bandpass filter 8-12Hz CSP Dimension reduction LDA Classification BCI Media Player Application
  24. BCI media player 24 Demo

  25. Results • Processing accuracy of 74% was achieved • Created

    a media player that can be operated using inputs from BCI using offline data sets and string inputs • Can be used as a proof of concept for software applications with multiple dimensions and functions • Efficient means of communication with the computer system 25
  26. What next? • Integrate with the OpenBCI hardware to complete

    an end-to-end functional demo with real-time data • Build a wrapper that can run on top of any OS so as to control any software • Integrate with browser to enable browsing a simple site • Improve accuracy of classification • Extend the paradigm to other software with multiple functions 26
  27. Thank You! Questions? #MozillaFestival #OpenBCI @abhi12ravi @myriadbhoom