Replacing the computer mouse

Abstract: In a few months the computer mouse will be half-a-century-old. It is known to have many drawbacks, the main ones being: loss of productivity due to constant switching between keyboard and mouse, health issues such as RSI, medical impossibility to use the mouse e.g. broken or amputated arm and unnatural human-computer interface like the keyboard. However almost everybody still uses a computer mouse nowadays.

In this presentation, we explore computer mouse alternatives. Our research shows that moving the mouse cursor can be done efficiently with the SmartNav device and mouse clicks can be emulated in many complementary ways. We believe that computer users can increase their productivity and their health by using those alternatives. There are a few exceptions such as advanced users of graphics editing programs or FPS gamers, who will still be more efficient using a computer mouse.

This article is voluntary short and not overly technical, our main motivation being to make the readers aware of these solutions and their efficiencies. Details can be found in the appendices and by following the URLs and references. The primarily intended readers are computer scientists, people with RSI, physicians and interface pioneers.

http://francky.me/publications.php#mouse2012

Transcript

1. > Replacing the computer mouse September 15th, 2012 @2012 Boston Accessibility Conference (http://a11y-bos.org/); Microsoft NERD Center. Franck.Dernoncourt@gmail.com http://francky.me
2. 2012-09-15 Franck Dernoncourt - http://francky.me 2 Franck Dernoncourt 2 0) Foreword
3. 2012-09-15 Franck Dernoncourt - http://francky.me 3 Franck Dernoncourt 3 0) Foreword Hands-free paintings!
4. 2012-09-15 Franck Dernoncourt - http://francky.me 4 Franck Dernoncourt 4 0) Foreword Video http://youtu.be/fIcGJCUAMJg
5. 2012-09-15 Franck Dernoncourt - http://francky.me 5 Franck Dernoncourt 5 0) Foreword
6. 2012-09-15 Franck Dernoncourt - http://francky.me 6 Franck Dernoncourt 6 0) Foreword The first mouse prototype was created in 1963 by Douglas Engelbart at the Stanford Research Institute… almost 50-year-old!
7. 2012-09-15 Franck Dernoncourt - http://francky.me 7 Table of Contents 1.Introduction 2.Moving the mouse cursor 3.Emulating the mouse click 4.Getting rid of the keyboard as well 5.Conclusion & perspectives
8. 2012-09-15 Franck Dernoncourt - http://francky.me 8 Franck Dernoncourt 8 1) Introduction Important drawbacks  Loss of productivity: constantly switching between the mouse and the keyboard.  Health issues: RSI (repetitive stress injuries).
9. 2012-09-15 Franck Dernoncourt - http://francky.me 9 Franck Dernoncourt 9 1) Introduction Adhesive Capsulitis (Frozen Shoulder), Bursitis, Carpal Tunnel Syndrome, Cramp of the Hand (Writers’ Cramp), Cubital Tunnel Syndrome, De Quervain’s Syndrome, Dupuytren’s Contracture, Epicondylitis (tennis / golfer’s elbow), Ganglion Cyst, Peritendinitis, Rotator Cuff Syndrome, Tendinitis, Tenosynovitis, Trigger Finger / Thumb, Vibration-induced White Finger…
10. 2012-09-15 Franck Dernoncourt - http://francky.me 10 Franck Dernoncourt 10 1) Introduction Adhesive Capsulitis (Frozen Shoulder), Bursitis, Carpal Tunnel Syndrome, Cramp of the Hand (Writers’ Cramp), Cubital Tunnel Syndrome, De Quervain’s Syndrome, Dupuytren’s Contracture, Epicondylitis (tennis / golfer’s elbow), Ganglion Cyst, Peritendinitis, Rotator Cuff Syndrome, Tendinitis, Tenosynovitis, Trigger Finger / Thumb, Vibration-induced White Finger… RSI!  15-25% of all computer users worldwide are estimated to have RSI.  A survey of 500 software professionals at Hyderabad in 2000 revealed that over 50% had symptoms of established RSI.  Billions of dollars are spent worldwide annually as a consequence of RSI
11. 2012-09-15 Franck Dernoncourt - http://francky.me 11 Franck Dernoncourt 11 1) Introduction Chronic pain may cost U.S... $635 billion a year !! Source: http://medicalxpress.com/news/2012-09-chronic-pain-billion-year.html
12. 2012-09-15 Franck Dernoncourt - http://francky.me 12 Franck Dernoncourt 12 1) Introduction Important drawbacks  Loss of productivity: constantly switching between the mouse and the keyboard.  Health issues: RSI (repetitive stress injuries).
13. 2012-09-15 Franck Dernoncourt - http://francky.me 13 Franck Dernoncourt 13 1) Introduction Important drawbacks  Loss of productivity: constantly switching between the mouse and the keyboard.  Health issues: RSI (repetitive stress injuries).  Unnatural user interface.
14. 2012-09-15 Franck Dernoncourt - http://francky.me 14 Franck Dernoncourt 14 1) Introduction We do not aim at replacing the mouse as a pointing device but simply the mouse as a piece of hardware. We therefore stay in the mouse paradigm and focus on the hardware interface. The computer mouse allows two kinds of action: • moving the mouse cursor, • sending mouse clicks.
15. 2012-09-15 Franck Dernoncourt - http://francky.me 15 Franck Dernoncourt 15 1) AI > definitions Hands-free mice are numerous:  Camera based head tracking systems: SmartNav, Tracker Pro, FreeTrack, HeadMouse Extreme and HeadMaster,  Mouth-operated joystick types: the TetraMouse, the QuadJoy, the Jouse2, the IntegraMouse,  Footmice: BiLiPro, Flip Flop Mouse, Footime Foot ControlledMouse,  Brain-computer interaction: the Emotiv EPOC neuroheadset, the NeuroSky MindSet/MindWave,  Eye tracking.
16. 2012-09-15 Franck Dernoncourt - http://francky.me 16 Table of Contents 1.Introduction 2.Moving the mouse cursor 3.Emulating the mouse click 4.Getting rid of the keyboard as well 5.Conclusion & perspectives
17. 2012-09-15 Franck Dernoncourt - http://francky.me 17 Franck Dernoncourt 17 2) Moving the mouse cursor Footmice might cause stress on the feet or the legs and takes a while to get precise with them. Still worth the try as it is inexpensive and one can get impressively accurate with one’s feet. Footime® Foot Mouse (150 USD)
18. 2012-09-15 Franck Dernoncourt - http://francky.me 18 Franck Dernoncourt 18 2) Moving the mouse cursor Eye tracking: either pretty bad or expensive (5,000 USD), and don’t support multiscreen configuration. EyeTech (> 4,500 USD) BUT this is might be the future: basic webcam + open source software (e.g. ITU GazeGroup).
19. 2012-09-15 Franck Dernoncourt - http://francky.me 19 Franck Dernoncourt 19 2) Moving the mouse cursor
20. 2012-09-15 Franck Dernoncourt - http://francky.me 20 Franck Dernoncourt 20 2) Moving the mouse cursor
21. 2012-09-15 Franck Dernoncourt - http://francky.me 21 Franck Dernoncourt 21 2) Moving the mouse cursor Among the camera based head tracking systems, SmartNav is the cheapest (300-400 USD). As its precision is as good as the one of a computer mouse, there is no need to invest into the Tracker Pro, FreeTrack, HeadMouse Extreme or HeadMaster (over 1,000 USD).
22. 2012-09-15 Franck Dernoncourt - http://francky.me 22 Franck Dernoncourt 22 2) Moving the mouse cursor The mouse-operated joysticks are pretty intrusive since one has to put them in the mouth, but this solution is interesting for people who can't move their head or have severe pain in the neck. The TetraMouse is the cheapest by far.
23. 2012-09-15 Franck Dernoncourt - http://francky.me 23 Franck Dernoncourt 23 2) Moving the mouse cursor Brain computer interaction are so far mostly useless to move the mouse cursor. Note that the Emotiv EPOC neuroheadset contains a gyroscope (=device for measuring or maintaining orientation), thanks to which the user can move the mouse cursor as precisely as a computer mouse. Emotiv EPOC neuroheadset (300 USD)
24. 2012-09-15 Franck Dernoncourt - http://francky.me 24 Table of Contents 1.Introduction 2.Moving the mouse cursor 3.Emulating the mouse click 4.Getting rid of the keyboard as well 5.Conclusion & perspectives
25. 2012-09-15 Franck Dernoncourt - http://francky.me 25 Franck Dernoncourt 25 3) Emulating the mouse click Hotkeys: re-map keys from the keyboard and assign them to emulate the left, right and middle mouse buttons.
26. 2012-09-15 Franck Dernoncourt - http://francky.me 26 Franck Dernoncourt 26 3) Emulating the mouse click Dwell clicking software: when the cursor stops moving for a pre-determined amount of time (usually around 1 second), the dwell clicking software will initiate a mouse click. The user can have the software send left clicks, right clicks or double clicks. SmartNav's dwell clicking software (free)
27. 2012-09-15 Franck Dernoncourt - http://francky.me 27 Franck Dernoncourt 27 3) Emulating the mouse click Footswitches: allow the user to send mouse clicks by pushing a pedal. !! Beware of repetitive strain injuries at the ankles.
28. 2012-09-15 Franck Dernoncourt - http://francky.me 28 Franck Dernoncourt 28 3) Emulating the mouse click Sip-and-puff: send signals to a device using air pressure by "sipping" (inhaling) or "puffing" (exhaling) on a straw, tube or "wand."
29. 2012-09-15 Franck Dernoncourt - http://francky.me 29 Franck Dernoncourt 29 3) Emulating the mouse click Speech recognition: set a few voice commands that the user can say to emulate mouse clicks.
30. 2012-09-15 Franck Dernoncourt - http://francky.me 30 Franck Dernoncourt 30 3) Emulating the mouse click Facial expression recognition: maps facial expressions such as eye blink, wink or smile to mouse clicks. EyeTech (> 4,500 USD)
31. 2012-09-15 Franck Dernoncourt - http://francky.me 31 Franck Dernoncourt 31 3) Emulating the mouse click Facial expression recognition: maps facial expressions such as eye blink, wink or smile to mouse clicks. NeuroSky MindWave (100 USD) Emotiv EPOC neuroheadset (300 USD)
32. 2012-09-15 Franck Dernoncourt - http://francky.me 32 Franck Dernoncourt 32 3) Emulating the mouse click Brain-computer interaction: maps concepts to mouse clicks. When the user thinks of one concept, a mouse click is sent. NeuroSky MindWave (100 USD) Emotiv EPOC neuroheadset (300 USD)
33. 2012-09-15 Franck Dernoncourt - http://francky.me 33 Franck Dernoncourt 33 3) Emulating the mouse click Headset comfortability
34. 2012-09-15 Franck Dernoncourt - http://francky.me 34 3) Emulating the mouse click Quite expensive (100-300USD), detection is not 100% accurate, headsets are not comfortable Hands-free Facial expression High latency, put some strain on the voice, noisy Easy to use, already integrated within Dragon NaturallySpeaking Speech recognition Latency, quite expensive (100-300USD), not accurate enough to emulate mouse clicks, headsets are not comfortable Hands-free Brain-computer interaction Dwell clicking Hotkeys Solution Requires to wait ~1s before click is sent, takes some time (~1s) to switch between mouse click type Free, easy to use, no latency, hands-free Requires to use the keyboard, not so good for RSI (much better than mouse clicks though) Free, easy to use, no latency Cons Pros Synopsis
35. 2012-09-15 Franck Dernoncourt - http://francky.me 35 Franck Dernoncourt 35 3) Emulating the mouse click My current configuration: Dwell clicking + hotkeys + speech recognition
36. 2012-09-15 Franck Dernoncourt - http://francky.me 36 Franck Dernoncourt 36 3) Emulating the mouse click Next test (hopefully in a few days): MaKey MaKey!
37. 2012-09-15 Franck Dernoncourt - http://francky.me 37 Table of Contents 1.Introduction 2.Moving the mouse cursor 3.Emulating the mouse click 4.Getting rid of the keyboard as well 5.Conclusion & perspectives
38. 2012-09-15 Franck Dernoncourt - http://francky.me 38 4) Getting rid of the keyboard as well
39. 2012-09-15 Franck Dernoncourt - http://francky.me 39 4) Getting rid of the keyboard as well
40. 2012-09-15 Franck Dernoncourt - http://francky.me 40 4) Getting rid of the keyboard as well The average rate for transcription is 33 words per minute, and 19 words per minute for composition. An average professional typist types usually in speeds of 50 to 80 words per minute. More statistics on http://en.wikipedia.org/wiki/Words_per_minute
41. 2012-09-15 Franck Dernoncourt - http://francky.me 41 4) Getting rid of the keyboard as well The average rate for transcription is 33 words per minute, and 19 words per minute for composition. An average professional typist types usually in speeds of 50 to 80 words per minute. More statistics on http://en.wikipedia.org/wiki/Words_per_minute Using speech recognition, one can easily achieve over 100 words per minute with more than 95% accuracy.
42. 2012-09-15 Franck Dernoncourt - http://francky.me 42 4) Getting rid of the keyboard as well Many voice commands are available, such as any shortcut (e.g. copy paste), typing a predefined text, switching windows, browsing the web, sending e-mail and launching programs. Custom commands can also be easily defined:
43. 2012-09-15 Franck Dernoncourt - http://francky.me 43 Franck Dernoncourt 43 4) Getting rid of the keyboard as well
44. 2012-09-15 Franck Dernoncourt - http://francky.me 44 Franck Dernoncourt 44 4) Getting rid of the keyboard as well Work in progress!
45. 2012-09-15 Franck Dernoncourt - http://francky.me 45 Franck Dernoncourt 45 4) Getting rid of the keyboard as well Subvocal recognition
46. 2012-09-15 Franck Dernoncourt - http://francky.me 46 Franck Dernoncourt 46 4) Getting rid of the keyboard as well Throat microphones
47. 2012-09-15 Franck Dernoncourt - http://francky.me 47 Table of Contents 1.Introduction 2.Moving the mouse cursor 3.Emulating the mouse click 4.Getting rid of the keyboard as well 5.Conclusion & perspectives
48. 2012-09-15 Franck Dernoncourt - http://francky.me 48 Franck Dernoncourt 48 5) Conclusion & perspectives Conclusion:  SmartNAV + Dwell clicking + hotkeys + speech recognition  Speech recognition  Replace 95% of the use of the mouse and the keyboard.  More efficient, more comfortable.
49. 2012-09-15 Franck Dernoncourt - http://francky.me 49 Franck Dernoncourt 49 5) Conclusion & perspectives
50. 2012-09-15 Franck Dernoncourt - http://francky.me 50 Franck Dernoncourt 50 5) Conclusion & perspectives Conclusion: Natural user interfaces
51. 2012-09-15 Franck Dernoncourt - http://francky.me 51 Franck Dernoncourt 51 5) Conclusion & perspectives Conclusion: Natural user interfaces Eye tracking Brain computer interfaces
52. 2012-09-15 Franck Dernoncourt - http://francky.me 52 18/09/2012 Motion-induced blindness 52 Questions ? franck.dernoncourt@gmail.com