Intelligently Recommending Key Bindings on Physical Keyboards with Demonstrations in Emacs

The presentation slides of the paper "Shudan Zhong and Hong Xu. Intelligently recommending key bindings on physical keyboards with demonstrations in Emacs. In Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI), 12–17. 2019."

More details: http://www.hong.me/papers/zhong2019a.html
Link to published paper: https://doi.org/10.1145/3301275.3302272

Transcript

1. Intelligently Recommending Key Bindings on Physical Keyboards with Demonstrations in Emacs Shudan Zhong1 Hong Xu2 sdz@berkeley.edu, hongx@usc.edu March 18, 2019 1University of California, Berkeley 2University of Southern California the 24th International Conference on Intelligent User Interfaces (IUI) Marina del Ray, CA, USA
2. Executive Summary • Key bindings are the main means to interact with a computer with a physical keyboard. • Different key bindings can lead to drastically different user experience and productivity. • Surprisingly, research on recommender systems for key bindings are lacking. • We propose the first (to the best of our knowledge) such recommender system and demonstrate it in Emacs. Zhong and Xu Recommending Key Bindings on Physical Keyboards 1 / 14
3. Agenda Key Bindings on Physical Keyboards The Emacs Key Binding Recommender System (EKBRS) as a Demonstration Details of the Emacs Key Binding Recommender System (EKBRS) Empirical Evaluation Conclusion Zhong and Xu Recommending Key Bindings on Physical Keyboards 2 / 14
4. Agenda Key Bindings on Physical Keyboards The Emacs Key Binding Recommender System (EKBRS) as a Demonstration Details of the Emacs Key Binding Recommender System (EKBRS) Empirical Evaluation Conclusion
5. What Are Key Bindings? • Key bindings define how users interact with computers via physical keyboards. • (Classic) examples: • Copy: Ctrl + c • Paste: Ctrl + v • Save: Ctrl + s • Applications define their own key bindings. Zhong and Xu Recommending Key Bindings on Physical Keyboards 3 / 14
6. Key Bindings Are Important Key bindings play essential roles in many professional applications. (a) Programming (b) Computer-Aided Design (c) Professional Gaming Image sources: (a) http://blog.markpearl.co.za/Ultimate-Developer-Keyboard (b) https://allaboutcad.com/tutorial-create-a-command-alias-keyboard-shortcut/ (c) https://www.logitechg.com/en-us/products/gaming-mice/g600-mmo-gaming-mouse.html Zhong and Xu Recommending Key Bindings on Physical Keyboards 4 / 14
7. Key Bindings Are Important Different key bindings can lead to drastically different productivity and user experience. • An absurd example: for moving down and for moving up. Image source: https://i.redd.it/al15957ctaky.jpg Zhong and Xu Recommending Key Bindings on Physical Keyboards 5 / 14
8. Key Bindings Are Important Different key bindings can lead to drastically different user experience. • In browsers, Space is commonly used for scrolling one page down. Users can comfortably sit back and tap the spacious Space . Image source: https: //softwarerecs.stackexchange.com/questions/19804/browser-plugin-to-change-the-amount-of-scroll-on-space-key Zhong and Xu Recommending Key Bindings on Physical Keyboards 6 / 14
9. Key Bindings Are Important Different key bindings can lead to drastically different productivity and user experience. Caps Lock Shift Shift (a) qwerty Caps Lock Shift Shift (b) Dvorak Shift Shift (c) Colemak Ctrl + z , Ctrl + x , Ctrl + c , Ctrl + v Image source: (a) https://commons.wikimedia.org/wiki/File:KB_United_States.svg (b) https://commons.wikimedia.org/wiki/File:KB_United_States_Dvorak.svg (c) https://commons.wikimedia.org/wiki/File:KB_US-Colemak.svg Zhong and Xu Recommending Key Bindings on Physical Keyboards 7 / 14
10. Intelligently Recommending Key Bindings • Although key bindings are important, surprisingly, research on applying recommender systems to key bindings on physical keyboards is lacking. • We propose the first (to the best of our knowledge) recommender system that recommends key bindings. • We demonstrate such the recommender system in Emacs. Zhong and Xu Recommending Key Bindings on Physical Keyboards 8 / 14
11. Agenda Key Bindings on Physical Keyboards The Emacs Key Binding Recommender System (EKBRS) as a Demonstration Details of the Emacs Key Binding Recommender System (EKBRS) Empirical Evaluation Conclusion
12. Why Demonstrate in Emacs? • Key bindings matter in Emacs: It heavily exploits key bindings in its user interface. • Extensive customizability: • Emacs is a Lisp machine (Lisp interpreter + Lisp library + user interface). • Sequences of key strokes can be effectively bound to invoke virtually any existing/user-defined Lisp function. • It allows sophisticated sequence of key strokes in key bindings, e.g., Ctrl + u Ctrl + x s . Zhong and Xu Recommending Key Bindings on Physical Keyboards 9 / 14
13. Agenda Key Bindings on Physical Keyboards The Emacs Key Binding Recommender System (EKBRS) as a Demonstration Details of the Emacs Key Binding Recommender System (EKBRS) Empirical Evaluation Conclusion
14. Emacs Key Binding Recommender System (EKBRS) Lisp function describe-variable function-function module word-suffix module posterior-word-suffix prior-word-suffix Prefix Suffix Ctrl + h v combine recommends recommends recommends Zhong and Xu Recommending Key Bindings on Physical Keyboards 10 / 14
15. Emacs Key Binding Recommender System (EKBRS) prior-word-suffix posterior-word-suffix function-function Exploit relation- ship between suffices, English words suffices, English words Lisp functions, Lisp functions Relationship derived from Prior knowledge Key binding database Key binding database Example next-line/previous-line is bound to Ctrl + n / Ctrl + p . Lisp functions with “buffer” in their names are often bound by a key sequence with b . Most help-related Lisp functions are bound by a key sequence with prefix Ctrl + h . Zhong and Xu Recommending Key Bindings on Physical Keyboards 11 / 14
16. Agenda Key Bindings on Physical Keyboards The Emacs Key Binding Recommender System (EKBRS) as a Demonstration Details of the Emacs Key Binding Recommender System (EKBRS) Empirical Evaluation Conclusion
17. Setup • We used the default global key binding database in vanilla Emacs 26.1. • 391 key bindings after removing key bindings involving keys that are outside the main area of a qwerty keyboard. • Evaluation: • Leave-one-out strategy: In each round, we temporarily remove one key binding (for a Lisp function ) from the key binding database. • We let EKBRS recommend n = 1 . . . 5 key bindings for . The key binding is recovered if the removed key binding is among the recommended key bindings. • We use the percentage of recovery as an evaluation metric. Zhong and Xu Recommending Key Bindings on Physical Keyboards 12 / 14
18. Results 1 2 3 4 5 n 0.5 0.6 0.7 0.8 0.9 1.0 Percentage of Recovery 0.62 0.79 0.89 0.93 0.95 0.64 0.80 0.88 0.92 0.95 0.63 0.82 0.88 0.90 0.91 0.63 0.76 0.83 0.89 0.92 dice embedding-n embedding-s embedding-sd (a) Prefixes 1 2 3 4 5 n 0.0 0.2 0.4 0.6 0.8 Percentage of Recovery 0.21 0.38 0.48 0.49 0.49 0.22 0.34 0.39 0.43 0.44 0.29 0.48 0.62 0.65 0.67 prior-word-suffix posterior-word-suffix combined (b) Suffices 1 2 3 4 5 n 0.0 0.1 0.2 0.3 0.4 0.5 Percentage of Recovery 0.18 0.31 0.37 0.41 0.43 0.20 0.27 0.33 0.36 0.37 0.22 0.28 0.33 0.35 0.35 0.19 0.25 0.29 0.32 0.33 dice embedding-n embedding-s embedding-sd (c) Key bindings Zhong and Xu Recommending Key Bindings on Physical Keyboards 13 / 14
19. Agenda Key Bindings on Physical Keyboards The Emacs Key Binding Recommender System (EKBRS) as a Demonstration Details of the Emacs Key Binding Recommender System (EKBRS) Empirical Evaluation Conclusion
20. Conclusion • Key bindings for physical keyboards often play critical roles in productivity and user experience. We need recommender systems for key bindings. • We proposed the first (to the best of our knowledge) recommender system for key bindings, with a demonstration in Emacs. • Future work: Develop recommender systems for key bindings in other contexts. Zhong and Xu Recommending Key Bindings on Physical Keyboards 14 / 14