Intelligently Recommending Key Bindings on Physical Keyboards with Demonstrations in Emacs

The presentation slides of the paper "Shudan Zhong and Hong Xu. Intelligently recommending key bindings on physical keyboards with demonstrations in Emacs. In Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI), 12–17. 2019."

More details: http://www.hong.me/papers/zhong2019a.html
Link to published paper: https://doi.org/10.1145/3301275.3302272

Transcript

1. Intelligently Recommending Key Bindings on
   Physical Keyboards with Demonstrations in Emacs
   Shudan Zhong1 Hong Xu2
   sdz@berkeley.edu, hongx@usc.edu
   March 18, 2019
   1University of California, Berkeley
   2University of Southern California
   the 24th International Conference on Intelligent User Interfaces (IUI)
   Marina del Ray, CA, USA
   

   View Slide

2. Executive Summary
   • Key bindings are the main means to interact with a computer with a
   physical keyboard.
   • Different key bindings can lead to drastically different user experience
   and productivity.
   • Surprisingly, research on recommender systems for key bindings are
   lacking.
   • We propose the first (to the best of our knowledge) such
   recommender system and demonstrate it in Emacs.
   Zhong and Xu Recommending Key Bindings on Physical Keyboards 1 / 14
   

   View Slide

3. Agenda
   Key Bindings on Physical Keyboards
   The Emacs Key Binding Recommender System (EKBRS) as a
   Demonstration
   Details of the Emacs Key Binding Recommender System (EKBRS)
   Empirical Evaluation
   Conclusion
   Zhong and Xu Recommending Key Bindings on Physical Keyboards 2 / 14
   

   View Slide

4. Agenda
   Key Bindings on Physical Keyboards
   The Emacs Key Binding Recommender System (EKBRS) as a
   Demonstration
   Details of the Emacs Key Binding Recommender System (EKBRS)
   Empirical Evaluation
   Conclusion
   

   View Slide

5. What Are Key Bindings?
   • Key bindings define how users interact with computers via physical
   keyboards.
   • (Classic) examples:
   • Copy: Ctrl + c
   • Paste: Ctrl + v
   • Save: Ctrl + s
   • Applications define their own key bindings.
   Zhong and Xu Recommending Key Bindings on Physical Keyboards 3 / 14
   

   View Slide

6. Key Bindings Are Important
   Key bindings play essential roles in many professional applications.
   (a) Programming (b) Computer-Aided Design (c) Professional Gaming
   Image sources:
   (a) http://blog.markpearl.co.za/Ultimate-Developer-Keyboard
   (b) https://allaboutcad.com/tutorial-create-a-command-alias-keyboard-shortcut/
   (c) https://www.logitechg.com/en-us/products/gaming-mice/g600-mmo-gaming-mouse.html
   Zhong and Xu Recommending Key Bindings on Physical Keyboards 4 / 14
   

   View Slide

7. Key Bindings Are Important
   Different key bindings can lead to drastically different productivity and
   user experience.
   • An absurd example: for
   moving down and for
   moving up.
   Image source: https://i.redd.it/al15957ctaky.jpg
   Zhong and Xu Recommending Key Bindings on Physical Keyboards 5 / 14
   

   View Slide

8. Key Bindings Are Important
   Different key bindings can lead to drastically different user experience.
   • In browsers, Space is
   commonly used for scrolling one
   page down. Users can
   comfortably sit back and tap the
   spacious Space .
   Image source: https:
   //softwarerecs.stackexchange.com/questions/19804/browser-plugin-to-change-the-amount-of-scroll-on-space-key
   Zhong and Xu Recommending Key Bindings on Physical Keyboards 6 / 14
   

   View Slide

9. Key Bindings Are Important
   Different key bindings can lead to drastically different productivity and
   user experience.
   Caps Lock
   Shift Shift
   (a) qwerty
   Caps Lock
   Shift Shift
   (b) Dvorak
   Shift Shift
   (c) Colemak
   Ctrl + z , Ctrl + x , Ctrl + c , Ctrl + v
   Image source:
   (a) https://commons.wikimedia.org/wiki/File:KB_United_States.svg
   (b) https://commons.wikimedia.org/wiki/File:KB_United_States_Dvorak.svg
   (c) https://commons.wikimedia.org/wiki/File:KB_US-Colemak.svg
   Zhong and Xu Recommending Key Bindings on Physical Keyboards 7 / 14
   

   View Slide

10. Intelligently Recommending Key Bindings
    • Although key bindings are important, surprisingly, research on
    applying recommender systems to key bindings on physical
    keyboards is lacking.
    • We propose the first (to the best of our knowledge) recommender
    system that recommends key bindings.
    • We demonstrate such the recommender system in Emacs.
    Zhong and Xu Recommending Key Bindings on Physical Keyboards 8 / 14
    

    View Slide

11. Agenda
    Key Bindings on Physical Keyboards
    The Emacs Key Binding Recommender System (EKBRS) as a
    Demonstration
    Details of the Emacs Key Binding Recommender System (EKBRS)
    Empirical Evaluation
    Conclusion
    

    View Slide

12. Why Demonstrate in Emacs?
    • Key bindings matter in Emacs: It heavily exploits key bindings in its
    user interface.
    • Extensive customizability:
    • Emacs is a Lisp machine (Lisp interpreter + Lisp library + user interface).
    • Sequences of key strokes can be effectively bound to invoke virtually
    any existing/user-defined Lisp function.
    • It allows sophisticated sequence of key strokes in key bindings, e.g.,
    Ctrl + u Ctrl + x s .
    Zhong and Xu Recommending Key Bindings on Physical Keyboards 9 / 14
    

    View Slide

13. Agenda
    Key Bindings on Physical Keyboards
    The Emacs Key Binding Recommender System (EKBRS) as a
    Demonstration
    Details of the Emacs Key Binding Recommender System (EKBRS)
    Empirical Evaluation
    Conclusion
    

    View Slide

14. Emacs Key Binding Recommender System (EKBRS)
    Lisp function describe-variable
    function-function module word-suffix module
    posterior-word-suffix prior-word-suffix
    Prefix Suffix
    Ctrl + h v
    combine
    recommends
    recommends recommends
    Zhong and Xu Recommending Key Bindings on Physical Keyboards 10 / 14
    

    View Slide

15. Emacs Key Binding Recommender System (EKBRS)
    prior-word-suffix posterior-word-suffix function-function
    Exploit relation-
    ship between
    suffices,
    English words
    suffices,
    English words
    Lisp functions,
    Lisp functions
    Relationship
    derived from
    Prior knowledge Key binding database Key binding database
    Example next-line/previous-line
    is bound to Ctrl + n / Ctrl + p .
    Lisp functions with
    “buffer” in their names
    are often bound by a
    key sequence with b .
    Most help-related Lisp
    functions are bound by
    a key sequence with
    prefix Ctrl + h .
    Zhong and Xu Recommending Key Bindings on Physical Keyboards 11 / 14
    

    View Slide

16. Agenda
    Key Bindings on Physical Keyboards
    The Emacs Key Binding Recommender System (EKBRS) as a
    Demonstration
    Details of the Emacs Key Binding Recommender System (EKBRS)
    Empirical Evaluation
    Conclusion
    

    View Slide

17. Setup
    • We used the default global key binding database in vanilla Emacs 26.1.
    • 391 key bindings after removing key bindings involving keys that are
    outside the main area of a qwerty keyboard.
    • Evaluation:
    • Leave-one-out strategy: In each round, we temporarily remove one key
    binding (for a Lisp function ) from the key binding database.
    • We let EKBRS recommend n = 1 . . . 5 key bindings for . The key binding
    is recovered if the removed key binding is among the recommended key
    bindings.
    • We use the percentage of recovery as an evaluation metric.
    Zhong and Xu Recommending Key Bindings on Physical Keyboards 12 / 14
    

    View Slide

18. Results
    1 2 3 4 5
    n
    0.5
    0.6
    0.7
    0.8
    0.9
    1.0
    Percentage of Recovery
    0.62
    0.79
    0.89
    0.93
    0.95
    0.64
    0.80
    0.88
    0.92
    0.95
    0.63
    0.82
    0.88
    0.90 0.91
    0.63
    0.76
    0.83
    0.89
    0.92
    dice
    embedding-n
    embedding-s
    embedding-sd
    (a) Prefixes
    1 2 3 4 5
    n
    0.0
    0.2
    0.4
    0.6
    0.8
    Percentage of Recovery
    0.21
    0.38
    0.48 0.49 0.49
    0.22
    0.34
    0.39
    0.43 0.44
    0.29
    0.48
    0.62
    0.65
    0.67
    prior-word-suffix
    posterior-word-suffix
    combined
    (b) Suffices
    1 2 3 4 5
    n
    0.0
    0.1
    0.2
    0.3
    0.4
    0.5
    Percentage of Recovery
    0.18
    0.31
    0.37
    0.41
    0.43
    0.20
    0.27
    0.33
    0.36
    0.37
    0.22
    0.28
    0.33
    0.35 0.35
    0.19
    0.25
    0.29
    0.32 0.33
    dice
    embedding-n
    embedding-s
    embedding-sd
    (c) Key bindings
    Zhong and Xu Recommending Key Bindings on Physical Keyboards 13 / 14
    

    View Slide

19. Agenda
    Key Bindings on Physical Keyboards
    The Emacs Key Binding Recommender System (EKBRS) as a
    Demonstration
    Details of the Emacs Key Binding Recommender System (EKBRS)
    Empirical Evaluation
    Conclusion
    

    View Slide

20. Conclusion
    • Key bindings for physical keyboards often play critical roles in
    productivity and user experience. We need recommender systems for
    key bindings.
    • We proposed the first (to the best of our knowledge) recommender
    system for key bindings, with a demonstration in Emacs.
    • Future work: Develop recommender systems for key bindings in other
    contexts.
    Zhong and Xu Recommending Key Bindings on Physical Keyboards 14 / 14
    

    View Slide