Exploring Reline: Enhancing Command Line Usability

1. Exploring Reline: Enhancing Command Line Usability Mari Imaizumi @ima1zumi 2024-05-15
2. 2 IRB
3. IRB Interactive Ruby Default Ruby REPL (Read-Eval-Print Loop) Enter Ruby code, eval, and display results Run using the `irb` command Default Rails console is IRB 3
4. IRB 4
5. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Do you know how the IRB works? 5
6. 6 IRB Reline Terminal Emulator
9. About me Mari Imaizumi (@ima1zumi) IRB and Reline committer STORES, Inc. 9
10. Nursery Sponser Speakers: LT: Sponser booth (Ruby quiz presented by ) 10
11. Agenda What is Reline? Feature Gap between Reline and GNU Readline Understanding Reline 11
12. What is Reline?
13. Reline is a command line editor A command line editor operates on terminal emulators, manipulating user input While not a common term, 'command line editor' is used here due to the lack of a generic term for software like GNU Readline. Examples: GNU Readline, libedit, Reline Reline has GNU Readline and editline compatibility 13
14. Without a Command Line Editor 14 puts "Please enter your name:" text = gets.chomp puts "You entered: #{text}"
15. Without a Command Line Editor 15 Cursor Back(CSI + D)
16. Use Reline 16 require "reline" puts "Please enter your name:" text = Reline.readline puts "You entered: #{text}"
17. Use Reline 17
18. Command Line Editor Provides a set of operations for editing entered text Command line editors manage key operations in tools like IRB and bash 18
19. Command Line Editors GNU Readline Used in bash, IRB (up to Ruby 2.6), GDB NetBSD Editline Library (libedit) Utilized in macOS Reline Used in IRB (Ruby 2.7 and above), debug, highline, hexapdf 19
20. Capabilities of GNU Readline Line Editing History management Autocompletion Editing Command and more... 20
21. Feature Comparison Line Editing History Autocompletion Editing Command Others GNU Readline ✅ ✅ ✅ ✅ ✅ Reline ✅ ✅ ✅ ˚ ˚ 21
22. Capabilities of Reline Display dialogs Multiline input including prompts Extensible with Ruby Modifications to input lines, dialogs, completion suggestions with procs Completion suggestions also possible with readline-ext 22
23. However, it lacks many features: Key bindings Editing commands Configurable variables and more Issues are occasionally reported about functionalities available in Readline but missing in Reline Indicating user interest in these features 23 Reline is Compatible with GNU Readline
24. From Ruby 3.3, readline-ext removed from default gems If readline-ext is unavailable, require 'readline' will automatically use Reline. [1][2] Eliminates installation issues with GNU Readline during Ruby builds. Increased need for Reline's compatibility features. Users may unknowingly switch dependency from readline-ext to Reline. 24 require 'readline' loads Reline if missing readline-ext
25. My wishes Hoping for Reline to be used instead of readline-ext. Hope that users switch to Reline without noticing, ensuring a seamless transition. Aim for Reline to become the chosen command line editor for users considering alternatives. Add initial support for Reline on Ruby 3.3 #2298 · pry/pry https://github.com/pry/pry/pull/2298 25
26. Challenges Reline aims for GNU Readline compatibility; better compatibility is desired. Expectation for require 'readline' to seamlessly use Reline. 26
27. Challenges Reline aims for GNU Readline compatibility; better compatibility is desired. Expectation for require 'readline' to seamlessly use Reline. More GNU Readline features needed in Reline What specific features are missing? 27
28. Feature Gap between Reline and GNU Readline
29. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Have you ever written an .inputrc fi le? 🙋 29
30. What is .inputrc? Configuration file for GNU Readline and Reline ~/.inputrc or INPUTRC=path/to/file Configurable options: Configurable variables Editing commands Key bindings Set editing mode (emacs or vi) 30
31. Emacs mode, Vi mode Emacs mode (default) Vi mode has command mode and insert mode does not have a full set of vi editing functions 31
32. .inputrc 32 Variables Key bindings
33. Key Bindings Default key bindings are set in command line editor For example, in Emacs mode, "\C-a" moves the cursor to the beginning of the line In Vi mode, there are different bindings for command mode and insert mode Customization is possible through .inputrc Some settings written in .inputrc may not work with Reline 33
34. Con fi gurations usable in Reline Variables: 12 / 46 (26%) Editing commands: 53 / 120 (44%) April 16, 2024 34
35. Importance Assessment Wanted to know what settings are used how frequently Conducted investigation using .inputrc files pushed to GitHub https://github.com/search?q=language%3AReadline&type=code 35
36. Usage Analysis Investigated how many out of 166 configurable items were mentioned in .inputrc Includes commented-out entries For items with default key bindings set, the presence in .inputrc indicates usage Total files examined: 6600 36
37. High usage 37
38. Low usage 38
39. Implemented Features 39 Undo show-all-if-ambiguous vi-editing-mode, emacs-editing-mode mode condition in .inputrc Bracketed paste mode (Implemented by tompng )
40. Understanding Reline
41. Exploring Reline Implementation 1. Enter "a" 2. Enter "C-a" (Control-a) 3. Implement the undo command 41
42. First Step: Follow the input text 42 require "reline" text = Reline.readline puts text Enter "a\n"
43. First Step: Follow the input text 43
44. Flow 44 Handle input keys Render Wait inputs Initialize Reline.readline
45. 45 # reline.rb # Simplified original code private def inner_readline(prompt, add_hist, multiline, &confirm_multiline_termination) l = line_editor l.reset(prompt, encoding: encoding) # l.confirm_multiline_termination_proc = confirm_multiline_termination l.output = output l.completion_proc = completion_proc l.completion_append_character = completion_append_character l.output_modifier_proc = output_modifier_proc l.prompt_proc = prompt_proc l.auto_indent_proc = auto_indent_proc l.dig_perfect_match_proc = dig_perfect_match_proc pre_input_hook&.call @dialog_proc_list.each_pair do |name, d| l.add_dialog_proc(name,d.dialog_proc,d.context) end config.read io_gate.set_default_key_bindings(config) Handle input keys Render Wait inputs Initialize Reline.readline
46. 46 # reline.rb private def inner_readline(prompt, add_hist, multiline, &confirm_multiline_termination) # Snip loop do read_io(config.keyseq_timeout) { |inputs| inputs.each { |key| line_editor.update(key) } } if line_editor.finished? line_editor.render_finished break else line_editor.rerender end end end Handle input keys Render Wait inputs Initialize Reline.readline
47. # ansi.rb def self.inner_getc(timeout_second) unless @@buf.empty? return @@buf.shift end until @@input.wait_readable(0.01) timeout_second -= 0.01 return nil if timeout_second <= 0 Reline.core.line_editor.handle_signal end c = @@input.getbyte (c == 0x16 && @@input.raw( min: 0, time: 0, &:getbyte)) || c rescue Errno::EIO # Maybe the I/O has been closed. nil rescue Errno::ENOTTY nil end 47 Handle input keys Render Wait inputs Initialize Reline.readline c is 97(Integer)
48. 48 # reline.rb private def inner_readline(prompt, add_hist, multiline, &confirm_multiline_termination) # Snip loop do read_io(config.keyseq_timeout) { |inputs| inputs.each { |key| line_editor.update(key) } } if line_editor.finished? line_editor.render_finished break else line_editor.rerender end end end Handle input keys Render Wait inputs Initialize Reline.readline
49. 49 Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def normal_char(key) @buf << key.combined_char if @buf.size > 1 # multi byte keys = @buf.dup.force_encoding(@encoding) process_key(keys, nil) @multibyte_buffer.clear else # single byte return if key.char >= 128 method_symbol = @config.editing_mode.get_method( key.combined_char) process_key(key.combined_char, method_symbol) @multibyte_buffer.clear end end
50. 50 Handle input keys Render Wait inputs Initialize Reline.readline method_symbol is :ed_insert # emacs.rb class Reline::KeyActor::Emacs MAPPING = [ # 0 ^@ :em_set_mark, # 1 ^A :ed_move_to_beg, # snip # 97 a :ed_insert, key is 97 ->
51. 51 Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def ed_insert(key) key.chr.encode(Encoding::UTF_8) str = key.chr insert_text(str) end def insert_text(text) current_line = @buffer_of_lines[@line_index] if current_line.bytesize == @byte_pointer current_line += text else current_line = byteinsert(current_line, @byte_pointer, text) end @byte_pointer += text.bytesize process_auto_indent end
52. 52 Handle input keys Render Wait inputs Initialize Reline.readline 001* if 1 002* 2 003* 3▪ @bu ff er_of_lines = ["if 1", " 2", " 3"] @line_index = 2 @byte_pointer = 3 @buffer_of_lines ←@line_index=2 ↑@byte_pointer=3
53. 53 # reline.rb private def inner_readline(prompt, add_hist, multiline, &confirm_multiline_termination) # Snip loop do read_io(config.keyseq_timeout) { |inputs| inputs.each { |key| line_editor.update(key) } } if line_editor.finished? line_editor.render_finished break else line_editor.rerender end end end Handle input keys Render Wait inputs Initialize Reline.readline finished? == false finished? == true
54. Follow the editing command 54 require "reline" text = Reline.readline puts text Enter "C-a"
55. 55 Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def normal_char(key) @buf << key.combined_char if @buf.size > 1 # multi byte keys = @buf.dup.force_encoding(@encoding) process_key(keys, nil) @multibyte_buffer.clear else # single byte return if key.char >= 128 method_symbol = @config.editing_mode.get_method( key.combined_char) process_key(key.combined_char, method_symbol) @multibyte_buffer.clear end end
56. 56 Handle input keys Render Wait inputs Initialize Reline.readline # emacs.rb class Reline::KeyActor::Emacs MAPPING = [ # 0 ^@ :em_set_mark, # 1 ^A :ed_move_to_beg, # snip # 97 a :ed_insert, key is 1 -> :ed_move_to_beg
57. 57 Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def ed_move_to_beg(_key) @byte_pointer = 0 end alias_method :beginning_of_line, :ed_move_to_beg alias_method :vi_zero, :ed_move_to_beg
58. Implementing Undo Functionality 58 Press "C-_" to undo actions. "C-_" is the default key binding for undo in GNU Readline.
59. What is Undo? Undo reverses previous actions. Example: Typing "abc" and then undoing reverts it to "ab". The unit of undo depends on the editor: GNU Readline likely uses keystroke timing. Zsh Line Editor considers each input as one unit. Reline treats each input as one unit. 59
60. Storing Operation History 60 A B C Operation
61. Storing Operation History 61 A B Operation C
62. Retrieving from Operation History 62 A B Undo C
63. Retrieving from Operation History 63 A B Undo C
64. 64 "" Current line past_lines
65. 65 Input "a" "" Current line past_lines
66. 66 "a" Current line Input "a" "" Old line add past_lines
67. 67 "a" Current line Input "a" "" Old line push past_lines ""
68. 68 "a" Current line Input "b" "" Old line past_lines ""
69. 69 "ab" Current line Input "b" "a" Old line past_lines "" add
70. 70 "ab" Current line Input "b" "a" Old line past_lines "" push "a"
71. 71 "ab" Current line Input "C-_" "a" Old line past_lines "" "a"
72. 72 "ab" Current line Input "C-_" "a" Old line past_lines "" "a" Undo
73. 73 "a" Current line Input "C-_" "a" Old line past_lines "" pop
74. Implementation details https://github.com/ruby/reline/pull/701 Restores both text and cursor position to previous states. 74
75. 75 Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def normal_char(key) @buf << key.combined_char if @buf.size > 1 # multi byte keys = @buf.dup.force_encoding(@encoding) process_key(keys, nil) @multibyte_buffer.clear else # single byte return if key.char >= 128 method_symbol = @config.editing_mode.get_method( key.combined_char) process_key(key.combined_char, method_symbol) @multibyte_buffer.clear end end method_symbol is :undo
76. 76 Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def undo(_key) return if @past_lines.empty? @undoing = true target_lines, target_cursor_x, target_cursor_y = @past_lines.last set_current_lines( target_lines, target_cursor_x, target_cursor_y ) @past_lines.pop end
77. Undo demo 77
78. Future work Implement Redo Missing features compared to GNU Readline Documentation 78
79. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan [1] https://bugs.ruby-lang.org/issues/19616 [2] https://github.com/ruby/ruby/blob/ cb636fe7075a260e4d47327764924a3ad10b502a/lib/ readline.rb Footnote 79
80. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Versions 80 Ruby 3.3.1 IRB 1.13.1 Reline 0.5.7 GNU Readline 8.2.10
81. May 15th - 17th, 2024 NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan Enjoy command line coding!