Exploring Reline: Enhancing Command Line Usability

Slide 1

Slide 1 text

Mari Imaizumi Exploring Reline Enhancing Command Line Usability RubyConf 2024

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

IRB

Slide 4

Slide 4 text

IRB • Interactive Ruby • Default Ruby REPL (Read-Eval-Print Loop) • Run using the `irb` command • Rails console defaults to IRB 4

Slide 5

Slide 5 text

IRB 5

Slide 6

Slide 6 text

Do you know how the IRB works? 6

Slide 7

Slide 7 text

IRB Reline Terminal Emulator 7

Slide 8

Slide 8 text

8 Tip: repl_type_completor will become a bundled gem starting with Ruby 3.4.

Slide 9

Slide 9 text

9 Tip: You can customize the dialog color. See the details in Reline’s README.

Slide 10

Slide 10 text

• Mari Imaizumi (@ima1zumi) • She/her • IRB and Reline committer • STORES, Inc. About me 10

Slide 11

Slide 11 text

Agenda • What is Reline? • Feature Gap between Reline and GNU Readline • Understanding Reline 11

Slide 12

Slide 12 text

What is Reline?

Slide 13

Slide 13 text

Reline is a command line editor • A command line editor runs on terminal emulators, handling user input. • Examples: GNU Readline, libedit, Reline • Reline is compatible with GNU Readline and libedit. • Ruby’s default gem (default before Ruby 3.3, bundled gem from Ruby 3.4). 13 Although not a common term, ‘command line editor’ is used here because there is no general term for software like GNU Readline.

Slide 14

Slide 14 text

Without a Command Line Editor puts "Please enter your name:" text = gets.chomp puts "You entered: #{text}" 14

Slide 15

Slide 15 text

Without a Command Line Editor 15 This is the cursor-back escape sequence (CSI + D)

Slide 16

Slide 16 text

Use Reline require "reline" puts "Please enter your name:" text = Reline.readline puts "You entered: #{text}" 16

Slide 17

Slide 17 text

Use Reline 17

Slide 18

Slide 18 text

Command Line Editor • Provides a set of operations for editing entered text • Command line editors manage key operations in tools like IRB and bash 18

Slide 19

Slide 19 text

Command Line Editors • GNU Readline • Used in bash, IRB (up to Ruby 2.6), GDB • NetBSD Editline Library (libedit) • Utilized in macOS • Reline • Used in IRB (Ruby 2.7 and above), debug, highline, hexapdf 19

Slide 20

Slide 20 text

Capabilities of GNU Readline • Line Editing • History management • Autocompletion • Editing Command • and more... 20

Slide 21

Slide 21 text

Feature Comparison Line Editing History Autocompletion Editing Command Others GNU Readline ✅ ✅ ✅ ✅ ✅ Reline ✅ ✅ ✅ ˚ ˚ 21

Slide 22

Slide 22 text

Capabilities of Reline • Display dialogs • Multiline input including prompts • Extensible with Ruby 22

Slide 23

Slide 23 text

• However, it lacks many features: • Key bindings • Editing commands • Con fi gurable variables • and more • Issues are occasionally reported about functionalities available in Readline but missing in Reline • Indicating user interest in these features Reline is Compatible with GNU Readline 23

Slide 24

Slide 24 text

• From Ruby 3.3, readline-ext removed from default gems • If readline-ext is unavailable, require 'readline' will automatically use Reline. [1][2] • Eliminates installation issues with GNU Readline during Ruby builds. • Increased need for Reline's compatibility features. • Users may unknowingly switch dependency from readline-ext to Reline. require 'readline' loads Reline if missing readline-ext 24

Slide 25

Slide 25 text

My wishes • Hoping for Reline to be used instead of readline-ext. • Hope that users switch to Reline without noticing, ensuring a seamless transition. • Aim for Reline to become the chosen command line editor for users considering alternatives. • Add initial support for Reline on Ruby 3.3 #2298 · pry/pry https:// github.com/pry/pry/pull/2298 25

Slide 26

Slide 26 text

Challenges • Reline aims for GNU Readline compatibility; better compatibility is desired. • Expectation for require 'readline' to seamlessly use Reline. • What speci fi c features are missing? 26

Slide 27

Slide 27 text

Feature Gap between Reline and GNU Readline

Slide 28

Slide 28 text

Have you ever written an .inputrc? 🙋

Slide 29

Slide 29 text

What is .inputrc? • Con fi guration fi le for GNU Readline and Reline • ~/.inputrc or INPUTRC=path/to/ fi le • Con fi gurable options: • Con fi gurable variables • Editing commands • Key bindings • Set editing mode (emacs or vi) 29

Slide 30

Slide 30 text

Emacs mode, Vi mode • Emacs mode (default) • Vi mode • has command mode and insert mode • does not have a full set of vi editing functions 30 I'm a Vimmer but I use Emacs mode. :p

Slide 31

Slide 31 text

.inputrc Variables Key bindings 31

Slide 32

Slide 32 text

Key Bindings • Default key bindings are set in command line editor • For example, in Emacs mode, "\C-a" moves the cursor to the beginning of the line • In Vi mode, there are di ff erent bindings for command mode and insert mode • Customization is possible through .inputrc • Some settings written in .inputrc may not work with Reline 32

Slide 33

Slide 33 text

Configurations usable in Reline • Variables: 12 / 46 (26%) • Editing commands: 53 / 120 (44%) • April 16, 2024 33

Slide 34

Slide 34 text

Importance Assessment • Wanted to know what settings are used how frequently • Conducted investigation using .inputrc fi les pushed to GitHub • https://github.com/search?q=language%3AReadline&type=code 34

Slide 35

Slide 35 text

Usage Analysis • Investigated how many out of 166 con fi gurable items were mentioned in .inputrc • Includes commented-out entries • For items with default key bindings set, the presence in .inputrc indicates usage • Total fi les examined: 6600 35

Slide 36

Slide 36 text

High usage 36

Slide 37

Slide 37 text

Low usage 37

Slide 38

Slide 38 text

Implemented Features Undo, Redo show-all-if-ambiguous vi-editing-mode, emacs-editing-mode mode condition in .inputrc Bracketed paste mode 38

Slide 39

Slide 39 text

Understanding Reline

Slide 40

Slide 40 text

Exploring Reline Implementation • 1. Enter "a" • 2. Enter "C-a" (Control-a) • 3. Implement the undo command 40

Slide 41

Slide 41 text

First Step: Follow the input text require "reline" text = Reline.readline puts text • Enter "a\n" 41

Slide 42

Slide 42 text

First Step: Follow the input text 42

Slide 43

Slide 43 text

Flow Handle input keys Render Wait inputs Initialize Reline.readline 43

Slide 44

Slide 44 text

# reline.rb # Simplified original code private def inner_readline(prompt, add_hist, multiline, &confirm_multiline_termination) l = line_editor l.reset(prompt, encoding: encoding) # l.confirm_multiline_termination_proc = confirm_multiline_termination l.output = output l.completion_proc = completion_proc l.completion_append_character = completion_append_character l.output_modifier_proc = output_modifier_proc l.prompt_proc = prompt_proc l.auto_indent_proc = auto_indent_proc l.dig_perfect_match_proc = dig_perfect_match_proc pre_input_hook&.call @dialog_proc_list.each_pair do |name, d| l.add_dialog_proc(name,d.dialog_proc,d.context) end config.read io_gate.set_default_key_bindings(config) Handle input keys Render Wait inputs Initialize Reline.readline 44

Slide 45

Slide 45 text

# reline.rb private def inner_readline(prompt, add_hist, multiline, &confirm_multiline_termination) # Snip loop do read_io(config.keyseq_timeout) { |inputs| inputs.each { |key| line_editor.update(key) } } if line_editor.finished? line_editor.render_finished break else line_editor.rerender end end end Handle input keys Render Wait inputs Initialize Reline.readline 45

Slide 46

Slide 46 text

# ansi.rb def self.inner_getc(timeout_second) unless @@buf.empty? return @@buf.shift end until @@input.wait_readable(0.01) timeout_second -= 0.01 return nil if timeout_second <= 0 Reline.core.line_editor.handle_signal end c = @@input.getbyte (c == 0x16 && @@input.raw( min: 0, time: 0, &:getbyte)) || c rescue Errno::EIO # Maybe the I/O has been closed. nil rescue Errno::ENOTTY nil end Handle input keys Render Wait inputs Initialize Reline.readline c is 97(Integer) 46

Slide 47

Slide 47 text

Slide 48

Slide 48 text

Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def normal_char(key) @buf << key.combined_char if @buf.size > 1 # multi byte keys = @buf.dup.force_encoding(@encoding) process_key(keys, nil) @multibyte_buffer.clear else # single byte return if key.char >= 128 method_symbol = @config.editing_mode.get_method( key.combined_char) process_key(key.combined_char, method_symbol) @multibyte_buffer.clear end end 48

Slide 49

Slide 49 text

Handle input keys Render Wait inputs Initialize Reline.readline method_symbol is :ed_insert # emacs.rb class Reline::KeyActor::Emacs MAPPING = [ # 0 ^@ :em_set_mark, # 1 ^A :ed_move_to_beg, # snip # 97 a :ed_insert, key is 97 -> 49

Slide 50

Slide 50 text

Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def ed_insert(key) key.chr.encode(Encoding::UTF_8) str = key.chr insert_text(str) end def insert_text(text) current_line = @buffer_of_lines[@line_index] if current_line.bytesize == @byte_pointer current_line += text else current_line = byteinsert(current_line, @byte_pointer, text) end @byte_pointer += text.bytesize process_auto_indent end 50

Slide 51

Slide 51 text

Handle input keys Render Wait inputs Initialize Reline.readline 001* if 1 002* 2 003* 3■ @bu ff er_of_lines = ["if 1", " 2", " 3"] @line_index = 2 @byte_pointer = 3 @bu ff er_of_lines ←@line_index=2 ↑@byte_pointer=3 51

Slide 52

Slide 52 text

Slide 53

Slide 53 text

Follow the editing command require "reline" text = Reline.readline puts text • Enter "C-a" 53

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Handle input keys Render Wait inputs Initialize Reline.readline # emacs.rb class Reline::KeyActor::Emacs MAPPING = [ # 0 ^@ :em_set_mark, # 1 ^A :ed_move_to_beg, # snip # 97 a :ed_insert, key is 1 -> :ed_move_to_beg 55

Slide 56

Slide 56 text

Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def ed_move_to_beg(_key) @byte_pointer = 0 end alias_method :beginning_of_line, :ed_move_to_beg alias_method :vi_zero, :ed_move_to_beg 56

Slide 57

Slide 57 text

Implementing Undo Functionality • Press "C-_" to undo actions. • "C-_" is the default key binding for undo in GNU Readline. 57

Slide 58

Slide 58 text

What is Undo? • Undo reverses previous actions. • Example: Typing "abc" and then undoing reverts it to "ab". • The unit of undo depends on the editor: • GNU Readline likely uses keystroke timing. • Zsh Line Editor considers each input as one unit. • Reline treats each input as one unit. 58

Slide 59

Slide 59 text

Storing Operation History A B C Operation 59

Slide 60

Slide 60 text

Storing Operation History A B Operation C 60

Slide 61

Slide 61 text

Retrieving from Operation History A B Undo C 61

Slide 62

Slide 62 text

Retrieving from Operation History A B Undo C 62

Slide 63

Slide 63 text

"" Current line past_lines 63

Slide 64

Slide 64 text

Input "a" "" Current line past_lines 64

Slide 65

Slide 65 text

"a" Current line "a" "" add 65 past_lines Old line Input

Slide 66

Slide 66 text

"a" "a" "" push "" 66 past_lines Old line Current line Input

Slide 67

Slide 67 text

"a" "b" "" "" 67 past_lines Old line Current line Input

Slide 68

Slide 68 text

"ab" "b" "a" "" add 68 past_lines Old line Current line Input

Slide 69

Slide 69 text

"ab" "b" "a" "" push "a" 69 past_lines Old line Current line Input

Slide 70

Slide 70 text

"ab" "C-_" "a" "" "a" 70 past_lines Old line Current line Input

Slide 71

Slide 71 text

"ab" "C-_" "a" "" "a" Undo 71 past_lines Old line Current line Input

Slide 72

Slide 72 text

"a" "C-_" "a" "" pop 72 past_lines Old line Current line Input

Slide 73

Slide 73 text

Implementation details • https://github.com/ruby/reline/pull/701 • Restores both text and cursor position to previous states. 73

Slide 74

Slide 74 text

Slide 75

Slide 75 text

Handle input keys Render Wait inputs Initialize Reline.readline # line_editor.rb private def undo(_key) return if @past_lines.empty? @undoing = true target_lines, target_cursor_x, target_cursor_y = @past_lines.last set_current_lines( target_lines, target_cursor_x, target_cursor_y ) @past_lines.pop end 75

Slide 76

Slide 76 text

Undo demo 76

Slide 77

Slide 77 text

Summary • Reline is a pure Ruby gem compatible with GNU Readline and libedit. • It provides line-editing features and is extensible in Ruby. • require "readline-ext" loads Reline if readline-ext is missing. • For feature requests or bug reports, feel free to post on the ruby/reline repo.

Slide 78

Slide 78 text

Footnote • [1] https://bugs.ruby-lang.org/issues/19616 • [2] https://github.com/ruby/ruby/blob/ cb636fe7075a260e4d47327764924a3ad10b502a/lib/readline.rb 78

Slide 79

Slide 79 text

Versions • Ruby 3.3.6 • IRB 1.14.1 • Reline 0.5.11 • GNU Readline 8.2.10 79

Slide 80

Slide 80 text

Enjoy command line coding!