Slide 1
Slide 1 text
Exploring Reline:
Enhancing Command Line
Usability
Mari Imaizumi @ima1zumi
2024-05-15
Slide 2
Slide 2 text
2
IRB
Slide 3
Slide 3 text
IRB
Interactive Ruby
Default Ruby REPL (Read-Eval-Print Loop)
Enter Ruby code, eval, and display results
Run using the `irb` command
Default Rails console is IRB
3
Slide 4
Slide 4 text
IRB
4
Slide 5
Slide 5 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
Do you know
how the IRB works?
5
Slide 6
Slide 6 text
6
IRB Reline
Terminal
Emulator
Slide 7
Slide 7 text
7
Slide 8
Slide 8 text
8
Slide 9
Slide 9 text
About me
Mari Imaizumi
(@ima1zumi)
IRB and Reline committer
STORES, Inc.
9
Slide 10
Slide 10 text
Nursery Sponser
Speakers: LT:
Sponser booth (Ruby quiz presented by )
10
Slide 11
Slide 11 text
Agenda
What is Reline?
Feature Gap between Reline and GNU Readline
Understanding Reline
11
Slide 12
Slide 12 text
What is Reline?
Slide 13
Slide 13 text
Reline is a command line editor
A command line editor operates on terminal emulators,
manipulating user input
While not a common term, 'command line editor' is used
here due to the lack of a generic term for software like GNU
Readline.
Examples: GNU Readline, libedit, Reline
Reline has GNU Readline and editline compatibility
13
Slide 14
Slide 14 text
Without a Command Line Editor
14
puts "Please enter your name:"
text = gets.chomp
puts "You entered: #{text}"
Slide 15
Slide 15 text
Without a Command Line Editor
15
Cursor Back(CSI + D)
Slide 16
Slide 16 text
Use Reline
16
require "reline"
puts "Please enter your name:"
text = Reline.readline
puts "You entered: #{text}"
Slide 17
Slide 17 text
Use Reline
17
Slide 18
Slide 18 text
Command Line Editor
Provides a set of operations for editing entered text
Command line editors manage key operations in tools
like IRB and bash
18
Slide 19
Slide 19 text
Command Line Editors
GNU Readline
Used in bash, IRB (up to Ruby 2.6), GDB
NetBSD Editline Library (libedit)
Utilized in macOS
Reline
Used in IRB (Ruby 2.7 and above), debug, highline, hexapdf
19
Slide 20
Slide 20 text
Capabilities of GNU Readline
Line Editing
History management
Autocompletion
Editing Command
and more...
20
Slide 21
Slide 21 text
Feature Comparison
Line Editing History Autocompletion
Editing
Command
Others
GNU Readline ✅ ✅ ✅ ✅ ✅
Reline ✅ ✅ ✅ ˚ ˚
21
Slide 22
Slide 22 text
Capabilities of Reline
Display dialogs
Multiline input including prompts
Extensible with Ruby
Modifications to input lines, dialogs,
completion suggestions with procs
Completion suggestions also
possible with readline-ext
22
Slide 23
Slide 23 text
However, it lacks many features:
Key bindings
Editing commands
Configurable variables
and more
Issues are occasionally reported about functionalities available in Readline
but missing in Reline
Indicating user interest in these features
23
Reline is Compatible with
GNU Readline
Slide 24
Slide 24 text
From Ruby 3.3, readline-ext removed from default gems
If readline-ext is unavailable, require 'readline' will automatically
use Reline. [1][2]
Eliminates installation issues with GNU Readline during Ruby builds.
Increased need for Reline's compatibility features.
Users may unknowingly switch dependency from readline-ext to
Reline.
24
require 'readline' loads Reline
if missing readline-ext
Slide 25
Slide 25 text
My wishes
Hoping for Reline to be used instead of readline-ext.
Hope that users switch to Reline without noticing, ensuring a
seamless transition.
Aim for Reline to become the chosen command line editor
for users considering alternatives.
Add initial support for Reline on Ruby 3.3 #2298 · pry/pry
https://github.com/pry/pry/pull/2298
25
Slide 26
Slide 26 text
Challenges
Reline aims for GNU Readline compatibility; better
compatibility is desired.
Expectation for require 'readline' to seamlessly use
Reline.
26
Slide 27
Slide 27 text
Challenges
Reline aims for GNU Readline compatibility; better
compatibility is desired.
Expectation for require 'readline' to seamlessly use
Reline.
More GNU Readline features needed in Reline
What specific features are missing?
27
Slide 28
Slide 28 text
Feature Gap between
Reline and GNU Readline
Slide 29
Slide 29 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
Have you ever written
an .inputrc
fi
le?
🙋
29
Slide 30
Slide 30 text
What is .inputrc?
Configuration file for GNU Readline and Reline
~/.inputrc or INPUTRC=path/to/file
Configurable options:
Configurable variables
Editing commands
Key bindings
Set editing mode (emacs or vi)
30
Slide 31
Slide 31 text
Emacs mode, Vi mode
Emacs mode (default)
Vi mode
has command mode and insert mode
does not have a full set of vi editing functions
31
Slide 32
Slide 32 text
.inputrc
32
Variables
Key bindings
Slide 33
Slide 33 text
Key Bindings
Default key bindings are set in command line editor
For example, in Emacs mode, "\C-a" moves the cursor to the
beginning of the line
In Vi mode, there are different bindings for command mode
and insert mode
Customization is possible through .inputrc
Some settings written in .inputrc may not work with Reline
33
Slide 34
Slide 34 text
Con
fi
gurations usable in Reline
Variables: 12 / 46 (26%)
Editing commands: 53 / 120 (44%)
April 16, 2024
34
Slide 35
Slide 35 text
Importance Assessment
Wanted to know what settings are used how frequently
Conducted investigation using .inputrc files pushed to
GitHub
https://github.com/search?q=language%3AReadline&type=code
35
Slide 36
Slide 36 text
Usage Analysis
Investigated how many out of 166 configurable items
were mentioned in .inputrc
Includes commented-out entries
For items with default key bindings set, the presence
in .inputrc indicates usage
Total files examined: 6600
36
Slide 37
Slide 37 text
High usage
37
Slide 38
Slide 38 text
Low usage
38
Slide 39
Slide 39 text
Implemented Features
39
Undo
show-all-if-ambiguous
vi-editing-mode, emacs-editing-mode
mode condition in .inputrc
Bracketed paste mode (Implemented by tompng )
Slide 40
Slide 40 text
Understanding Reline
Slide 41
Slide 41 text
Exploring Reline Implementation
1. Enter "a"
2. Enter "C-a" (Control-a)
3. Implement the undo command
41
Slide 42
Slide 42 text
First Step: Follow the input text
42
require "reline"
text = Reline.readline
puts text
Enter "a\n"
Slide 43
Slide 43 text
First Step: Follow the input text
43
Slide 44
Slide 44 text
Flow
44
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
Slide 45
Slide 45 text
45
# reline.rb
# Simplified original code
private def inner_readline(prompt, add_hist,
multiline, &confirm_multiline_termination)
l = line_editor
l.reset(prompt, encoding: encoding)
# l.confirm_multiline_termination_proc =
confirm_multiline_termination
l.output = output
l.completion_proc = completion_proc
l.completion_append_character =
completion_append_character
l.output_modifier_proc = output_modifier_proc
l.prompt_proc = prompt_proc
l.auto_indent_proc = auto_indent_proc
l.dig_perfect_match_proc = dig_perfect_match_proc
pre_input_hook&.call
@dialog_proc_list.each_pair do |name, d|
l.add_dialog_proc(name,d.dialog_proc,d.context)
end
config.read
io_gate.set_default_key_bindings(config)
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
Slide 46
Slide 46 text
46
# reline.rb
private def inner_readline(prompt, add_hist,
multiline, &confirm_multiline_termination)
# Snip
loop do
read_io(config.keyseq_timeout) { |inputs|
inputs.each { |key| line_editor.update(key) }
}
if line_editor.finished?
line_editor.render_finished
break
else
line_editor.rerender
end
end
end
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
Slide 47
Slide 47 text
# ansi.rb
def self.inner_getc(timeout_second)
unless @@buf.empty?
return @@buf.shift
end
until @@input.wait_readable(0.01)
timeout_second -= 0.01
return nil if timeout_second <= 0
Reline.core.line_editor.handle_signal
end
c = @@input.getbyte
(c == 0x16 && @@input.raw(
min: 0,
time: 0,
&:getbyte)) || c
rescue Errno::EIO
# Maybe the I/O has been closed.
nil
rescue Errno::ENOTTY
nil
end
47
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
c is 97(Integer)
Slide 48
Slide 48 text
48
# reline.rb
private def inner_readline(prompt, add_hist,
multiline, &confirm_multiline_termination)
# Snip
loop do
read_io(config.keyseq_timeout) { |inputs|
inputs.each { |key| line_editor.update(key) }
}
if line_editor.finished?
line_editor.render_finished
break
else
line_editor.rerender
end
end
end
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
Slide 49
Slide 49 text
49
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
# line_editor.rb
private def normal_char(key)
@buf << key.combined_char
if @buf.size > 1 # multi byte
keys = @buf.dup.force_encoding(@encoding)
process_key(keys, nil)
@multibyte_buffer.clear
else # single byte
return if key.char >= 128
method_symbol =
@config.editing_mode.get_method(
key.combined_char)
process_key(key.combined_char, method_symbol)
@multibyte_buffer.clear
end
end
Slide 50
Slide 50 text
50
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
method_symbol is :ed_insert
# emacs.rb
class Reline::KeyActor::Emacs
MAPPING = [
# 0 ^@
:em_set_mark,
# 1 ^A
:ed_move_to_beg,
# snip
# 97 a
:ed_insert,
key is 97 ->
Slide 51
Slide 51 text
51
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
# line_editor.rb
private def ed_insert(key)
key.chr.encode(Encoding::UTF_8)
str = key.chr
insert_text(str)
end
def insert_text(text)
current_line = @buffer_of_lines[@line_index]
if current_line.bytesize == @byte_pointer
current_line += text
else
current_line =
byteinsert(current_line, @byte_pointer, text)
end
@byte_pointer += text.bytesize
process_auto_indent
end
Slide 52
Slide 52 text
52
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
001* if 1
002* 2
003* 3■
@bu
ff
er_of_lines = ["if 1", " 2", " 3"]
@line_index = 2
@byte_pointer = 3
@buffer_of_lines
←@line_index=2
↑@byte_pointer=3
Slide 53
Slide 53 text
53
# reline.rb
private def inner_readline(prompt, add_hist,
multiline, &confirm_multiline_termination)
# Snip
loop do
read_io(config.keyseq_timeout) { |inputs|
inputs.each { |key| line_editor.update(key) }
}
if line_editor.finished?
line_editor.render_finished
break
else
line_editor.rerender
end
end
end
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
finished? == false
finished? == true
Slide 54
Slide 54 text
Follow the editing command
54
require "reline"
text = Reline.readline
puts text
Enter "C-a"
Slide 55
Slide 55 text
55
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
# line_editor.rb
private def normal_char(key)
@buf << key.combined_char
if @buf.size > 1 # multi byte
keys = @buf.dup.force_encoding(@encoding)
process_key(keys, nil)
@multibyte_buffer.clear
else # single byte
return if key.char >= 128
method_symbol =
@config.editing_mode.get_method(
key.combined_char)
process_key(key.combined_char, method_symbol)
@multibyte_buffer.clear
end
end
Slide 56
Slide 56 text
56
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
# emacs.rb
class Reline::KeyActor::Emacs
MAPPING = [
# 0 ^@
:em_set_mark,
# 1 ^A
:ed_move_to_beg,
# snip
# 97 a
:ed_insert,
key is 1 -> :ed_move_to_beg
Slide 57
Slide 57 text
57
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
# line_editor.rb
private def ed_move_to_beg(_key)
@byte_pointer = 0
end
alias_method :beginning_of_line, :ed_move_to_beg
alias_method :vi_zero, :ed_move_to_beg
Slide 58
Slide 58 text
Implementing Undo Functionality
58
Press "C-_" to undo actions.
"C-_" is the default key binding for undo in GNU Readline.
Slide 59
Slide 59 text
What is Undo?
Undo reverses previous actions.
Example: Typing "abc" and then undoing reverts it to "ab".
The unit of undo depends on the editor:
GNU Readline likely uses keystroke timing.
Zsh Line Editor considers each input as one unit.
Reline treats each input as one unit.
59
Slide 60
Slide 60 text
Storing Operation History
60
A B C
Operation
Slide 61
Slide 61 text
Storing Operation History
61
A B
Operation
C
Slide 62
Slide 62 text
Retrieving from Operation
History
62
A B
Undo
C
Slide 63
Slide 63 text
Retrieving from Operation
History
63
A B
Undo
C
Slide 64
Slide 64 text
64
""
Current line
past_lines
Slide 65
Slide 65 text
65
Input
"a"
""
Current line
past_lines
Slide 66
Slide 66 text
66
"a"
Current line Input
"a"
""
Old line
add
past_lines
Slide 67
Slide 67 text
67
"a"
Current line Input
"a"
""
Old line
push
past_lines
""
Slide 68
Slide 68 text
68
"a"
Current line Input
"b"
""
Old line
past_lines
""
Slide 69
Slide 69 text
69
"ab"
Current line Input
"b"
"a"
Old line
past_lines
""
add
Slide 70
Slide 70 text
70
"ab"
Current line Input
"b"
"a"
Old line
past_lines
""
push
"a"
Slide 71
Slide 71 text
71
"ab"
Current line Input
"C-_"
"a"
Old line
past_lines
"" "a"
Slide 72
Slide 72 text
72
"ab"
Current line Input
"C-_"
"a"
Old line
past_lines
"" "a"
Undo
Slide 73
Slide 73 text
73
"a"
Current line Input
"C-_"
"a"
Old line
past_lines
""
pop
Slide 74
Slide 74 text
Implementation details
https://github.com/ruby/reline/pull/701
Restores both text and cursor position to previous states.
74
Slide 75
Slide 75 text
75
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
# line_editor.rb
private def normal_char(key)
@buf << key.combined_char
if @buf.size > 1 # multi byte
keys = @buf.dup.force_encoding(@encoding)
process_key(keys, nil)
@multibyte_buffer.clear
else # single byte
return if key.char >= 128
method_symbol =
@config.editing_mode.get_method(
key.combined_char)
process_key(key.combined_char, method_symbol)
@multibyte_buffer.clear
end
end method_symbol is
:undo
Slide 76
Slide 76 text
76
Handle input keys
Render
Wait inputs
Initialize
Reline.readline
# line_editor.rb
private def undo(_key)
return if @past_lines.empty?
@undoing = true
target_lines,
target_cursor_x,
target_cursor_y = @past_lines.last
set_current_lines(
target_lines,
target_cursor_x,
target_cursor_y
)
@past_lines.pop
end
Slide 77
Slide 77 text
Undo demo
77
Slide 78
Slide 78 text
Future work
Implement Redo
Missing features compared to GNU Readline
Documentation
78
Slide 79
Slide 79 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
[1] https://bugs.ruby-lang.org/issues/19616
[2] https://github.com/ruby/ruby/blob/
cb636fe7075a260e4d47327764924a3ad10b502a/lib/
readline.rb
Footnote
79
Slide 80
Slide 80 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
Versions
80
Ruby 3.3.1
IRB 1.13.1
Reline 0.5.7
GNU Readline 8.2.10
Slide 81
Slide 81 text
May 15th - 17th, 2024
NAHA CULTURAL ARTS THEATER NAHArt, Okinawa, Japan
Enjoy command line coding!