Slide 1
Slide 1 text
The new syntax highlighter
for Vim
4 Nov.
VimConf 2017
Slide 2
Slide 2 text
hi
Slide 3
Slide 3 text
:help pocke
Slide 4
Slide 4 text
:help pocke
● Masataka Kuwabara
● Vimmer / Rubyist
● Actcat, inc. / SideCI
● RuboCop's core developer
○ RuboCop is a static analyzer for Ruby.
Slide 5
Slide 5 text
My most starred Vim plug-in
is ...
Slide 6
Slide 6 text
sushibar.vim
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
Back to the topic
Slide 9
Slide 9 text
Agenda
Slide 10
Slide 10 text
Agenda
● Talk about syntax highlighters
○ Introduce the current syntax highlighter.
○ Introduce the new syntax highlighter.
■ I implement it
Slide 11
Slide 11 text
The current syntax highlighter
for Vim
Slide 12
Slide 12 text
What's the current syntax
highlighter?
Slide 13
Slide 13 text
:help syntax
Slide 14
Slide 14 text
:help syntax
Syntax highlighting enables Vim to show parts of the
text in another font or color.
Those parts can be specific keywords or text matching
a pattern.
Vim doesn't parse the whole file (to keep it fast), so the
highlighting has its limitations.
https://github.com/vim/vim/blob/1d68d9b2bd60d848552c08763e590edde16056c9/runtime/doc/syntax.txt#L9-L12
Slide 15
Slide 15 text
For example:
Slide 16
Slide 16 text
hello.vim
function! Hello(name) abort
echom 'Hello, ' . a:name . '-san!'
endfunction
Only two colors...
Understanding syntax of the code is difficult.
Slide 17
Slide 17 text
:syntax on
Slide 18
Slide 18 text
hello.vim
function! Hello(name) abort
echom 'Hello, ' . a:name . '-san!'
endfunction
Colorful
Slide 19
Slide 19 text
Useful!
Slide 20
Slide 20 text
But… The current highlighter
has problems
Slide 21
Slide 21 text
:help syntax
Syntax highlighting enables Vim to show parts of the
text in another font or color.
Those parts can be specific keywords or text matching
a pattern.
Vim doesn't parse the whole file (to keep it fast), so the
highlighting has its limitations.
https://github.com/vim/vim/blob/1d68d9b2bd60d848552c08763e590edde16056c9/runtime/doc/syntax.txt#L9-L12
Slide 22
Slide 22 text
For example: synmaxcol
● Vim gives up parsing very long line.
○ The limitation is configurable by
`synmaxcol`.
● DEMO
○ https://github.com/pocke/vimconf-2017/blob/mast
er/synmaxcol.json
Slide 23
Slide 23 text
And… the highlighter has
another problem
Slide 24
Slide 24 text
:help syntax
Syntax highlighting enables Vim to show parts of the
text in another font or color.
Those parts can be specific keywords or text matching
a pattern.
Vim doesn't parse the whole file (to keep it fast), so the
highlighting has its limitations.
https://github.com/vim/vim/blob/1d68d9b2bd60d848552c08763e590edde16056c9/runtime/doc/syntax.txt#L9-L12
Slide 25
Slide 25 text
Some people, when
confronted with a problem,
think “I know, I’ll use regular
expressions.” Now they have
two problems.
Jamie Zawinski http://regex.info/blog/2006-09-15/247
Slide 26
Slide 26 text
Regular expression is too
difficult for humans.
Slide 27
Slide 27 text
For example:
$ vim vim-ruby/syntax/ruby.vim
● vim-ruby defines syntax with many regular
expressions.
● DEMO
Slide 28
Slide 28 text
Regexp has another problem
Slide 29
Slide 29 text
For example: complex code
p(????::?:)
How should the code be highlight?(DEMO)
Slide 30
Slide 30 text
Conclusion of the Current Syntax
Highlighter
● The syntax highlighter has limitations for
performance.
○ e.g.) On long long line, on large .
● The syntax highlighter uses keywords and
regexp to highlight text.
○ The regexp is very difficult, and does not perfect.
Slide 31
Slide 31 text
Commercial: SideCI
Slide 32
Slide 32 text
No content
Slide 33
Slide 33 text
SideCI
● Automated Code Review Platform.
● Please access https://sideci.com !
● Sorry, we doesn't support Vim script…
Slide 34
Slide 34 text
Back to the topic
Slide 35
Slide 35 text
The new syntax highlighter
for Vim
Slide 36
Slide 36 text
Iro.vim(いろ.びむ)
https://github.com/pocke/iro.vim
Slide 37
Slide 37 text
Features of iro.vim
● The new syntax highlight plugin for Vim.
● Highlight perfectly with high performance
● Understandable code
● DEMO
Slide 38
Slide 38 text
Design of Iro.vim
Slide 39
Slide 39 text
Design
● Use `matchaddpos()` instead of `:syntax`.
● Use a parser that is implemented in the
target language to parse perfectly.
Slide 40
Slide 40 text
How iro.vim to work:
In Ruby case
Slide 41
Slide 41 text
Example: "Hello world" in Ruby
def foo
puts "Hello world!"
end
First, Iro.vim receives the code, and passes the
code to `iro#ruby#tokens()`.
Slide 42
Slide 42 text
iro#ruby#tokens()
code to tokens
def foo
puts "Hello world!"
end
[[[1, 0], :on_kw, "def"],
[[1, 3], :on_sp, " "],
[[1, 4], :on_ident, "foo"],
[[1, 7], :on_nl, "\n"],
[[2, 0], :on_sp, " "],
[[2, 2], :on_ident, "puts"],
[[2, 6], :on_sp, " "],
[[2, 7], :on_tstring_beg, "\""],
[[2, 8], :on_tstring_content, "Hello
world!"],
Ripper.lex(code)
Slide 43
Slide 43 text
iro#ruby#tokens()
tokens to highlights data
[[[1, 0], :on_kw, "def"],
[[1, 3], :on_sp, " "],
[[1, 4], :on_ident, "foo"],
[[1, 7], :on_nl, "\n"],
[[2, 0], :on_sp, " "],
[[2, 2], :on_ident, "puts"],
[[2, 6], :on_sp, " "],
[[2, 7], :on_tstring_beg, "\""],
[[2, 8], :on_tstring_content, "Hello
world!"],
{
"Keyword":[[1,1,3],[3,1,3]],
"String":[[2,9,12]],
"Delimiter":[[2,8,1], [2,21,1]]
}
Slide 44
Slide 44 text
Iro.vim
highlights with the data
{
"Keyword":[[1,1,3],[3,1,3]],
"String":[[2,9,12]],
"Delimiter":[[2,8,1], [2,21,1]]
}
def foo
puts "Hello world!"
end
Slide 45
Slide 45 text
Understandable code
Slide 46
Slide 46 text
A tour of iro.vim source code
Slide 47
Slide 47 text
autoload/iro.vim
● Define iro#redraw()
● load autoload/iro.vim.rb
● https://github.com/pocke/iro.vim/blob/mast
er/autoload/iro.vim
Slide 48
Slide 48 text
autoload/iro.vim.rb
● call matchaddpos() each highlights.
● https://github.com/pocke/iro.vim/blob/mast
er/autoload/iro.vim.rb
Slide 49
Slide 49 text
autoload/iro/ruby.vim.rb
● parse a ruby file, and generate highlight data
● https://github.com/pocke/iro.vim/blob/mast
er/autoload/iro/ruby.vim.rb
Slide 50
Slide 50 text
High Performance?
Slide 51
Slide 51 text
I have good news and bad
news.
Slide 52
Slide 52 text
Performance
● Bad news: A performance issue exists
○ Iro.vim is very slow on a ruby file that has 10,000+
lines(DEMO).
● Good news: Parsing is not a bottleneck
○ Vim is very slow when maybe many highlight is
registered by `matchaddpos()`.
Slide 53
Slide 53 text
Current status of iro.vim
Slide 54
Slide 54 text
Current status
● Still developing, but it works
● Support few languages.
○ Ruby, YAML and Python(experimental).
● Welcome Pull-Requests
Slide 55
Slide 55 text
:endslide
● Iro.vim is a new syntax highlighter.
○ It highlights perfectly
Thank you for listening