$30 off During Our Annual Pro Sale. View Details »

The new syntax highlighter for Vim

pocke
November 04, 2017

The new syntax highlighter for Vim

VimConf 2017

pocke

November 04, 2017
Tweet

More Decks by pocke

Other Decks in Programming

Transcript

  1. The new syntax highlighter for Vim 4 Nov. VimConf 2017

  2. hi

  3. :help pocke

  4. :help pocke • Masataka Kuwabara • Vimmer / Rubyist •

    Actcat, inc. / SideCI • RuboCop's core developer ◦ RuboCop is a static analyzer for Ruby.
  5. My most starred Vim plug-in is ...

  6. sushibar.vim

  7. None
  8. Back to the topic

  9. Agenda

  10. Agenda • Talk about syntax highlighters ◦ Introduce the current

    syntax highlighter. ◦ Introduce the new syntax highlighter. ▪ I implement it
  11. The current syntax highlighter for Vim

  12. What's the current syntax highlighter?

  13. :help syntax

  14. :help syntax Syntax highlighting enables Vim to show parts of

    the text in another font or color. Those parts can be specific keywords or text matching a pattern. Vim doesn't parse the whole file (to keep it fast), so the highlighting has its limitations. https://github.com/vim/vim/blob/1d68d9b2bd60d848552c08763e590edde16056c9/runtime/doc/syntax.txt#L9-L12
  15. For example:

  16. hello.vim function! Hello(name) abort echom 'Hello, ' . a:name .

    '-san!' endfunction Only two colors... Understanding syntax of the code is difficult.
  17. :syntax on

  18. hello.vim function! Hello(name) abort echom 'Hello, ' . a:name .

    '-san!' endfunction Colorful
  19. Useful!

  20. But… The current highlighter has problems

  21. :help syntax Syntax highlighting enables Vim to show parts of

    the text in another font or color. Those parts can be specific keywords or text matching a pattern. Vim doesn't parse the whole file (to keep it fast), so the highlighting has its limitations. https://github.com/vim/vim/blob/1d68d9b2bd60d848552c08763e590edde16056c9/runtime/doc/syntax.txt#L9-L12
  22. For example: synmaxcol • Vim gives up parsing very long

    line. ◦ The limitation is configurable by `synmaxcol`. • DEMO ◦ https://github.com/pocke/vimconf-2017/blob/mast er/synmaxcol.json
  23. And… the highlighter has another problem

  24. :help syntax Syntax highlighting enables Vim to show parts of

    the text in another font or color. Those parts can be specific keywords or text matching a pattern. Vim doesn't parse the whole file (to keep it fast), so the highlighting has its limitations. https://github.com/vim/vim/blob/1d68d9b2bd60d848552c08763e590edde16056c9/runtime/doc/syntax.txt#L9-L12
  25. Some people, when confronted with a problem, think “I know,

    I’ll use regular expressions.” Now they have two problems. Jamie Zawinski http://regex.info/blog/2006-09-15/247
  26. Regular expression is too difficult for humans.

  27. For example: $ vim vim-ruby/syntax/ruby.vim • vim-ruby defines syntax with

    many regular expressions. • DEMO
  28. Regexp has another problem

  29. For example: complex code p(????::?:) How should the code be

    highlight?(DEMO)
  30. Conclusion of the Current Syntax Highlighter • The syntax highlighter

    has limitations for performance. ◦ e.g.) On long long line, on large <script>. • The syntax highlighter uses keywords and regexp to highlight text. ◦ The regexp is very difficult, and does not perfect.
  31. Commercial: SideCI

  32. None
  33. SideCI • Automated Code Review Platform. • Please access https://sideci.com

    ! • Sorry, we doesn't support Vim script…
  34. Back to the topic

  35. The new syntax highlighter for Vim

  36. Iro.vim(いろ.びむ) https://github.com/pocke/iro.vim

  37. Features of iro.vim • The new syntax highlight plugin for

    Vim. • Highlight perfectly with high performance • Understandable code • DEMO
  38. Design of Iro.vim

  39. Design • Use `matchaddpos()` instead of `:syntax`. • Use a

    parser that is implemented in the target language to parse perfectly.
  40. How iro.vim to work: In Ruby case

  41. Example: "Hello world" in Ruby def foo puts "Hello world!"

    end First, Iro.vim receives the code, and passes the code to `iro#ruby#tokens()`.
  42. iro#ruby#tokens() code to tokens def foo puts "Hello world!" end

    [[[1, 0], :on_kw, "def"], [[1, 3], :on_sp, " "], [[1, 4], :on_ident, "foo"], [[1, 7], :on_nl, "\n"], [[2, 0], :on_sp, " "], [[2, 2], :on_ident, "puts"], [[2, 6], :on_sp, " "], [[2, 7], :on_tstring_beg, "\""], [[2, 8], :on_tstring_content, "Hello world!"], Ripper.lex(code)
  43. iro#ruby#tokens() tokens to highlights data [[[1, 0], :on_kw, "def"], [[1,

    3], :on_sp, " "], [[1, 4], :on_ident, "foo"], [[1, 7], :on_nl, "\n"], [[2, 0], :on_sp, " "], [[2, 2], :on_ident, "puts"], [[2, 6], :on_sp, " "], [[2, 7], :on_tstring_beg, "\""], [[2, 8], :on_tstring_content, "Hello world!"], { "Keyword":[[1,1,3],[3,1,3]], "String":[[2,9,12]], "Delimiter":[[2,8,1], [2,21,1]] }
  44. Iro.vim highlights with the data { "Keyword":[[1,1,3],[3,1,3]], "String":[[2,9,12]], "Delimiter":[[2,8,1], [2,21,1]]

    } def foo puts "Hello world!" end
  45. Understandable code

  46. A tour of iro.vim source code

  47. autoload/iro.vim • Define iro#redraw() • load autoload/iro.vim.rb • https://github.com/pocke/iro.vim/blob/mast er/autoload/iro.vim

  48. autoload/iro.vim.rb • call matchaddpos() each highlights. • https://github.com/pocke/iro.vim/blob/mast er/autoload/iro.vim.rb

  49. autoload/iro/ruby.vim.rb • parse a ruby file, and generate highlight data

    • https://github.com/pocke/iro.vim/blob/mast er/autoload/iro/ruby.vim.rb
  50. High Performance?

  51. I have good news and bad news.

  52. Performance • Bad news: A performance issue exists ◦ Iro.vim

    is very slow on a ruby file that has 10,000+ lines(DEMO). • Good news: Parsing is not a bottleneck ◦ Vim is very slow when maybe many highlight is registered by `matchaddpos()`.
  53. Current status of iro.vim

  54. Current status • Still developing, but it works • Support

    few languages. ◦ Ruby, YAML and Python(experimental). • Welcome Pull-Requests
  55. :endslide • Iro.vim is a new syntax highlighter. ◦ It

    highlights perfectly Thank you for listening