$30 off During Our Annual Pro Sale. View Details »

The Past and Future of Vim-go

Fatih Arslan
November 04, 2017

The Past and Future of Vim-go

How did vim-go started and evolved throughout the years

Fatih Arslan

November 04, 2017
Tweet

More Decks by Fatih Arslan

Other Decks in Technology

Transcript

  1. The Past and Future
    of Vim-go
    Fatih Arslan
    Sr. Software Engineer @DigitalOcean

    View Slide

  2. Me
    • Sr. Software Engineer @DigitalOcean
    • Creator of vim-go
    • Go contributor, author of many popular Go
    packages (i.e: color, structs, etc..)
    • Tool maker (i.e: gomodifytags, motion, etc...)
    • Coffee and bag geek

    View Slide

  3. Go
    Programming
    Language

    View Slide

  4. Go programming language
    • Started in 2007 with Rob Pike, Ken Thompson and Robert Griesemer
    • Open Source in 10 November, 2009

    • Go 1.0 released in 28 March, 2012

    • Simple, reliable and efficient

    View Slide

  5. It started 8 years ago (2009)

    View Slide

  6. View Slide

  7. One year later (2010)

    View Slide

  8. First commands (2011)
    :Import strings
    :ImportAs . strings
    :Drop strings

    View Slide

  9. gofmt support was added (2011)

    View Slide

  10. autoload/go folder (2011)

    View Slide

  11. :make support (2013)

    View Slide

  12. misc/vim removed from Go (2014)

    View Slide

  13. Last state of misc/vim
    • Some commands to manipulate the source

    • :Fmt, :Import, etc...

    • Syntax, compiler and indent support

    • Go file detection

    • Proper readme
    (before it was removed)

    View Slide

  14. vim-go
    Go development
    plugin for vim

    View Slide

  15. github.com/fatih/vim-go

    View Slide

  16. How and why was
    vim-go created?

    View Slide

  17. Too many (small)
    plugins

    View Slide

  18. Too many plugins
    • https://github.com/Blackrush/vim-gocode

    • https://github.com/fsouza/go.vim

    • https://github.com/dgryski/vim-godef

    • Go's misc/vim

    • etc...

    View Slide

  19. No clear roadmap

    View Slide

  20. misc/vim was
    missing features

    View Slide

  21. Flaws
    • No coherent feature set, every plugin had its own commands, etc..

    • No docs/ folder

    • No code/build/test commands (only :make).

    • Binary paths are hard coded

    • No coherent UI across all plugin commands, some look under the cursor some do
    not

    • No autocompletion

    • etc...

    View Slide

  22. A plan to fix it!

    View Slide

  23. A plan to fix it
    1. Start using misc/vim

    2. Integrate all plugins

    3. Fix flaws & add missing features

    4. Announce it

    View Slide

  24. View Slide

  25. 2014

    View Slide

  26. First vim-go commit

    View Slide

  27. Naming is hard!

    View Slide

  28. Naming is hard
    • First, it was called golang.vim

    • Then renamed to go.vim (Because golang was wrong)

    • Finally changed to vim-go as I didn't like the .vim
    extension

    View Slide

  29. Lack of documentation

    View Slide

  30. Lack of documentation
    • Added docs/vim-go.txt file

    • Improved installations steps to include pathogen, vim-plug, etc...

    • Added current existing features

    • Added information about commands and settings

    View Slide

  31. Writing help files is not fun

    View Slide

  32. View Slide

  33. Usability fixes

    View Slide

  34. Usability fixes
    • One command to download and install tools :GoInstallBinaries

    • Coherent command interface (i.e: :GoFoo without argument and with
    argument)

    • Improved commands by using (for example :GoBuild! to avoid
    jumping to first error)

    • Added :Go prefix to all commands, i.e. :Import > :GoImport

    • Added go_ prefix to all global variables, i.e. g:bin_path > g:go_bin_path

    View Slide

  35. View Slide

  36. Under the hood
    improvements

    View Slide

  37. Under the hood improvements
    • Prevent :make to produce binary

    • Fixed not losing history on format (gofmt)

    • Still problematic

    • Added mappings for provide custom mappings

    • Open cwindow after error or warning

    View Slide

  38. Features & improvements

    View Slide

  39. View Slide

  40. New features: tooling
    • :GoRun, :GoBuild, :GoTest- run, build or test your file

    • :GoLint, :GoVet, :GoErrcheck - linters and checkers

    • :GoCoverage - show code coverage (initial plugin by Yukinari Toyota)
    • :GoPlay - share your buffer to play.golang.org

    View Slide

  41. New features: tooling (cont)
    • :GoDoc - show package documentation

    • :GoInfo - show identifier information under the cursor

    • :GoDef - jump to a symbol/declaration (inital plugin by Damian Gryski)

    • :GoRename - refactor identifiers

    • ...

    View Slide

  42. New features: misc
    • Ultisnippet/Neosnippet integration
    • Run :GoFmt automatically on save
    • Windows OS support
    • Started to work on static analysis (via Oracle)
    • tagbar integration
    • ...

    View Slide

  43. GoCoverage
    demo

    View Slide

  44. 2015

    View Slide

  45. First release v1.0

    View Slide

  46. After 1347 commits

    View Slide

  47. Versioning changed

    View Slide

  48. Versioniong
    • We started with v1.0

    • Was increasing on patch levels:

    • v1.0.1, v1.0.2, v1.0.2

    • This was wrong, switched to minor levels:

    • v1.1, v1.2, v1.3, etc..

    • Allowed us to release patch releases if needed (i.e v1.3.1)

    View Slide

  49. Added Kindle e-book
    donation option

    View Slide

  50. Kindle e-book donation (2014)

    View Slide

  51. Kindle e-book donation
    • People started asking how to donate

    • No money was needed, but I was reading a lot

    • Added public Kindle wish list
    • Received over 40 books

    View Slide

  52. License added

    View Slide

  53. License added
    • Distros started packaging vim-go

    • No license meant no clear guidance
    • Added BSD 3-Clause license

    • Same as Go's own license

    • It's a very permissive license, but the name "vim-go" can't be used for
    advertisement or promotions.

    View Slide

  54. New
    features & improvements

    View Slide

  55. Improvements & features
    • Oracle support enabled (static analysis)

    • Show function callees, callers

    • Show all references of an identifier

    • etc...

    • Improved syntax highlighting even more (build constraints, all operators)

    • Automatic GOPATH detection

    • gb, Godeps support

    View Slide

  56. New Commands
    • :GoMetalinter (combines :GoLint, :GoErrcheck, :GoVet)

    • :GoTestCompile, :GoTestFunc

    • Makes testing workflow more efficient

    • :GoPath

    • Show or change GOPATH

    View Slide

  57. 2016

    View Slide

  58. NeoVim

    View Slide

  59. NeoVim
    • Everything Async

    • :GoBuild, :GoTest, etc..

    • Statusline integration for commands (feedback loop)

    • Terminal support for :GoRun and :GoTest

    View Slide

  60. View Slide

  61. Donation changed to Patreon

    View Slide

  62. View Slide

  63. Changed donation to Patreon
    • I had now so many Kindle books, I couldn't finish them

    • Started 1.5 years ago around April 2016
    • $5, $10, $25 and $100 tiers

    • Most used is $5 tier

    View Slide

  64. Donations per month

    View Slide

  65. Why donation
    • Open source is free to use, but doesn't mean it has free value
    • We spend our own time, a value that we can't buy

    • Donation helps us to buy value in other areas

    View Slide

  66. Vim 8.0

    View Slide

  67. Vim 8.0
    • Everything Async for Vim!

    • :GoBuild, :GoTest, etc..

    • Better Statusline integration
    • A lot of Vim bugs in the beginning, wasn't quite ready

    • JSON encode/decode was useful for Go tooling integration

    View Slide

  68. View Slide

  69. has_job()

    View Slide

  70. View Slide

  71. Using tools
    written in Go

    View Slide

  72. From Vim script to Go
    • Logic is replaced by a binary, written in Go

    • More and more features are reimplemented again

    • Vim calls it via system() or job_start()

    • Tools return JSON or Vim script output (but not all of them)

    View Slide

  73. View Slide

  74. The power of AST
    • Go has an excellent parser & scanner packages
    • Allows us to modify Go source code

    • Parser is very fast. It's not noticeable at all.

    • Direct access to AST (Abstract Syntax Tree)

    View Slide

  75. AST based feature

    View Slide

  76. package main
    type Server struct {
    Name string
    Port int
    EnableLogs bool
    BaseDomain string
    Credentials struct {
    Username string
    Password string
    }
    }
    {
    "start": 3,
    "end": 12,
    "lines": [
    "type Server struct {",
    " Name string `xml:\"name\"`",
    " Port int `xml:\"port\"`",
    " EnableLogs bool `xml:\"enable_logs\"`",
    " BaseDomain string `xml:\"base_domain\"`",
    " Credentials struct {",
    " Username string `xml:\"username\"`",
    " Password string `xml:\"password\"`",
    " } `xml:\"credentials\"`",
    "}"
    ]
    }
    Input
    JSON Output
    $ gomodifytags

    -file example.go

    -struct Server

    -add-tags json

    -format json

    View Slide

  77. View Slide

  78. Demo time!
    (selecting functions)

    View Slide

  79. Advantages of AST based text objects
    • Anonymous functions are supported

    • One liner functions can be selected easier

    • Cursor position can be anywhere as long as it makes sense

    • Comments are treated as a part of the function declaration

    View Slide

  80. OS differences
    • Supporting Windows, Linux and Mac requires handling edge cases

    • Tools output might be different

    • System specific commands are not the same (i.e opening a URL in
    Browser)

    • Paths, Path list separators, Line endings, etc.. are different based on
    the OS

    View Slide

  81. example: Path differences
    $ echo $GOPATH
    /Users/fatih/go:/Users/fatih/Code/do
    > echo %GOPATH%
    C:\go;C:\Code\do\go
    On Linux or macOS
    On Windows

    View Slide

  82. Split a list of paths?
    " from vim-go/autoload/go/path.vim
    " Get a list of current GOPATH's
    let go_paths = split($GOPATH, ":")

    View Slide

  83. Split a list of paths?
    " from vim-go/autoload/go/path.vim
    " Get a list of current GOPATH's
    let go_paths = split($GOPATH, ":")
    ['/Users/fatih/go', '/Users/fatih/Code/do']
    On Linux or macOS
    ['C', '\go;C', '\Code\do\go']
    On Windows


    View Slide

  84. Split a list of paths (fix)
    " from vim-go/autoload/go/path.vim
    " Get a list of current GOPATH's
    let go_paths = split($GOPATH, ":")
    let go_paths = split($GOPATH, go#util#PathListSep())

    View Slide

  85. Split a list of paths (fix)
    " from vim-go/autoload/go/path.vim
    " Get a list of current GOPATH's
    let go_paths = split($GOPATH, ":")
    let go_paths = split($GOPATH, go#util#PathListSep())
    ['/Users/fatih/go', '/Users/fatih/Code/do']
    On Linux or macOS
    ['C:\go', 'C:\Code\do\go']
    On Windows


    View Slide

  86. View Slide

  87. View Slide

  88. More competition

    View Slide

  89. Competition
    • Go constantly improves

    • New features and fundamental changes (i.e: default GOPATH, vendor/
    folder)

    • Editors: vscode-go (Microsoft), gogland (JetBrains)

    • User base increases, it's getting harder and harder to maintain and
    improve it

    View Slide

  90. New logo!

    View Slide

  91. vim-go tutorial

    View Slide

  92. View Slide

  93. 2017

    View Slide

  94. Features & improvements

    View Slide

  95. Features & improvements
    • New Commands — such as :GoModifyTags, :GoFillStruct, :GoKeyify,
    etc...

    • Folding support (syntax based)

    • Travis CI test integration

    • Removed Wiki and Readme.md examples. Moved all to docs/vim-go.txt
    • Over 800 Issues were closed!

    View Slide

  96. View Slide

  97. Travis (CI) integration

    View Slide

  98. $ cat .travis.yml
    language: go
    env:
    global:
    - DEPS=$HOME/deps
    - PATH=$DEPS/bin:$PATH
    - PATCH="v8.0.0134"
    install: |
    git clone --branch $PATCH --depth 1 https://github.com/vim/vim
    cd vim
    ./configure --prefix=$DEPS --with-features=huge --disable-gui
    make
    make install
    cd -
    script: ./scripts/test.sh
    01
    02
    03
    04
    05
    06
    07
    08
    09
    10
    11
    12
    13
    14
    15
    16
    17

    View Slide

  99. $ cat scripts/test.sh
    ...
    for test_file in ../autoload/go/*_test.vim
    do
    vim -u NONE -S runtest.vim $test_file
    done
    if [ -f "test.log" ]; then
    cat test.log
    fi
    # if Failed exists, test failed
    if [ -f "FAILED" ]; then
    echo 2>&1 "FAIL"
    exit 1
    fi
    echo 2>&1 "PASS"
    ...
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38

    View Slide

  100. Only _test.vim files are selected

    View Slide

  101. In runtest.vim
    1.Looking for functions that start with Test_

    2.Call the test

    3.If v:errors is not empty, populate error log file (test.log)

    View Slide

  102. $ cat autoload/go/fmt_test.vim
    func Test_run_fmt()
    let actual_file = tempname()
    call writefile(readfile("test-fixtures/fmt/hello.go"), actual_file)
    let expected = join(readfile("test-fixtures/fmt/hello_golden.go"), "\n")
    " run our code
    call go#fmt#run("gofmt", actual_file, "test-fixtures/fmt/hello.go")
    " this should now contain the formatted code
    let actual = join(readfile(actual_file), "\n")
    call assert_equal(expected, actual)
    endfunc
    01
    02
    03
    04
    05
    06
    07
    08
    09
    10
    11
    12
    13
    14

    View Slide

  103. View Slide

  104. Integrating vimhelplint

    View Slide

  105. Integrating vimhelplint (cont.)
    • Runs on every pull request

    View Slide

  106. Google Open Source Award

    View Slide

  107. View Slide

  108. Most used editor for Go

    View Slide

  109. #1 preferred editor by Go developers

    View Slide

  110. Collaborators

    View Slide

  111. Two new core contributors

    View Slide

  112. Stats

    View Slide

  113. Github Stars ⭐
    2014 2015 2016 2017 now
    0
    1400
    3800
    5820
    7403

    View Slide

  114. Github stats (end of 2017 ...)
    $ Contributors 207
    Forks 726
    Commits 1889
    Issues Closed 1464
    ✅ Pull Requests Closed 637

    View Slide

  115. 2018 – ...

    View Slide

  116. Future
    • Debugging
    • Most requested feature.

    • Problem? No debugging API in Vim.

    • Works with server running in background, need async API

    • @mattn started working on it

    View Slide

  117. Future 2
    • Vim 8.0 Terminal integration

    • Better :GoRun integration (reading from stdin)

    • Showing stdout/stderr in :GoTest

    • More vim specific features

    • Additional text objects, we have currently function

    • AST based syntax highlighting

    • Autocomplete still sucks, not sure how to fix it

    View Slide

  118. Verdict

    View Slide

  119. The bad
    • No sync between README.MD and docs/vim-go.txt

    • Changelog was in Github Releases, moved to Changelog.md

    • Vim 7.4, Vim 8.0 and NeoVim 0.2 compatibility is hard

    • For advanced features, better UI is needed for better UX

    View Slide

  120. The good
    • Go made it easy. Tooling that integrates very well

    • Constant development and release

    • Listened to users

    • UX and UI is important

    View Slide


  121. View Slide

  122. Thanks!
    Fatih Arslan
    @fatih
    @fatih
    [email protected]

    View Slide