The Past and Future of Vim-go

How did vim-go started and evolved throughout the years

Transcript

1. The Past and Future
   of Vim-go
   Fatih Arslan
   Sr. Software Engineer @DigitalOcean
   

   View Slide

2. Me
   • Sr. Software Engineer @DigitalOcean
   • Creator of vim-go
   • Go contributor, author of many popular Go
   packages (i.e: color, structs, etc..)
   • Tool maker (i.e: gomodifytags, motion, etc...)
   • Coffee and bag geek
   

   View Slide

3. Go
   Programming
   Language
   

   View Slide

4. Go programming language
   • Started in 2007 with Rob Pike, Ken Thompson and Robert Griesemer
   • Open Source in 10 November, 2009
   
   • Go 1.0 released in 28 March, 2012
   
   • Simple, reliable and efficient
   

   View Slide

5. It started 8 years ago (2009)
   

   View Slide

6. View Slide

7. One year later (2010)
   

   View Slide

8. First commands (2011)
   :Import strings
   :ImportAs . strings
   :Drop strings
   

   View Slide

9. gofmt support was added (2011)
   

   View Slide

10. autoload/go folder (2011)
    

    View Slide

11. :make support (2013)
    

    View Slide

12. misc/vim removed from Go (2014)
    

    View Slide

13. Last state of misc/vim
    • Some commands to manipulate the source
    
    • :Fmt, :Import, etc...
    
    • Syntax, compiler and indent support
    
    • Go file detection
    
    • Proper readme
    (before it was removed)
    

    View Slide

14. vim-go
    Go development
    plugin for vim
    

    View Slide

15. github.com/fatih/vim-go
    

    View Slide

16. How and why was
    vim-go created?
    

    View Slide

17. Too many (small)
    plugins
    

    View Slide

18. Too many plugins
    • https://github.com/Blackrush/vim-gocode
    
    • https://github.com/fsouza/go.vim
    
    • https://github.com/dgryski/vim-godef
    
    • Go's misc/vim
    
    • etc...
    

    View Slide

19. No clear roadmap
    

    View Slide

20. misc/vim was
    missing features
    

    View Slide

21. Flaws
    • No coherent feature set, every plugin had its own commands, etc..
    
    • No docs/ folder
    
    • No code/build/test commands (only :make).
    
    • Binary paths are hard coded
    
    • No coherent UI across all plugin commands, some look under the cursor some do
    not
    
    • No autocompletion
    
    • etc...
    

    View Slide

22. A plan to fix it!
    

    View Slide

23. A plan to fix it
    1. Start using misc/vim
    
    2. Integrate all plugins
    
    3. Fix flaws & add missing features
    
    4. Announce it
    

    View Slide

24. View Slide

25. View Slide

26. First vim-go commit
    

    View Slide

27. Naming is hard!
    

    View Slide

28. Naming is hard
    • First, it was called golang.vim
    
    • Then renamed to go.vim (Because golang was wrong)
    
    • Finally changed to vim-go as I didn't like the .vim
    extension
    

    View Slide

29. Lack of documentation
    

    View Slide

30. Lack of documentation
    • Added docs/vim-go.txt file
    
    • Improved installations steps to include pathogen, vim-plug, etc...
    
    • Added current existing features
    
    • Added information about commands and settings
    

    View Slide

31. Writing help files is not fun
    

    View Slide

32. View Slide

33. Usability fixes
    

    View Slide

34. Usability fixes
    • One command to download and install tools :GoInstallBinaries
    
    • Coherent command interface (i.e: :GoFoo without argument and with
    argument)
    
    • Improved commands by using (for example :GoBuild! to avoid
    jumping to first error)
    
    • Added :Go prefix to all commands, i.e. :Import > :GoImport
    
    • Added go_ prefix to all global variables, i.e. g:bin_path > g:go_bin_path
    

    View Slide

35. View Slide

36. Under the hood
    improvements
    

    View Slide

37. Under the hood improvements
    • Prevent :make to produce binary
    
    • Fixed not losing history on format (gofmt)
    
    • Still problematic
    
    • Added mappings for provide custom mappings
    
    • Open cwindow after error or warning
    

    View Slide

38. Features & improvements
    

    View Slide

39. View Slide

40. New features: tooling
    • :GoRun, :GoBuild, :GoTest- run, build or test your file
    
    • :GoLint, :GoVet, :GoErrcheck - linters and checkers
    
    • :GoCoverage - show code coverage (initial plugin by Yukinari Toyota)
    • :GoPlay - share your buffer to play.golang.org
    

    View Slide

41. New features: tooling (cont)
    • :GoDoc - show package documentation
    
    • :GoInfo - show identifier information under the cursor
    
    • :GoDef - jump to a symbol/declaration (inital plugin by Damian Gryski)
    
    • :GoRename - refactor identifiers
    
    • ...
    

    View Slide

42. New features: misc
    • Ultisnippet/Neosnippet integration
    • Run :GoFmt automatically on save
    • Windows OS support
    • Started to work on static analysis (via Oracle)
    • tagbar integration
    • ...
    

    View Slide

43. GoCoverage
    demo
    

    View Slide

44. View Slide

45. First release v1.0
    

    View Slide

46. After 1347 commits
    

    View Slide

47. Versioning changed
    

    View Slide

48. Versioniong
    • We started with v1.0
    
    • Was increasing on patch levels:
    
    • v1.0.1, v1.0.2, v1.0.2
    
    • This was wrong, switched to minor levels:
    
    • v1.1, v1.2, v1.3, etc..
    
    • Allowed us to release patch releases if needed (i.e v1.3.1)
    

    View Slide

49. Added Kindle e-book
    donation option
    

    View Slide

50. Kindle e-book donation (2014)
    

    View Slide

51. Kindle e-book donation
    • People started asking how to donate
    
    • No money was needed, but I was reading a lot
    
    • Added public Kindle wish list
    • Received over 40 books
    

    View Slide

52. License added
    

    View Slide

53. License added
    • Distros started packaging vim-go
    
    • No license meant no clear guidance
    • Added BSD 3-Clause license
    
    • Same as Go's own license
    
    • It's a very permissive license, but the name "vim-go" can't be used for
    advertisement or promotions.
    

    View Slide

54. New
    features & improvements
    

    View Slide

55. Improvements & features
    • Oracle support enabled (static analysis)
    
    • Show function callees, callers
    
    • Show all references of an identifier
    
    • etc...
    
    • Improved syntax highlighting even more (build constraints, all operators)
    
    • Automatic GOPATH detection
    
    • gb, Godeps support
    

    View Slide

56. New Commands
    • :GoMetalinter (combines :GoLint, :GoErrcheck, :GoVet)
    
    • :GoTestCompile, :GoTestFunc
    
    • Makes testing workflow more efficient
    
    • :GoPath
    
    • Show or change GOPATH
    

    View Slide

57. View Slide

58. NeoVim
    

    View Slide

59. NeoVim
    • Everything Async
    
    • :GoBuild, :GoTest, etc..
    
    • Statusline integration for commands (feedback loop)
    
    • Terminal support for :GoRun and :GoTest
    

    View Slide

60. View Slide

61. Donation changed to Patreon
    

    View Slide

62. View Slide

63. Changed donation to Patreon
    • I had now so many Kindle books, I couldn't finish them
    
    • Started 1.5 years ago around April 2016
    • $5, $10, $25 and $100 tiers
    
    • Most used is $5 tier
    

    View Slide

64. Donations per month
    

    View Slide

65. Why donation
    • Open source is free to use, but doesn't mean it has free value
    • We spend our own time, a value that we can't buy
    
    • Donation helps us to buy value in other areas
    

    View Slide

66. Vim 8.0
    

    View Slide

67. Vim 8.0
    • Everything Async for Vim!
    
    • :GoBuild, :GoTest, etc..
    
    • Better Statusline integration
    • A lot of Vim bugs in the beginning, wasn't quite ready
    
    • JSON encode/decode was useful for Go tooling integration
    

    View Slide

68. View Slide

69. has_job()
    

    View Slide

70. View Slide

71. Using tools
    written in Go
    

    View Slide

72. From Vim script to Go
    • Logic is replaced by a binary, written in Go
    
    • More and more features are reimplemented again
    
    • Vim calls it via system() or job_start()
    
    • Tools return JSON or Vim script output (but not all of them)
    

    View Slide

73. View Slide

74. The power of AST
    • Go has an excellent parser & scanner packages
    • Allows us to modify Go source code
    
    • Parser is very fast. It's not noticeable at all.
    
    • Direct access to AST (Abstract Syntax Tree)
    

    View Slide

75. AST based feature
    

    View Slide

76. package main
    type Server struct {
    Name string
    Port int
    EnableLogs bool
    BaseDomain string
    Credentials struct {
    Username string
    Password string
    }
    }
    {
    "start": 3,
    "end": 12,
    "lines": [
    "type Server struct {",
    " Name string `xml:\"name\"`",
    " Port int `xml:\"port\"`",
    " EnableLogs bool `xml:\"enable_logs\"`",
    " BaseDomain string `xml:\"base_domain\"`",
    " Credentials struct {",
    " Username string `xml:\"username\"`",
    " Password string `xml:\"password\"`",
    " } `xml:\"credentials\"`",
    "}"
    ]
    }
    Input
    JSON Output
    $ gomodifytags
    
    -file example.go
    
    -struct Server
    
    -add-tags json
    
    -format json
    

    View Slide

77. View Slide

78. Demo time!
    (selecting functions)
    

    View Slide

79. Advantages of AST based text objects
    • Anonymous functions are supported
    
    • One liner functions can be selected easier
    
    • Cursor position can be anywhere as long as it makes sense
    
    • Comments are treated as a part of the function declaration
    

    View Slide

80. OS differences
    • Supporting Windows, Linux and Mac requires handling edge cases
    
    • Tools output might be different
    
    • System specific commands are not the same (i.e opening a URL in
    Browser)
    
    • Paths, Path list separators, Line endings, etc.. are different based on
    the OS
    

    View Slide

81. example: Path differences
    $ echo $GOPATH
    /Users/fatih/go:/Users/fatih/Code/do
    > echo %GOPATH%
    C:\go;C:\Code\do\go
    On Linux or macOS
    On Windows
    

    View Slide

82. Split a list of paths?
    " from vim-go/autoload/go/path.vim
    " Get a list of current GOPATH's
    let go_paths = split($GOPATH, ":")
    

    View Slide

83. Split a list of paths?
    " from vim-go/autoload/go/path.vim
    " Get a list of current GOPATH's
    let go_paths = split($GOPATH, ":")
    ['/Users/fatih/go', '/Users/fatih/Code/do']
    On Linux or macOS
    ['C', '\go;C', '\Code\do\go']
    On Windows
    
    
    

    View Slide

84. Split a list of paths (fix)
    " from vim-go/autoload/go/path.vim
    " Get a list of current GOPATH's
    let go_paths = split($GOPATH, ":")
    let go_paths = split($GOPATH, go#util#PathListSep())
    

    View Slide

85. Split a list of paths (fix)
    " from vim-go/autoload/go/path.vim
    " Get a list of current GOPATH's
    let go_paths = split($GOPATH, ":")
    let go_paths = split($GOPATH, go#util#PathListSep())
    ['/Users/fatih/go', '/Users/fatih/Code/do']
    On Linux or macOS
    ['C:\go', 'C:\Code\do\go']
    On Windows
    
    
    

    View Slide

86. View Slide

87. View Slide

88. More competition
    

    View Slide

89. Competition
    • Go constantly improves
    
    • New features and fundamental changes (i.e: default GOPATH, vendor/
    folder)
    
    • Editors: vscode-go (Microsoft), gogland (JetBrains)
    
    • User base increases, it's getting harder and harder to maintain and
    improve it
    

    View Slide

90. New logo!
    

    View Slide

91. vim-go tutorial
    

    View Slide

92. View Slide

93. View Slide

94. Features & improvements
    

    View Slide

95. Features & improvements
    • New Commands — such as :GoModifyTags, :GoFillStruct, :GoKeyify,
    etc...
    
    • Folding support (syntax based)
    
    • Travis CI test integration
    
    • Removed Wiki and Readme.md examples. Moved all to docs/vim-go.txt
    • Over 800 Issues were closed!
    

    View Slide

96. View Slide

97. Travis (CI) integration
    

    View Slide

98. $ cat .travis.yml
    language: go
    env:
    global:
    - DEPS=$HOME/deps
    - PATH=$DEPS/bin:$PATH
    - PATCH="v8.0.0134"
    install: |
    git clone --branch $PATCH --depth 1 https://github.com/vim/vim
    cd vim
    ./configure --prefix=$DEPS --with-features=huge --disable-gui
    make
    make install
    cd -
    script: ./scripts/test.sh
    01
    02
    03
    04
    05
    06
    07
    08
    09
    10
    11
    12
    13
    14
    15
    16
    17
    

    View Slide

99. $ cat scripts/test.sh
    ...
    for test_file in ../autoload/go/*_test.vim
    do
    vim -u NONE -S runtest.vim $test_file
    done
    if [ -f "test.log" ]; then
    cat test.log
    fi
    # if Failed exists, test failed
    if [ -f "FAILED" ]; then
    echo 2>&1 "FAIL"
    exit 1
    fi
    echo 2>&1 "PASS"
    ...
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    

    View Slide

100. Only _test.vim files are selected
     

     View Slide

101. In runtest.vim
     1.Looking for functions that start with Test_
     
     2.Call the test
     
     3.If v:errors is not empty, populate error log file (test.log)
     

     View Slide

102. $ cat autoload/go/fmt_test.vim
     func Test_run_fmt()
     let actual_file = tempname()
     call writefile(readfile("test-fixtures/fmt/hello.go"), actual_file)
     let expected = join(readfile("test-fixtures/fmt/hello_golden.go"), "\n")
     " run our code
     call go#fmt#run("gofmt", actual_file, "test-fixtures/fmt/hello.go")
     " this should now contain the formatted code
     let actual = join(readfile(actual_file), "\n")
     call assert_equal(expected, actual)
     endfunc
     01
     02
     03
     04
     05
     06
     07
     08
     09
     10
     11
     12
     13
     14
     

     View Slide

103. View Slide

104. Integrating vimhelplint
     

     View Slide

105. Integrating vimhelplint (cont.)
     • Runs on every pull request
     

     View Slide

106. Google Open Source Award
     

     View Slide

107. View Slide

108. Most used editor for Go
     

     View Slide

109. #1 preferred editor by Go developers
     

     View Slide

110. Collaborators
     

     View Slide

111. Two new core contributors
     

     View Slide

112. Stats
     

     View Slide

113. Github Stars ⭐
     2014 2015 2016 2017 now
     0
     1400
     3800
     5820
     7403
     

     View Slide

114. Github stats (end of 2017 ...)
     $ Contributors 207
     Forks 726
     Commits 1889
     Issues Closed 1464
     ✅ Pull Requests Closed 637
     

     View Slide

115. 2018 – ...
     

     View Slide

116. Future
     • Debugging
     • Most requested feature.
     
     • Problem? No debugging API in Vim.
     
     • Works with server running in background, need async API
     
     • @mattn started working on it
     

     View Slide

117. Future 2
     • Vim 8.0 Terminal integration
     
     • Better :GoRun integration (reading from stdin)
     
     • Showing stdout/stderr in :GoTest
     
     • More vim specific features
     
     • Additional text objects, we have currently function
     
     • AST based syntax highlighting
     
     • Autocomplete still sucks, not sure how to fix it
     

     View Slide

118. Verdict
     

     View Slide

119. The bad
     • No sync between README.MD and docs/vim-go.txt
     
     • Changelog was in Github Releases, moved to Changelog.md
     
     • Vim 7.4, Vim 8.0 and NeoVim 0.2 compatibility is hard
     
     • For advanced features, better UI is needed for better UX
     

     View Slide

120. The good
     • Go made it easy. Tooling that integrates very well
     
     • Constant development and release
     
     • Listened to users
     
     • UX and UI is important
     

     View Slide

121. ❤
     

     View Slide

122. Thanks!
     Fatih Arslan
     @fatih
     @fatih
     fatih@digitalocean.com
     

     View Slide