The Past and Future of Vim-go

by Fatih Arslan

Slide 1

Slide 1 text

The Past and Future of Vim-go Fatih Arslan Sr. Software Engineer @DigitalOcean

Slide 2

Slide 2 text

Me • Sr. Software Engineer @DigitalOcean • Creator of vim-go • Go contributor, author of many popular Go packages (i.e: color, structs, etc..) • Tool maker (i.e: gomodifytags, motion, etc...) • Coﬀee and bag geek

Slide 3

Slide 3 text

Go Programming Language

Slide 4

Slide 4 text

Go programming language • Started in 2007 with Rob Pike, Ken Thompson and Robert Griesemer • Open Source in 10 November, 2009 • Go 1.0 released in 28 March, 2012 • Simple, reliable and eﬃcient

Slide 5

Slide 5 text

It started 8 years ago (2009)

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

One year later (2010)

Slide 8

Slide 8 text

First commands (2011) :Import strings :ImportAs . strings :Drop strings

Slide 9

Slide 9 text

gofmt support was added (2011)

Slide 10

Slide 10 text

autoload/go folder (2011)

Slide 11

Slide 11 text

:make support (2013)

Slide 12

Slide 12 text

misc/vim removed from Go (2014)

Slide 13

Slide 13 text

Last state of misc/vim • Some commands to manipulate the source • :Fmt, :Import, etc... • Syntax, compiler and indent support • Go ﬁle detection • Proper readme (before it was removed)

Slide 14

Slide 14 text

vim-go Go development plugin for vim

Slide 15

Slide 15 text

github.com/fatih/vim-go

Slide 16

Slide 16 text

How and why was vim-go created?

Slide 17

Slide 17 text

Too many (small) plugins

Slide 18

Slide 18 text

Too many plugins • https://github.com/Blackrush/vim-gocode • https://github.com/fsouza/go.vim • https://github.com/dgryski/vim-godef • Go's misc/vim • etc...

Slide 19

Slide 19 text

No clear roadmap

Slide 20

Slide 20 text

misc/vim was missing features

Slide 21

Slide 21 text

Flaws • No coherent feature set, every plugin had its own commands, etc.. • No docs/ folder • No code/build/test commands (only :make). • Binary paths are hard coded • No coherent UI across all plugin commands, some look under the cursor some do not • No autocompletion • etc...

Slide 22

Slide 22 text

A plan to ﬁx it!

Slide 23

Slide 23 text

A plan to ﬁx it 1. Start using misc/vim 2. Integrate all plugins 3. Fix ﬂaws & add missing features 4. Announce it

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

2014

Slide 26

Slide 26 text

First vim-go commit

Slide 27

Slide 27 text

Naming is hard!

Slide 28

Slide 28 text

Naming is hard • First, it was called golang.vim • Then renamed to go.vim (Because golang was wrong) • Finally changed to vim-go as I didn't like the .vim extension

Slide 29

Slide 29 text

Lack of documentation

Slide 30

Slide 30 text

Lack of documentation • Added docs/vim-go.txt ﬁle • Improved installations steps to include pathogen, vim-plug, etc... • Added current existing features • Added information about commands and settings

Slide 31

Slide 31 text

Writing help ﬁles is not fun

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

Usability ﬁxes

Slide 34

Slide 34 text

Usability fixes • One command to download and install tools :GoInstallBinaries • Coherent command interface (i.e: :GoFoo without argument and with argument) • Improved commands by using (for example :GoBuild! to avoid jumping to first error) • Added :Go prefix to all commands, i.e. :Import > :GoImport • Added go_ prefix to all global variables, i.e. g:bin_path > g:go_bin_path

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

Under the hood improvements

Slide 37

Slide 37 text

Under the hood improvements • Prevent :make to produce binary • Fixed not losing history on format (gofmt) • Still problematic • Added mappings for provide custom mappings • Open cwindow after error or warning

Slide 38

Slide 38 text

Features & improvements

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

New features: tooling • :GoRun, :GoBuild, :GoTest- run, build or test your ﬁle • :GoLint, :GoVet, :GoErrcheck - linters and checkers • :GoCoverage - show code coverage (initial plugin by Yukinari Toyota) • :GoPlay - share your buﬀer to play.golang.org

Slide 41

Slide 41 text

New features: tooling (cont) • :GoDoc - show package documentation • :GoInfo - show identiﬁer information under the cursor • :GoDef - jump to a symbol/declaration (inital plugin by Damian Gryski) • :GoRename - refactor identiﬁers • ...

Slide 42

Slide 42 text

New features: misc • Ultisnippet/Neosnippet integration • Run :GoFmt automatically on save • Windows OS support • Started to work on static analysis (via Oracle) • tagbar integration • ...

Slide 43

Slide 43 text

GoCoverage demo

Slide 44

Slide 44 text

2015

Slide 45

Slide 45 text

First release v1.0

Slide 46

Slide 46 text

After 1347 commits

Slide 47

Slide 47 text

Versioning changed

Slide 48

Slide 48 text

Versioniong • We started with v1.0 • Was increasing on patch levels: • v1.0.1, v1.0.2, v1.0.2 • This was wrong, switched to minor levels: • v1.1, v1.2, v1.3, etc.. • Allowed us to release patch releases if needed (i.e v1.3.1)

Slide 49

Slide 49 text

Added Kindle e-book donation option

Slide 50

Slide 50 text

Kindle e-book donation (2014)

Slide 51

Slide 51 text

Kindle e-book donation • People started asking how to donate • No money was needed, but I was reading a lot • Added public Kindle wish list • Received over 40 books

Slide 52

Slide 52 text

License added

Slide 53

Slide 53 text

License added • Distros started packaging vim-go • No license meant no clear guidance • Added BSD 3-Clause license • Same as Go's own license • It's a very permissive license, but the name "vim-go" can't be used for advertisement or promotions.

Slide 54

Slide 54 text

New features & improvements

Slide 55

Slide 55 text

Improvements & features • Oracle support enabled (static analysis) • Show function callees, callers • Show all references of an identiﬁer • etc... • Improved syntax highlighting even more (build constraints, all operators) • Automatic GOPATH detection • gb, Godeps support

Slide 56

Slide 56 text

New Commands • :GoMetalinter (combines :GoLint, :GoErrcheck, :GoVet) • :GoTestCompile, :GoTestFunc • Makes testing workﬂow more eﬃcient • :GoPath • Show or change GOPATH

Slide 57

Slide 57 text

2016

Slide 58

Slide 58 text

NeoVim

Slide 59

Slide 59 text

NeoVim • Everything Async • :GoBuild, :GoTest, etc.. • Statusline integration for commands (feedback loop) • Terminal support for :GoRun and :GoTest

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

Donation changed to Patreon

Slide 62

Slide 62 text

No content

Slide 63

Slide 63 text

Changed donation to Patreon • I had now so many Kindle books, I couldn't ﬁnish them • Started 1.5 years ago around April 2016 • $5, $10, $25 and $100 tiers • Most used is $5 tier

Slide 64

Slide 64 text

Donations per month

Slide 65

Slide 65 text

Why donation • Open source is free to use, but doesn't mean it has free value • We spend our own time, a value that we can't buy • Donation helps us to buy value in other areas

Slide 66

Slide 66 text

Vim 8.0

Slide 67

Slide 67 text

Vim 8.0 • Everything Async for Vim! • :GoBuild, :GoTest, etc.. • Better Statusline integration • A lot of Vim bugs in the beginning, wasn't quite ready • JSON encode/decode was useful for Go tooling integration

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

has_job()

Slide 70

Slide 70 text

No content

Slide 71

Slide 71 text

Using tools written in Go

Slide 72

Slide 72 text

From Vim script to Go • Logic is replaced by a binary, written in Go • More and more features are reimplemented again • Vim calls it via system() or job_start() • Tools return JSON or Vim script output (but not all of them)

Slide 73

Slide 73 text

No content

Slide 74

Slide 74 text

The power of AST • Go has an excellent parser & scanner packages • Allows us to modify Go source code • Parser is very fast. It's not noticeable at all. • Direct access to AST (Abstract Syntax Tree)

Slide 75

Slide 75 text

AST based feature

Slide 76

Slide 76 text

package main type Server struct { Name string Port int EnableLogs bool BaseDomain string Credentials struct { Username string Password string } } { "start": 3, "end": 12, "lines": [ "type Server struct {", " Name string `xml:\"name\"`", " Port int `xml:\"port\"`", " EnableLogs bool `xml:\"enable_logs\"`", " BaseDomain string `xml:\"base_domain\"`", " Credentials struct {", " Username string `xml:\"username\"`", " Password string `xml:\"password\"`", " } `xml:\"credentials\"`", "}" ] } Input JSON Output $ gomodifytags -ﬁle example.go -struct Server -add-tags json -format json

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

Demo time! (selecting functions)

Slide 79

Slide 79 text

Advantages of AST based text objects • Anonymous functions are supported • One liner functions can be selected easier • Cursor position can be anywhere as long as it makes sense • Comments are treated as a part of the function declaration

Slide 80

Slide 80 text

OS differences • Supporting Windows, Linux and Mac requires handling edge cases • Tools output might be different • System specific commands are not the same (i.e opening a URL in Browser) • Paths, Path list separators, Line endings, etc.. are different based on the OS

Slide 81

Slide 81 text

example: Path differences $ echo $GOPATH /Users/fatih/go:/Users/fatih/Code/do > echo %GOPATH% C:\go;C:\Code\do\go On Linux or macOS On Windows

Slide 82

Slide 82 text

Split a list of paths? " from vim-go/autoload/go/path.vim " Get a list of current GOPATH's let go_paths = split($GOPATH, ":")

Slide 83

Slide 83 text

Split a list of paths? " from vim-go/autoload/go/path.vim " Get a list of current GOPATH's let go_paths = split($GOPATH, ":") ['/Users/fatih/go', '/Users/fatih/Code/do'] On Linux or macOS ['C', '\go;C', '\Code\do\go'] On Windows

Slide 84

Slide 84 text

Split a list of paths (ﬁx) " from vim-go/autoload/go/path.vim " Get a list of current GOPATH's let go_paths = split($GOPATH, ":") let go_paths = split($GOPATH, go#util#PathListSep())

Slide 85

Slide 85 text

Split a list of paths (ﬁx) " from vim-go/autoload/go/path.vim " Get a list of current GOPATH's let go_paths = split($GOPATH, ":") let go_paths = split($GOPATH, go#util#PathListSep()) ['/Users/fatih/go', '/Users/fatih/Code/do'] On Linux or macOS ['C:\go', 'C:\Code\do\go'] On Windows

Slide 86

Slide 86 text

No content

Slide 87

Slide 87 text

No content

Slide 88

Slide 88 text

More competition

Slide 89

Slide 89 text

Competition • Go constantly improves • New features and fundamental changes (i.e: default GOPATH, vendor/ folder) • Editors: vscode-go (Microsoft), gogland (JetBrains) • User base increases, it's getting harder and harder to maintain and improve it

Slide 90

Slide 90 text

New logo!

Slide 91

Slide 91 text

vim-go tutorial

Slide 92

Slide 92 text

No content

Slide 93

Slide 93 text

2017

Slide 94

Slide 94 text

Features & improvements

Slide 95

Slide 95 text

Features & improvements • New Commands — such as :GoModifyTags, :GoFillStruct, :GoKeyify, etc... • Folding support (syntax based) • Travis CI test integration • Removed Wiki and Readme.md examples. Moved all to docs/vim-go.txt • Over 800 Issues were closed!

Slide 96

Slide 96 text

No content

Slide 97

Slide 97 text

Travis (CI) integration

Slide 98

Slide 98 text

$ cat .travis.yml language: go env: global: - DEPS=$HOME/deps - PATH=$DEPS/bin:$PATH - PATCH="v8.0.0134" install: | git clone --branch $PATCH --depth 1 https://github.com/vim/vim cd vim ./configure --prefix=$DEPS --with-features=huge --disable-gui make make install cd - script: ./scripts/test.sh 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17

Slide 99

Slide 99 text

$ cat scripts/test.sh ... for test_file in ../autoload/go/*_test.vim do vim -u NONE -S runtest.vim $test_file done if [ -f "test.log" ]; then cat test.log fi # if Failed exists, test failed if [ -f "FAILED" ]; then echo 2>&1 "FAIL" exit 1 fi echo 2>&1 "PASS" ... 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

Slide 100

Slide 100 text

Only _test.vim ﬁles are selected

Slide 101

Slide 101 text

In runtest.vim 1.Looking for functions that start with Test_ 2.Call the test 3.If v:errors is not empty, populate error log ﬁle (test.log)

Slide 102

Slide 102 text

$ cat autoload/go/fmt_test.vim func Test_run_fmt() let actual_file = tempname() call writefile(readfile("test-fixtures/fmt/hello.go"), actual_file) let expected = join(readfile("test-fixtures/fmt/hello_golden.go"), "\n") " run our code call go#fmt#run("gofmt", actual_file, "test-fixtures/fmt/hello.go") " this should now contain the formatted code let actual = join(readfile(actual_file), "\n") call assert_equal(expected, actual) endfunc 01 02 03 04 05 06 07 08 09 10 11 12 13 14

Slide 103

Slide 103 text

No content

Slide 104

Slide 104 text

Integrating vimhelplint

Slide 105

Slide 105 text

Integrating vimhelplint (cont.) • Runs on every pull request

Slide 106

Slide 106 text

Google Open Source Award

Slide 107

Slide 107 text

No content

Slide 108

Slide 108 text

Most used editor for Go

Slide 109

Slide 109 text

#1 preferred editor by Go developers

Slide 110

Slide 110 text

Collaborators

Slide 111

Slide 111 text

Two new core contributors

Slide 112

Slide 112 text

Stats

Slide 113

Slide 113 text

Github Stars ⭐ 2014 2015 2016 2017 now 0 1400 3800 5820 7403

Slide 114

Slide 114 text

Github stats (end of 2017 ...) $ Contributors 207 Forks 726 Commits 1889 Issues Closed 1464 ✅ Pull Requests Closed 637

Slide 115

Slide 115 text

2018 – ...

Slide 116

Slide 116 text

Future • Debugging • Most requested feature. • Problem? No debugging API in Vim. • Works with server running in background, need async API • @mattn started working on it

Slide 117

Slide 117 text

Future 2 • Vim 8.0 Terminal integration • Better :GoRun integration (reading from stdin) • Showing stdout/stderr in :GoTest • More vim speciﬁc features • Additional text objects, we have currently function • AST based syntax highlighting • Autocomplete still sucks, not sure how to ﬁx it

Slide 118

Slide 118 text

Verdict

Slide 119

Slide 119 text

The bad • No sync between README.MD and docs/vim-go.txt • Changelog was in Github Releases, moved to Changelog.md • Vim 7.4, Vim 8.0 and NeoVim 0.2 compatibility is hard • For advanced features, better UI is needed for better UX

Slide 120

Slide 120 text

The good • Go made it easy. Tooling that integrates very well • Constant development and release • Listened to users • UX and UI is important