Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Writing a modern CLI tool

Writing a modern CLI tool

Talk I gave at work about my experience writing a modern CLI tool and why I have chosen Go for writing it.

Henrique Vicente

September 02, 2016
Tweet

More Decks by Henrique Vicente

Other Decks in Programming

Transcript

  1. Date
    Writing a modern CLI tool.
    Henrique Vicente
    27th July, 2016

    View full-size slide

  2. Originally I/O happened via a physically
    connected system console. Then…

    View full-size slide

  3. From Time-sharing to
    today’s iTerm
    ✤ Type on the terminal


    cat ~/Downloads/behind-the-firewall.gif | imgcat

    View full-size slide

  4. IBM XT
    ✤ First computer coming with standard hard-
    drive

    View full-size slide

  5. Classic hypothetical program
    1. Takes an input
    2. Does some magic
    3. Writes to output

    View full-size slide

  6. POSIX real world program
    ✤ Standard in / out streams (stdin, stdout, stderr)
    ✤ Environment variables
    ✤ Working directory
    ✤ Arguments
    ✤ Exit code
    ✤ parent process, process id, process group,

    View full-size slide

  7. What you want
    ✤ Static binaries
    ✤ Easy and secure distribution channel
    ✤ Easy to program
    ✤ Garbage collection
    ✤ High-level programming language
    ✤ Modern, clean, safe (type safety)
    ✤ Avoid the hype (unless if its arguments win)
    ✤ Lightweight startup for short-lived processes
    ✤ Great performance for long-lived daemons
    ✤ Reliable tests
    ✤ Culture
    ✤ No unbundled or dynamically linked
    dependencies
    ✤ No interpretation or virtual machines
    ✤ Trustworthy dependency managing

    View full-size slide

  8. stdout and stderr unexpected merging issues

    View full-size slide

  9. stdout and stderr unexpected merging issues
    ✤ ordering sort (e.g., verifying what is printed on tests)
    ✤ "updating" output (i.e., progress bar) modifying what you get

    View full-size slide

  10. What about the children?

    View full-size slide

  11. Child processes
    ✤ Zombies, orphans…
    ✤ Quite hard to work with

    reapers, actually.

    View full-size slide

  12. From “gh” to “we”

    View full-size slide

  13. Maintenance hell
    ✤ No tests
    ✤ Node.js 0.8 was still borning, API changes
    ✤ dumb npm packages… left-pad ¯\_(ツ)_/¯
    ✤ dependency hell
    ✤ callback hell: no Promise()
    ✤ At certain point [I thought?] people hated me

    View full-size slide

  14. Choosing a language for WeDeploy’s CLI

    View full-size slide

  15. Build one pet project before committing to an
    unfamiliar language or ecosystem
    ✤ I wrote picel: is a light-weight, blazing fast* REST-ful micro service for
    image processing with a lean API https://github.com/henvic/picel on my
    own time


    After feeling comfortable with it, I sold the idea of building the we CLI tool



    * this statement isn't guaranteed.

    View full-size slide

  16. Static executable binaries = less headaches
    ✤ No dependency hell
    ✤ No virtual machines
    ✤ Implication: disadvantage for interpreted languages

    View full-size slide

  17. Why not C
    ✤ No protection against programmer mistakes
    ✤ No code coverage suite for high-level programming
    ✤ Pointers are too low level (memory based, not object based)
    ✤ Supporting different platforms is a hell.

    View full-size slide

  18. Why not Bourne shell
    ✤ Environment
    ✤ No code coverage tool
    ✤ External dependencies hell (curl, etc)
    ✤ Performance
    ✤ OS X stuck in outdated 9 years old bash (shame on you, GNU GPLv3)

    View full-size slide

  19. Why not JavaScript (ES6 + Node.JS)
    ✤ Almost good enough, but…
    ✤ Environment hell
    ✤ V8 virtual machine is light-weight, but not light enough
    ✤ requiring files is not cheap
    ✤ Dependency hell
    ✤ Can't trust npm today (and probably not in anything less than 5-years)

    View full-size slide

  20. Why not Java, Scala, or…
    ✤ JVM
    ✤ Performance
    ✤ Lack of culture for this use-case
    ✤ Not trivial to try new things (Java 8 is way better, actually)

    View full-size slide

  21. Why not Python, Ruby, or interpreted ones…
    ✤ Almost good enough, but…
    ✤ Environment hell
    ✤ Versioning issues (less dramatic than Node)
    ✤ Less files to require than npm ecosystem, still some…
    ✤ Dependency hell
    ✤ Why bother with runtimes?

    View full-size slide

  22. LLVM is a killer compiler infrastructure…
    Why not Swift?
    ✤ A really serious competitor to Go on this area, but…


    Ecosystem is missing, hence couldn't even consider.

    View full-size slide

  23. Go for-the-win
    ✤ Emerging, proven language by Google
    ✤ docker, Kubernetes, and many other great tools are written in Go
    ✤ ______ is written in Go
    ✤ Sexy syntax, simple API
    ✤ Garbage collected, small static binaries

    View full-size slide

  24. Programming in Go vs. Node.js
    ✤ Static binaries vs. Environment nightmare
    ✤ Lean garbage collector vs. Virtual machine weight
    ✤ Almost no experience vs. Quite experienced
    ✤ Reliable, above average quality dependencies vs. lots of dependencies
    ✤ Powerful, above average quality built-in dependencies vs. fewer
    ✤ Compiled, better performance vs. interpreted, high startup cost for CLI tools
    ✤ Go routines easier concurrency support vs. Promises, events

    View full-size slide

  25. Infrastructure & tests
    ✤ Complexity tests (unchecked errors, CRAP, etc)
    ✤ Missing documentation
    ✤ Unit tests
    ✤ Integration tests (simulated environment)
    ✤ Functional tests (real environments, can destroy things)

    View full-size slide

  26. Configuration
    ✤ .ini format >>
    JSON, JSON 5,
    XML, etc.

    View full-size slide

  27. Commands, subcommands, flags, persistent flags,
    suggestions, help command auto generation…
    ✤ You want all that: use a framework
    ✤ Don't forget about autocompletion

    View full-size slide

  28. Autocompletion
    ✤ several levels of completeness and trade-offs
    ✤ from easy to very complex…
    ✤ zsh's autocomplete is very powerful:

    http://zsh.sourceforge.net/Doc/Release/Completion-System.html
    ✤ asking yourself: support autocomplete on what shells?

    zsh? bash? fish?

    View full-size slide

  29. Simple autocompletion for bash [and zsh
    trough plugin]
    ✤ Add hidden "we autocomplete" command
    ✤ Traverses all commands and flags, filtering by arguments
    ✤ Prints out an output command or flag per line
    ✤ Uses custom function (next slide) to add dead simple (but limited) autocomplete functionality
    ✤ Works on bash natively
    ✤ Doesn't work on fish
    ✤ Works on zsh with plugin (bash_complete for zsh) we autocomplete script loads

    View full-size slide

  30. ANSI escape code (colors, bold, italic…)
    ✤ You don't have to use rainbows. 256-color and even 24 bit true color
    support is possible nowadays on some shells, but your user loses control
    and you don't want it
    ✤ Remember you don't have exact control over the color palette your users
    use for ANSI colors. Just expect green to look green and red to look red. Not
    #ff0000 or #cc0000 or whatever you use
    ✤ Don't use fancy experimental features such as opening images on the
    terminal unless you like death threats (or have an experimental use case)

    View full-size slide

  31. “oh, it compiles for Windows. So, I am going to setup a SSH
    server on a Windows 10 vm and all should work”… Except…
    ✤ Good luck with setting up a nice SSH setup on Windows with keys and everything.

    95 > 10 ⇒ Windows 95 > Windows 10
    ✤ Good luck with autocomplete (kidding, just forget about')
    ✤ Wait, where are my colors? What are these weird characters?

    "fix": —no-color by default
    ✤ 10+ consoles: cmd, PowerShell, PowerCmd, cygwin, bash, …

    All inconsistent with each other (or it wouldn't be Windows)
    ✤ new bash support solves everything (just kidding - it's a lie)
    ✤ Everything can go goes wrong

    View full-size slide

  32. How to distribute?
    ✤ curl | bash issues: security, errors, etc.
    ✤ need solid, replicable, release procedures to avoid breaking production
    (which you don't have control, probably)
    ✤ need auto-update strategy that doesn't suck

    View full-size slide

  33. Release procedure
    ✤ git status should report directory is clean
    ✤ check for unchecked errors
    ✤ check for code issues and defects
    ✤ run tests (but functional)
    ✤ run tests (but functional) on a docker instance
    ✤ run functional tests on OS X, Linux, and Windows virtual machines
    ✤ ask for a release tag
    ✤ edit release tag message
    ✤ publish (if for the stable channel, publish on unstable actually)
    ✤ push tag to GitHub
    ✤ download release and run functional tests on OS X, Linux, and Windows virtual machines
    ✤ if for the stable channel, publish (promote) release [from the unstable] to the stable channel
    ✤ if for the stable channel, download release and run functional tests on OS X, Linux, and Windows virtual machines to report any abnormalitiesa

    View full-size slide

  34. Images ripped from
    ✤ https://en.wikipedia.org/wiki/System_console
    ✤ https://commons.wikimedia.org/wiki/File:IBM_1620.jpg
    ✤ https://commons.wikimedia.org/wiki/File:ASR-33_at_CHM.agr.jpg
    ✤ https://commons.wikimedia.org/wiki/File:Ibm_px_xt_color.jpg
    ✤ https://www.youtube.com/watch?v=RybNI0KB1bg
    ✤ https://en.wikipedia.org/wiki/File:Graveling.jpg
    ✤ http://deadlikeme.wikia.com/wiki/Dead_Like_Me_Wiki
    ✤ https://lastlog.de/blog/posts/nodejs_on_nixos_status.html
    ✤ http://maxogden.com/nested-dependencies.html

    View full-size slide