Writing a modern CLI tool

Writing a modern CLI tool

Talk I gave at work about my experience writing a modern CLI tool and why I have chosen Go for writing it.


Henrique Vicente

September 02, 2016


  1. Date Writing a modern CLI tool. Henrique Vicente 27th July,

  2. Originally I/O happened via a physically connected system console. Then…

  3. None
  4. From Time-sharing to today’s iTerm ✤ Type on the terminal

 cat ~/Downloads/behind-the-firewall.gif | imgcat
  5. IBM XT ✤ First computer coming with standard hard- drive

  6. Classic hypothetical program 1. Takes an input 2. Does some

    magic 3. Writes to output
  7. POSIX real world program ✤ Standard in / out streams

    (stdin, stdout, stderr) ✤ Environment variables ✤ Working directory ✤ Arguments ✤ Exit code ✤ parent process, process id, process group,
  8. What you want ✤ Static binaries ✤ Easy and secure

    distribution channel ✤ Easy to program ✤ Garbage collection ✤ High-level programming language ✤ Modern, clean, safe (type safety) ✤ Avoid the hype (unless if its arguments win) ✤ Lightweight startup for short-lived processes ✤ Great performance for long-lived daemons ✤ Reliable tests ✤ Culture ✤ No unbundled or dynamically linked dependencies ✤ No interpretation or virtual machines ✤ Trustworthy dependency managing
  9. stdout and stderr unexpected merging issues

  10. stdout and stderr unexpected merging issues ✤ ordering sort (e.g.,

    verifying what is printed on tests) ✤ "updating" output (i.e., progress bar) modifying what you get
  11. None
  12. What about the children?

  13. Child processes ✤ Zombies, orphans… ✤ Quite hard to work

 reapers, actually.
  14. From “gh” to “we”

  15. None
  16. Maintenance hell ✤ No tests ✤ Node.js 0.8 was still

    borning, API changes ✤ dumb npm packages… left-pad ¯\_(ツ)_/¯ ✤ dependency hell ✤ callback hell: no Promise() ✤ At certain point [I thought?] people hated me
  17. None
  18. Choosing a language for WeDeploy’s CLI

  19. Build one pet project before committing to an unfamiliar language

    or ecosystem ✤ I wrote picel: is a light-weight, blazing fast* REST-ful micro service for image processing with a lean API https://github.com/henvic/picel on my own time
 After feeling comfortable with it, I sold the idea of building the we CLI tool
 * this statement isn't guaranteed.
  20. Static executable binaries = less headaches ✤ No dependency hell

    ✤ No virtual machines ✤ Implication: disadvantage for interpreted languages
  21. Why not C ✤ No protection against programmer mistakes ✤

    No code coverage suite for high-level programming ✤ Pointers are too low level (memory based, not object based) ✤ Supporting different platforms is a hell.
  22. Why not Bourne shell ✤ Environment ✤ No code coverage

    tool ✤ External dependencies hell (curl, etc) ✤ Performance ✤ OS X stuck in outdated 9 years old bash (shame on you, GNU GPLv3)
  23. Why not JavaScript (ES6 + Node.JS) ✤ Almost good enough,

    but… ✤ Environment hell ✤ V8 virtual machine is light-weight, but not light enough ✤ requiring files is not cheap ✤ Dependency hell ✤ Can't trust npm today (and probably not in anything less than 5-years)
  24. Why not Java, Scala, or… ✤ JVM ✤ Performance ✤

    Lack of culture for this use-case ✤ Not trivial to try new things (Java 8 is way better, actually)
  25. Why not Python, Ruby, or interpreted ones… ✤ Almost good

    enough, but… ✤ Environment hell ✤ Versioning issues (less dramatic than Node) ✤ Less files to require than npm ecosystem, still some… ✤ Dependency hell ✤ Why bother with runtimes?
  26. LLVM is a killer compiler infrastructure… Why not Swift? ✤

    A really serious competitor to Go on this area, but…
 Ecosystem is missing, hence couldn't even consider.
  27. Go for-the-win ✤ Emerging, proven language by Google ✤ docker,

    Kubernetes, and many other great tools are written in Go ✤ ______ is written in Go ✤ Sexy syntax, simple API ✤ Garbage collected, small static binaries
  28. Programming in Go vs. Node.js ✤ Static binaries vs. Environment

    nightmare ✤ Lean garbage collector vs. Virtual machine weight ✤ Almost no experience vs. Quite experienced ✤ Reliable, above average quality dependencies vs. lots of dependencies ✤ Powerful, above average quality built-in dependencies vs. fewer ✤ Compiled, better performance vs. interpreted, high startup cost for CLI tools ✤ Go routines easier concurrency support vs. Promises, events
  29. Infrastructure & tests ✤ Complexity tests (unchecked errors, CRAP, etc)

    ✤ Missing documentation ✤ Unit tests ✤ Integration tests (simulated environment) ✤ Functional tests (real environments, can destroy things)
  30. Configuration ✤ .ini format >> JSON, JSON 5, XML, etc.

  31. None
  32. Commands, subcommands, flags, persistent flags, suggestions, help command auto generation…

    ✤ You want all that: use a framework ✤ Don't forget about autocompletion
  33. None
  34. Autocompletion ✤ several levels of completeness and trade-offs ✤ from

    easy to very complex… ✤ zsh's autocomplete is very powerful:
 http://zsh.sourceforge.net/Doc/Release/Completion-System.html ✤ asking yourself: support autocomplete on what shells?
 zsh? bash? fish?
  35. Simple autocompletion for bash [and zsh trough plugin] ✤ Add

    hidden "we autocomplete" command ✤ Traverses all commands and flags, filtering by arguments ✤ Prints out an output command or flag per line ✤ Uses custom function (next slide) to add dead simple (but limited) autocomplete functionality ✤ Works on bash natively ✤ Doesn't work on fish ✤ Works on zsh with plugin (bash_complete for zsh) we autocomplete script loads
  36. None
  37. ANSI escape code (colors, bold, italic…) ✤ You don't have

    to use rainbows. 256-color and even 24 bit true color support is possible nowadays on some shells, but your user loses control and you don't want it ✤ Remember you don't have exact control over the color palette your users use for ANSI colors. Just expect green to look green and red to look red. Not #ff0000 or #cc0000 or whatever you use ✤ Don't use fancy experimental features such as opening images on the terminal unless you like death threats (or have an experimental use case)
  38. “oh, it compiles for Windows. So, I am going to

    setup a SSH server on a Windows 10 vm and all should work”… Except… ✤ Good luck with setting up a nice SSH setup on Windows with keys and everything.
 95 > 10 ⇒ Windows 95 > Windows 10 ✤ Good luck with autocomplete (kidding, just forget about') ✤ Wait, where are my colors? What are these weird characters?
 "fix": —no-color by default ✤ 10+ consoles: cmd, PowerShell, PowerCmd, cygwin, bash, …
 All inconsistent with each other (or it wouldn't be Windows) ✤ new bash support solves everything (just kidding - it's a lie) ✤ Everything can go goes wrong
  39. How to distribute? ✤ curl | bash issues: security, errors,

    etc. ✤ need solid, replicable, release procedures to avoid breaking production (which you don't have control, probably) ✤ need auto-update strategy that doesn't suck
  40. Release procedure ✤ git status should report directory is clean

    ✤ check for unchecked errors ✤ check for code issues and defects ✤ run tests (but functional) ✤ run tests (but functional) on a docker instance ✤ run functional tests on OS X, Linux, and Windows virtual machines ✤ ask for a release tag ✤ edit release tag message ✤ publish (if for the stable channel, publish on unstable actually) ✤ push tag to GitHub ✤ download release and run functional tests on OS X, Linux, and Windows virtual machines ✤ if for the stable channel, publish (promote) release [from the unstable] to the stable channel ✤ if for the stable channel, download release and run functional tests on OS X, Linux, and Windows virtual machines to report any abnormalitiesa
  41. Questions?

  42. Images ripped from ✤ https://en.wikipedia.org/wiki/System_console ✤ https://commons.wikimedia.org/wiki/File:IBM_1620.jpg ✤ https://commons.wikimedia.org/wiki/File:ASR-33_at_CHM.agr.jpg ✤

    https://commons.wikimedia.org/wiki/File:Ibm_px_xt_color.jpg ✤ https://www.youtube.com/watch?v=RybNI0KB1bg ✤ https://en.wikipedia.org/wiki/File:Graveling.jpg ✤ http://deadlikeme.wikia.com/wiki/Dead_Like_Me_Wiki ✤ https://lastlog.de/blog/posts/nodejs_on_nixos_status.html ✤ http://maxogden.com/nested-dependencies.html