Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Writing a modern CLI tool

Writing a modern CLI tool

Talk I gave at work about my experience writing a modern CLI tool and why I have chosen Go for writing it.

Henrique Vicente

September 02, 2016
Tweet

More Decks by Henrique Vicente

Other Decks in Programming

Transcript

  1. Date
    Writing a modern CLI tool.
    Henrique Vicente
    27th July, 2016

    View Slide

  2. Originally I/O happened via a physically
    connected system console. Then…

    View Slide

  3. View Slide

  4. From Time-sharing to
    today’s iTerm
    ✤ Type on the terminal


    cat ~/Downloads/behind-the-firewall.gif | imgcat

    View Slide

  5. IBM XT
    ✤ First computer coming with standard hard-
    drive

    View Slide

  6. Classic hypothetical program
    1. Takes an input
    2. Does some magic
    3. Writes to output

    View Slide

  7. POSIX real world program
    ✤ Standard in / out streams (stdin, stdout, stderr)
    ✤ Environment variables
    ✤ Working directory
    ✤ Arguments
    ✤ Exit code
    ✤ parent process, process id, process group,

    View Slide

  8. What you want
    ✤ Static binaries
    ✤ Easy and secure distribution channel
    ✤ Easy to program
    ✤ Garbage collection
    ✤ High-level programming language
    ✤ Modern, clean, safe (type safety)
    ✤ Avoid the hype (unless if its arguments win)
    ✤ Lightweight startup for short-lived processes
    ✤ Great performance for long-lived daemons
    ✤ Reliable tests
    ✤ Culture
    ✤ No unbundled or dynamically linked
    dependencies
    ✤ No interpretation or virtual machines
    ✤ Trustworthy dependency managing

    View Slide

  9. stdout and stderr unexpected merging issues

    View Slide

  10. stdout and stderr unexpected merging issues
    ✤ ordering sort (e.g., verifying what is printed on tests)
    ✤ "updating" output (i.e., progress bar) modifying what you get

    View Slide

  11. View Slide

  12. What about the children?

    View Slide

  13. Child processes
    ✤ Zombies, orphans…
    ✤ Quite hard to work with

    reapers, actually.

    View Slide

  14. From “gh” to “we”

    View Slide

  15. View Slide

  16. Maintenance hell
    ✤ No tests
    ✤ Node.js 0.8 was still borning, API changes
    ✤ dumb npm packages… left-pad ¯\_(ツ)_/¯
    ✤ dependency hell
    ✤ callback hell: no Promise()
    ✤ At certain point [I thought?] people hated me

    View Slide

  17. View Slide

  18. Choosing a language for WeDeploy’s CLI

    View Slide

  19. Build one pet project before committing to an
    unfamiliar language or ecosystem
    ✤ I wrote picel: is a light-weight, blazing fast* REST-ful micro service for
    image processing with a lean API https://github.com/henvic/picel on my
    own time


    After feeling comfortable with it, I sold the idea of building the we CLI tool



    * this statement isn't guaranteed.

    View Slide

  20. Static executable binaries = less headaches
    ✤ No dependency hell
    ✤ No virtual machines
    ✤ Implication: disadvantage for interpreted languages

    View Slide

  21. Why not C
    ✤ No protection against programmer mistakes
    ✤ No code coverage suite for high-level programming
    ✤ Pointers are too low level (memory based, not object based)
    ✤ Supporting different platforms is a hell.

    View Slide

  22. Why not Bourne shell
    ✤ Environment
    ✤ No code coverage tool
    ✤ External dependencies hell (curl, etc)
    ✤ Performance
    ✤ OS X stuck in outdated 9 years old bash (shame on you, GNU GPLv3)

    View Slide

  23. Why not JavaScript (ES6 + Node.JS)
    ✤ Almost good enough, but…
    ✤ Environment hell
    ✤ V8 virtual machine is light-weight, but not light enough
    ✤ requiring files is not cheap
    ✤ Dependency hell
    ✤ Can't trust npm today (and probably not in anything less than 5-years)

    View Slide

  24. Why not Java, Scala, or…
    ✤ JVM
    ✤ Performance
    ✤ Lack of culture for this use-case
    ✤ Not trivial to try new things (Java 8 is way better, actually)

    View Slide

  25. Why not Python, Ruby, or interpreted ones…
    ✤ Almost good enough, but…
    ✤ Environment hell
    ✤ Versioning issues (less dramatic than Node)
    ✤ Less files to require than npm ecosystem, still some…
    ✤ Dependency hell
    ✤ Why bother with runtimes?

    View Slide

  26. LLVM is a killer compiler infrastructure…
    Why not Swift?
    ✤ A really serious competitor to Go on this area, but…


    Ecosystem is missing, hence couldn't even consider.

    View Slide

  27. Go for-the-win
    ✤ Emerging, proven language by Google
    ✤ docker, Kubernetes, and many other great tools are written in Go
    ✤ ______ is written in Go
    ✤ Sexy syntax, simple API
    ✤ Garbage collected, small static binaries

    View Slide

  28. Programming in Go vs. Node.js
    ✤ Static binaries vs. Environment nightmare
    ✤ Lean garbage collector vs. Virtual machine weight
    ✤ Almost no experience vs. Quite experienced
    ✤ Reliable, above average quality dependencies vs. lots of dependencies
    ✤ Powerful, above average quality built-in dependencies vs. fewer
    ✤ Compiled, better performance vs. interpreted, high startup cost for CLI tools
    ✤ Go routines easier concurrency support vs. Promises, events

    View Slide

  29. Infrastructure & tests
    ✤ Complexity tests (unchecked errors, CRAP, etc)
    ✤ Missing documentation
    ✤ Unit tests
    ✤ Integration tests (simulated environment)
    ✤ Functional tests (real environments, can destroy things)

    View Slide

  30. Configuration
    ✤ .ini format >>
    JSON, JSON 5,
    XML, etc.

    View Slide

  31. View Slide

  32. Commands, subcommands, flags, persistent flags,
    suggestions, help command auto generation…
    ✤ You want all that: use a framework
    ✤ Don't forget about autocompletion

    View Slide

  33. View Slide

  34. Autocompletion
    ✤ several levels of completeness and trade-offs
    ✤ from easy to very complex…
    ✤ zsh's autocomplete is very powerful:

    http://zsh.sourceforge.net/Doc/Release/Completion-System.html
    ✤ asking yourself: support autocomplete on what shells?

    zsh? bash? fish?

    View Slide

  35. Simple autocompletion for bash [and zsh
    trough plugin]
    ✤ Add hidden "we autocomplete" command
    ✤ Traverses all commands and flags, filtering by arguments
    ✤ Prints out an output command or flag per line
    ✤ Uses custom function (next slide) to add dead simple (but limited) autocomplete functionality
    ✤ Works on bash natively
    ✤ Doesn't work on fish
    ✤ Works on zsh with plugin (bash_complete for zsh) we autocomplete script loads

    View Slide

  36. View Slide

  37. ANSI escape code (colors, bold, italic…)
    ✤ You don't have to use rainbows. 256-color and even 24 bit true color
    support is possible nowadays on some shells, but your user loses control
    and you don't want it
    ✤ Remember you don't have exact control over the color palette your users
    use for ANSI colors. Just expect green to look green and red to look red. Not
    #ff0000 or #cc0000 or whatever you use
    ✤ Don't use fancy experimental features such as opening images on the
    terminal unless you like death threats (or have an experimental use case)

    View Slide

  38. “oh, it compiles for Windows. So, I am going to setup a SSH
    server on a Windows 10 vm and all should work”… Except…
    ✤ Good luck with setting up a nice SSH setup on Windows with keys and everything.

    95 > 10 ⇒ Windows 95 > Windows 10
    ✤ Good luck with autocomplete (kidding, just forget about')
    ✤ Wait, where are my colors? What are these weird characters?

    "fix": —no-color by default
    ✤ 10+ consoles: cmd, PowerShell, PowerCmd, cygwin, bash, …

    All inconsistent with each other (or it wouldn't be Windows)
    ✤ new bash support solves everything (just kidding - it's a lie)
    ✤ Everything can go goes wrong

    View Slide

  39. How to distribute?
    ✤ curl | bash issues: security, errors, etc.
    ✤ need solid, replicable, release procedures to avoid breaking production
    (which you don't have control, probably)
    ✤ need auto-update strategy that doesn't suck

    View Slide

  40. Release procedure
    ✤ git status should report directory is clean
    ✤ check for unchecked errors
    ✤ check for code issues and defects
    ✤ run tests (but functional)
    ✤ run tests (but functional) on a docker instance
    ✤ run functional tests on OS X, Linux, and Windows virtual machines
    ✤ ask for a release tag
    ✤ edit release tag message
    ✤ publish (if for the stable channel, publish on unstable actually)
    ✤ push tag to GitHub
    ✤ download release and run functional tests on OS X, Linux, and Windows virtual machines
    ✤ if for the stable channel, publish (promote) release [from the unstable] to the stable channel
    ✤ if for the stable channel, download release and run functional tests on OS X, Linux, and Windows virtual machines to report any abnormalitiesa

    View Slide

  41. Questions?

    View Slide

  42. Images ripped from
    ✤ https://en.wikipedia.org/wiki/System_console
    ✤ https://commons.wikimedia.org/wiki/File:IBM_1620.jpg
    ✤ https://commons.wikimedia.org/wiki/File:ASR-33_at_CHM.agr.jpg
    ✤ https://commons.wikimedia.org/wiki/File:Ibm_px_xt_color.jpg
    ✤ https://www.youtube.com/watch?v=RybNI0KB1bg
    ✤ https://en.wikipedia.org/wiki/File:Graveling.jpg
    ✤ http://deadlikeme.wikia.com/wiki/Dead_Like_Me_Wiki
    ✤ https://lastlog.de/blog/posts/nodejs_on_nixos_status.html
    ✤ http://maxogden.com/nested-dependencies.html

    View Slide