How the Command-Line Can Empower You

Presented live on 2018-11-27 as a webinar.

As a developer, you probably use a modern IDE that lets you write, debug, test, and deploy your code quickly and easily. But your job often includes activities that need to be done outside your IDE, such as working with APIs, creating screenshots, or massaging data. I want to show you how the command-line can simplify, improve, or even automate some of these tasks

I'll provide an overview of utilities that will give you more ways to get your work done, and get it done faster. Using real-world examples, you'll learn how to type less in the terminal, search your files with ease, manipulate images and JSON files, write code automatically, and more. All without needing to point, click, or swipe!

If you are curious about command-line tools, or want to learn more about their awesome capabilities, this talk is for you.

This presentation will focus on software available for macOS computers, but Linux and Windows users will also benefit, as many of the tools mentioned are cross-platform.

Transcript

1. How The Command-Line Can Empower You Eric Fung • @gnufmuffin
2. ‣ /remind me Lunch for Gemma at noon ‣ inspect(document.body); ‣ efung@host:~$ ls -l ‣ C:\>dir /w Tux image by Larry Ewing lewing@isc.tamu.edu
3. Command-line interface: a way of interacting with a computer by giving it commands using text How The Command-Line Can Empower You • Eric Fung
4. But… ! why would I? How The Command-Line Can Empower You • Eric Fung
5. Image credit: burst.shopify.com
6. Why should I use the command line? ‣ Gets you results more quickly ‣ Lets you control the output ‣ Be less reliant on others ‣ Makes you more versatile ‣ Gives you transferable skills How The Command-Line Can Empower You • Eric Fung
7. What we'll cover ‣ Basic command-line setup ‣ Typing less ‣ Navigating folders ‣ Working with Finder and clipboard ‣ Searching files ‣ Integrating with remote APIs ‣ Dealing with JSON data ‣ Manipulating images How The Command-Line Can Empower You • Eric Fung
8. The basics • Terminal • Shell • Package manager Image credit: burst.shopify.com
9. Terminal: iTerm2 ‣ Extensive features and customization ‣ Favorites: ‣ ⌘-click to open URLs and files ‣ Highlights all search results ‣ ⌘-; autocompletes from anything on screen How The Command-Line Can Empower You • Eric Fung
10. Shell: bash ‣ macOS default shell ‣ Widely used → lots of support ‣ May want to install newer version How The Command-Line Can Empower You • Eric Fung
11. Package manager: Homebrew ‣ Many packages available (over 4000+) ‣ Software, fonts, plugins, GUI apps ‣ Self-healing How The Command-Line Can Empower You • Eric Fung
12. Typing less Image credit: burst.shopify.com
13. Tab completion ‣ Autocompletes commands, options, files, directories ‣ Context-sensitive ‣ Installed with many Homebrew packages How The Command-Line Can Empower You • Eric Fung
14. Demo: command completion
15. Demo: file and folder completion
16. Working with Finder Image credit: burst.shopify.com
17. From shell to Finder ‣ open <file> ‣ Equivalent to double-clicking on <file> ‣ open . ‣ Opens Finder with current directory ‣ open <url> ‣ Opens default browser to url How The Command-Line Can Empower You • Eric Fung
18. Demo: macOS open
19. From Finder to shell ‣ Drag-and-drop file(s) onto Terminal ‣ Pastes their filename(s), space-separated How The Command-Line Can Empower You • Eric Fung
20. Demo: Drag files to shell → paste filename
21. Navigation Image credit: burst.shopify.com
22. cd 101 ‣ cd app/src/main/res/layout ‣ cd ../../java ‣ cd - ‣ Go to previous directory How The Command-Line Can Empower You • Eric Fung
23. Demo: enhanced navigation
24. autojump ‣ Provide fast access to directories further away ‣ Selects directories based on usage ‣ Caveat: must be visited first How The Command-Line Can Empower You • Eric Fung
25. autojump ‣ j name ‣ Go to most-used directory containing name ‣ jc name ‣ Go to a subdirectory containing name ‣ j name <Tab> ‣ Offers choices when there are multiple matches How The Command-Line Can Empower You • Eric Fung
26. Searching Image credit: burst.shopify.com
27. Demo: enhanced file search
28. ripgrep ‣ Supports grep options ‣ Much faster ‣ Knows about file types ‣ Respects revision control ignores ‣ Support for emoji ! How The Command-Line Can Empower You • Eric Fung
29. Demo: Search Android documentation offline
30. I/O Image credit: burst.shopify.com
31. I/O 101 ‣ Streams: stdin and stdout ‣ Redirection: > and < ‣ Pipelines: | How The Command-Line Can Empower You • Eric Fung
32. Working with Clipboard ‣ pbcopy (⌘-C for the shell) ‣ Reads from stdin, copies to clipboard ‣ pbpaste (⌘-V for the shell) ‣ Copies contents of clipboard, writes to stdout How The Command-Line Can Empower You • Eric Fung
33. The Web Image credit: burst.shopify.com
34. Demo: Web from command-line
35. curl ‣ Comprehensive client for working with Internet protocol transfers ‣ "the number of features will make your head spin!" ‣ Supports HTTP, SMTP, IMAP, RTSP, and lots more ‣ Designed to be unattended How The Command-Line Can Empower You • Eric Fung
36. Debugging endpoints: --include $ curl 'https://search.icons8.com/api/iconsets/v4/search' --include HTTP/2 200 server: keycdn-engine date: Thu, 09 Aug 2018 19:39:52 GMT content-type: application/json; charset=utf-8 content-length: 96 cache-control: max-age=604800 expires: Thu, 16 Aug 2018 19:39:52 GMT x-cache: HIT x-shield: active x-edge-location: usch access-control-allow-origin: * accept-ranges: bytes How The Command-Line Can Empower You • Eric Fung
37. Demo: POSTing data to an API
38. POSTing data to an API $ curl -X POST \ --header "Content-Type: application/json" \ https://jsonplaceholder.typicode.com/posts \ --data-ascii @body How The Command-Line Can Empower You • Eric Fung
39. JSON Image credit: burst.shopify.com
40. Demo: Visualize data
41. gron ‣ Turns JSON into line-oriented data ‣ Then, search with tools like ripgrep ‣ Identify individual array elements How The Command-Line Can Empower You • Eric Fung
42. Example gron output How The Command-Line Can Empower You • Eric Fung
43. Demo: Slice and dice data
44. jq ‣ Powerful, flexible processor ‣ Transform data you have, to data you want ‣ JSON, CSV, URI, plain text ‣ Concise syntax ‣ Handles JSON Lines, e.g. log files How The Command-Line Can Empower You • Eric Fung
45. Example JSON data { "photos": { "page": 1, "pages": 125, "total": 100, "photo": [ { "id": "2348183", "title": "Fuzzy cat", "url": "https://example.com/1.jpg" }, … } How The Command-Line Can Empower You • Eric Fung
46. jq example $ jq -r '.photos.photo' flickr.json [ { "id": "2348183", "title": "Fuzzy cat", "url": "https://example.com/1.jpg" }, { "id": "7937001", "title": "Good boi", "url": "https://example.com/2.jpg" }, … ] How The Command-Line Can Empower You • Eric Fung
47. jq example $ jq -r '.photos.photo | .[]' flickr.json { "id": "2348183", "title": "Fuzzy cat", "url": "https://example.com/1.jpg" } { "id": "7937001", "title": "Good boi", "url": "https://example.com/2.jpg" } … How The Command-Line Can Empower You • Eric Fung
48. Extract values from array of objects $ jq -r '.photos.photo | .[] | .title' flickr.json AJH-20180814-467-Edit.jpg East Garston series: quintessential English countryside IMG_20180723_215912_323 P1030365.jpg Frankrijk 2018 Shirts on Sale (+6 03 6143 5225)Contact Us 03 6143 5225 https://t.co/dBTffHWW99 Tratamiento Alternativo Para El Colon Irritable How The Command-Line Can Empower You • Eric Fung
49. Image manipulation Image credit: burst.shopify.com
50. Compositing images How The Command-Line Can Empower You • Eric Fung
51. ImageMagick ‣ Suite of command-line utilities ‣ Create, edit, compose, or convert bitmaps ‣ Read/write 200 image formats ‣ Handles really large images How The Command-Line Can Empower You • Eric Fung
52. Example: Composite images How The Command-Line Can Empower You • Eric Fung
53. bash function using IM's montage side_by_side() { montage -background lightgray -pointsize 60 \ -label '%t' \ $* \ -tile x1 -geometry '+30+30' \ combined.png } ‣ $* is all the function's arguments ‣ -label '%t' is IM syntax defining label from each argument's base filename How The Command-Line Can Empower You • Eric Fung
54. Usage $ side_by_side before.png after.png How The Command-Line Can Empower You • Eric Fung
55. Example: Batch processing How The Command-Line Can Empower You • Eric Fung
56. Example: Batch processing How The Command-Line Can Empower You • Eric Fung
57. bash loop with IM's convert for f in image1.jpg image2.jpg image3.jpg; do convert -brightness-contrast -30 \ -gravity SouthWest \ -crop 512x512+0+0 \ $f \ `basename $f .jpg`.cropped.jpg done How The Command-Line Can Empower You • Eric Fung
58. bash loop with IM's convert for f in image1.jpg image2.jpg image3.jpg; do convert -brightness-contrast -30 \ -gravity SouthWest \ -crop 512x512+0+0 \ $f \ `basename $f .jpg`.cropped.jpg done ‣ for loop assigns each filename to variable f ‣ gravity defines origin ‣ basename removes extension, backticks inline result How The Command-Line Can Empower You • Eric Fung
59. Summary Image credit: burst.shopify.com
60. What we've learned ‣ Command-line utilities are powerful tools ‣ Can be combined in different ways to get stuff done ‣ They can make you more productive ‣ Excellent additions to your developer toolbelt How The Command-Line Can Empower You • Eric Fung
61. End ‣ Twitter @gnufmuffin • @ShopifyEng ‣ Webinar Recording on Zoom ‏ with demo videos ‣ Slides speakerdeck.com/efung ‏ static deck How The Command-Line Can Empower You • Eric Fung
62. Appendix: How to install (1/2) ‣ iTerm2: full-featured terminal ‣ Homebrew: package manager ‣ bash completion: completions for macOS builtins ‣ ! 1 bash-completion ‣ autojump (j): navigate directories faster ‣ ! autojump ‣ ripgrep (rg): fast, intelligent search tool ‣ ! ripgrep 1 ! denotes Homebrew package name How The Command-Line Can Empower You • Eric Fung
63. Appendix: How to install (2/2) ‣ curl: transfer data with URLs ‣ ! curl ‣ everything curl book ‣ gron: make JSON line-oriented and searchable ‣ ! gron ‣ jq: transform and query JSON ‣ ! jq ‣ jq manual ‣ jq cookbook ‣ ImageMagick: utilities for image manipulation ‣ ! imagemagick ‣ usage examples How The Command-Line Can Empower You • Eric Fung