Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How the Command-Line Can Give You Superpowers (GDG Toronto Android)

How the Command-Line Can Give You Superpowers (GDG Toronto Android)

I want to show you how the command line can simplify, and even automate some of the things Android developers need to do. Learning how to use command line tools will give you more ways to get your work done.

Using Mac OS X as the platform, I’ll demonstrate real-world examples of how you can boost your productivity when navigating, searching, working with JSON and Web responses, manipulating images, automating interaction with devices, scripting Java and Kotlin code generation, and more. All from a terminal, without using a GUI

Note that the slides don't include the videos/animations.

Video recording: https://www.youtube.com/watch?v=YMS_AplpmZg

877215a85ea128b67b4334142a6df260?s=128

Eric Fung

August 09, 2018
Tweet

More Decks by Eric Fung

Other Decks in Programming

Transcript

  1. How The Command-Line Can Give You Superpowers Eric Fung GDG

    Toronto Android, 2018-08-09
  2. Both screenshots [Public domain], via Wikimedia Commons

  3. Linux screenshot © 2011, Anderson Silva. Released under the Open

    Publication License — gcc screenshot [Public domain], via Wikimedia Commons
  4. Why Learn the Command-Line? — Gives you more ways to

    solve problems — Learn tools to get results quickly — Makes you more valuable — Feel like a superhero How The Command-Line Can Give You Superpowers — @gnufmuffin
  5. Overview — Setup — Navigation — Search — I/O —

    JSON — Web — Images — Moving Things Around — Scripting How The Command-Line Can Give You Superpowers — @gnufmuffin
  6. Overview (of superpowers) — Setup — Navigation Teleportation — Search

    Locality Tracking — I/O Information Transferal — JSON Knowledge Manipulation — Web Arachnid Physiology — Images Skill Replication — Moving Things Around Telekinesis — Scripting Automatic Tasking How The Command-Line Can Give You Superpowers — @gnufmuffin
  7. Setup How The Command-Line Can Give You Superpowers — @gnufmuffin

  8. Terminal Emulator — Get it: https://iterm2.com/ How The Command-Line Can

    Give You Superpowers — @gnufmuffin
  9. ⌘-click URL to open it Click on filenames too How

    The Command-Line Can Give You Superpowers — @gnufmuffin
  10. Highlight all matches Regex support How The Command-Line Can Give

    You Superpowers — @gnufmuffin
  11. ⌘-; to autocomplete From anything that has ever appeared How

    The Command-Line Can Give You Superpowers — @gnufmuffin
  12. Command line shell — macOS default shell — may want

    to install newer version How The Command-Line Can Give You Superpowers — @gnufmuffin
  13. Package manager — GNU Linux/UNIX software — Get it: https://brew.sh/

    How The Command-Line Can Give You Superpowers — @gnufmuffin
  14. Homebrew — brew install — Read caveats and post-install notes!

    — brew update — brew upgrade How The Command-Line Can Give You Superpowers — @gnufmuffin
  15. Homebrew — brew uninstall — brew cleanup — Remove old

    versions to free up disk How The Command-Line Can Give You Superpowers — @gnufmuffin
  16. Completion — Press <Tab> instead of typing — Context sensitive:

    command, subcommand, options — Get it: $ brew install bash_completion How The Command-Line Can Give You Superpowers — @gnufmuffin
  17. adb completion — Completes install with APKs — Completes uninstall

    with application IDs — Get it: — https://android.googlesource.com/ platform/system/core/+/master/adb/ adb.bash — https://android.googlesource.com/ platform/build/+/master/envsetup.sh How The Command-Line Can Give You Superpowers — @gnufmuffin
  18. adb install completion How The Command-Line Can Give You Superpowers

    — @gnufmuffin
  19. adb uninstall completion How The Command-Line Can Give You Superpowers

    — @gnufmuffin
  20. Gradle completion — Works with standalone and wrapper — Maintains

    a cache — Implicit tasks disabled by default — Get it: brew install gradle-completion How The Command-Line Can Give You Superpowers — @gnufmuffin
  21. Gradle completion How The Command-Line Can Give You Superpowers —

    @gnufmuffin
  22. To Finder — open <thing> — Equivalent to double-clicking on

    <thing> — open image.png — Opens image in Preview — open . — Opens Finder with current directory How The Command-Line Can Give You Superpowers — @gnufmuffin
  23. open -R <file> — Opens enclosing folder in Finder, and

    highlights file — Handy for dragging files to upload, e.g. Slack How The Command-Line Can Give You Superpowers — @gnufmuffin
  24. open -R <file> How The Command-Line Can Give You Superpowers

    — @gnufmuffin
  25. open -a <app> <file> — Open file with a specific

    application — Useful if file does not have default handler — Tip: define a function or alias asopen() { open -a "/Applications/Android Studio.app" "$1" } How The Command-Line Can Give You Superpowers — @gnufmuffin
  26. From Finder — Drag-and-drop file(s) onto Terminal — Pastes their

    filename(s), space-separated How The Command-Line Can Give You Superpowers — @gnufmuffin
  27. Drag files to shell → paste filename How The Command-Line

    Can Give You Superpowers — @gnufmuffin
  28. Navigation How The Command-Line Can Give You Superpowers — @gnufmuffin

  29. cd path/to/file — Instead of cd path1㾑 cd path2 㾑

    cd path3㾑 — use path separators cd path1/path2/path3 How The Command-Line Can Give You Superpowers — @gnufmuffin
  30. cd - — Instead of $ pwd /Users/efung/src/github.com/example/project/app/src/main/java $ cd

    $ANDROID_SDK $ <do some stuff> $ cd ~/src/github.com/example/project/app/src/main/java — jump to previous directory $ cd $ANDROID_SDK $ <do some stuff> $ cd - How The Command-Line Can Give You Superpowers — @gnufmuffin
  31. Beyond cd — Reduce typing — Provide quicker access to

    folders further away — Intelligently select folders based on "frecency" How The Command-Line Can Give You Superpowers — @gnufmuffin
  32. Beyond cd — autojump: https://github.com/wting/autojump — z: https://github.com/rupa/z — fasd:

    https://github.com/clvv/fasd — Get it: brew install {autojump,fasd,z} How The Command-Line Can Give You Superpowers — @gnufmuffin
  33. autojump — j name — Go to most-used folder containing

    name — jc name — Go to a subdirectory containing name — j name <Tab> — Offers choices when there are multiple matches How The Command-Line Can Give You Superpowers — @gnufmuffin
  34. autojump How The Command-Line Can Give You Superpowers — @gnufmuffin

  35. Searching How The Command-Line Can Give You Superpowers — @gnufmuffin

  36. Beyond grep — The Silver Searcher: https://github.com/ggreer/ the_silver_searcher — ripgrep:

    https://github.com/BurntSushi/ripgrep — Get it: brew install {the_silver_searcher,ripgrep} How The Command-Line Can Give You Superpowers — @gnufmuffin
  37. The Silver Searcher — Really fast — Builtin file types

    — Ignores files from .gitignore How The Command-Line Can Give You Superpowers — @gnufmuffin
  38. ag <term> How The Command-Line Can Give You Superpowers —

    @gnufmuffin
  39. ag <term> --<filetype> -w How The Command-Line Can Give You

    Superpowers — @gnufmuffin
  40. Search Android documentation offline ag <term> $ANDROID_SDK/docs How The Command-Line

    Can Give You Superpowers — @gnufmuffin
  41. Spotlight (⌘-Space) — mdfind: Search metadata — mdls: List known

    attributes — e.g. show where an APK was downloaded from $ mdls -name kMDItemWhereFroms 1850000.apk kMDItemWhereFroms = ( "https://play.google.com/apps/publish/artifact/download?artifact=12 34737584826475678", "https://play.google.com/apps/publish/?account=9876232438554103210" ) How The Command-Line Can Give You Superpowers — @gnufmuffin
  42. I/O How The Command-Line Can Give You Superpowers — @gnufmuffin

  43. I/O Review — Streams: stdin and stdout — Redirection: >

    and < — Pipelines: | How The Command-Line Can Give You Superpowers — @gnufmuffin
  44. Pasteboard (⌘-C, ⌘-V) — pbpaste — Copies contents of pasteboard,

    writes to stdout — pbcopy — Reads from stdin, copies to pasteboard How The Command-Line Can Give You Superpowers — @gnufmuffin
  45. Copying and pasting from StackOverflow — Format code snippet —

    Write to file $ pbpaste | tidy -xml > file.xml How The Command-Line Can Give You Superpowers — @gnufmuffin
  46. Copying and pasting from StackOverflow How The Command-Line Can Give

    You Superpowers — @gnufmuffin
  47. JSON How The Command-Line Can Give You Superpowers — @gnufmuffin

  48. gron — Turns JSON data into line-oriented data — Then,

    search with line-oriented tools, like ag — Identify individual array elements — Get it: brew install gron How The Command-Line Can Give You Superpowers — @gnufmuffin
  49. Sample flickr.photos.search response { "photos": { "page": 1, "pages": "25074",

    "perpage": 7, "total": "250739", "photo": [ { "id": "29012739647", "owner": "159732495@N03", "secret": "39dd96b21d", "server": "1840", "farm": 2, "title": "BombSquad MOD (Unlocked)", "ispublic": 1, "isfriend": 0, "isfamily": 0 }, { "id": "43231929164", "owner": "159732495@N03", "secret": "56fd0acc30", "server": "1836", "farm": 2, "title": "MARVEL Strike Force MOD (Unlimited Energy)", "ispublic": 1, "isfriend": 0, "isfamily": 0 }, { "id": "43950331011", "owner": "7171588@N05", "secret": "c45ba149c1", "server": "1797", "farm": 2, "title": "New photo added to gallery", "ispublic": 1, "isfriend": 0, "isfamily": 0 }, { "id": "43950279251", "owner": "7171588@N05", "secret": "b9dc92ff2b", "server": "1836", "farm": 2, "title": "New photo added to gallery", "ispublic": 1, "isfriend": 0, "isfamily": 0 }, { "id": "43231327754", "owner": "150238411@N07", "secret": "dc3af72ee3", "server": "851", "farm": 1, "title": "New photo added to gallery", "ispublic": 1, "isfriend": 0, "isfamily": 0 }, { "id": "43901777582", "owner": "125141703@N02", "secret": "e76813d131", "server": "1775", "farm": 2, "title": "Samsung Galaxy Note 9, así encaja en el catálogo completo de smartphones Samsung en 2018", "ispublic": 1, "isfriend": 0, "isfamily": 0 }, { "id": "42140715770", "owner": "159732495@N03", "secret": "6b7156103c", "server": "1837", "farm": 2, "title": "WeTorrent Premium – Torrent Downloader (Cracked)", "ispublic": 1, "isfriend": 0, "isfamily": 0 } ] }, "stat": "ok" } How The Command-Line Can Give You Superpowers — @gnufmuffin
  50. Example gron output How The Command-Line Can Give You Superpowers

    — @gnufmuffin
  51. jq — Powerful, flexible processor — Hard to learn —

    Massage data into other formats — Get it: brew install jq How The Command-Line Can Give You Superpowers — @gnufmuffin
  52. Pretty printing $ jq '.' flickr.json { "photos": { "page":

    1, "pages": "25074", "perpage": 7, "total": "250739", "photo": [ { "id": "29012739647", "owner": "159732495@N03", "secret": "39dd96b21d", "server": "1840", "farm": 2, "title": "BombSquad MOD (Unlocked)", }}} How The Command-Line Can Give You Superpowers — @gnufmuffin
  53. Extract list of values $ jq -r '.photos.photo | .[]

    | .title' flickr.json BombSquad MOD (Unlocked) MARVEL Strike Force MOD (Unlimited Energy) New photo added to gallery New photo added to gallery New photo added to gallery Samsung Galaxy Note 9, así encaja en el catálogo … WeTorrent Premium – Torrent Downloader (Cracked) How The Command-Line Can Give You Superpowers — @gnufmuffin
  54. Create array of numeric IDs $ jq '.photos.photo | [.[].id

    | tonumber]' flickr.json [ 29012739647, 43231929164, 43950331011, 43950279251, 43231327754, 43901777582, 42140715770 ] How The Command-Line Can Give You Superpowers — @gnufmuffin
  55. Web How The Command-Line Can Give You Superpowers — @gnufmuffin

  56. cURL — Comprehensive client for working with URLs — "the

    number of features will make your head spin!" — Documented in a new book https://bookcurl.haxx.se/ — Has extensive support of protocols: HTTP, SMTP, IMAP, RTSP, etc. — Get it: built-in How The Command-Line Can Give You Superpowers — @gnufmuffin
  57. Debugging endpoints: --include $ curl 'https://search.icons8.com/api/iconsets/v4/search' --include HTTP/2 200 server:

    keycdn-engine date: Thu, 09 Aug 2018 19:39:52 GMT content-type: application/json; charset=utf-8 content-length: 96 cache-control: max-age=604800 expires: Thu, 16 Aug 2018 19:39:52 GMT x-cache: HIT x-shield: active x-edge-location: usch access-control-allow-origin: * accept-ranges: bytes {"parameters":{"amount":"25","offset":"0","impresser_preview":false, "language":"en"},"icons":[]} How The Command-Line Can Give You Superpowers — @gnufmuffin
  58. Debugging Firebase Cloud Messaging — Typically, app server initiates push

    — Console can only send notification messages — Can simulate server from command-line — More details in the Firebase docs How The Command-Line Can Give You Superpowers — @gnufmuffin
  59. Sending a push notification $ curl \ --header "Authorization: key=AJzx8oIelephanT4gELATOi99elkbffevOon03F"

    \ --header "Content-Type: application/json" \ --data payload.json \ https://fcm.googleapis.com/fcm/send { "name": "projects/myproject-b5ae1/messages/0:1500415314455276%31bd1c9631bd1c96" } — Use legacy authentication to avoid OAuth headaches — Exercise: Wrap in function and parameterize How The Command-Line Can Give You Superpowers — @gnufmuffin
  60. Generating data model from JSON — quicktype https://app.quicktype.io/ generates code

    from JSON — Rapidly integrate with an unfamiliar API — Get sample JSON with cURL, then paste into website How The Command-Line Can Give You Superpowers — @gnufmuffin
  61. How The Command-Line Can Give You Superpowers — @gnufmuffin

  62. Images How The Command-Line Can Give You Superpowers — @gnufmuffin

  63. Image Considerations — Supporting different pixel densities — Converting from

    XML to VectorDrawable — Minimizing APK size How The Command-Line Can Give You Superpowers — @gnufmuffin
  64. ImageMagick — Suite of utilities — Read/write 200 image formats

    — Any kind of transformation you can think of — Best to learn by example — Get it: brew install imagemagick How The Command-Line Can Give You Superpowers — @gnufmuffin
  65. ImageMagick — compare — annotate differences — convert — convert

    file format, apply transformations — identify — describe format and attributes — montage — combine images into another image — Watch out for order of options, punctuation How The Command-Line Can Give You Superpowers — @gnufmuffin
  66. Example: Include before/after in pull requests How The Command-Line Can

    Give You Superpowers — @gnufmuffin
  67. Shell script calling IM #!/bin/bash beforeafter() { montage -background lightgray

    -pointsize 60 -label '%t' $* \ -tile x1 -geometry '+30+30' out.png } — Define beforeafter function — $* is all arguments — %t is IM syntax for each argument's filename in label How The Command-Line Can Give You Superpowers — @gnufmuffin
  68. Calling the script $ beforeafter before.png after.png How The Command-Line

    Can Give You Superpowers — @gnufmuffin
  69. Pull Request How The Command-Line Can Give You Superpowers —

    @gnufmuffin
  70. Example: Crop bitmaps across density sizes How The Command-Line Can

    Give You Superpowers — @gnufmuffin
  71. None
  72. Shell script calling IM's montage $ for r in m

    h xh xxh; do \ convert -gravity SouthWest -crop 100%x40%+0+0 \ android/onboarding/src/main/res/drawable-${r}dpi/background.jpg \ android/onboarding/src/main/res/drawable-${r}dpi/background_out.jpg; \ done — bash for loop to iterate the density prefixes — Using percentage avoids calculating pixel size per density — "Crop 40% of vertical (100% of horizontal) starting from (0,0)" — Visually check the *_out.jpg files before copying over originals How The Command-Line Can Give You Superpowers — @gnufmuffin
  73. Image optimization — Reduce file size — pngcrush https://pmt.sourceforge.io/pngcrush/ —

    svgo https://github.com/svg/svgo — Get it: brew install {pngcrush,svgo} How The Command-Line Can Give You Superpowers — @gnufmuffin
  74. Input SVG How The Command-Line Can Give You Superpowers —

    @gnufmuffin
  75. Apply svgo $ svgo --precision=1 butterfly.svg -o optimized.svg butterfly.svg: Done

    in 221 ms! 115.713 KiB - 19.2% = 93.482 KiB — Much smaller! — But does it look the same? How The Command-Line Can Give You Superpowers — @gnufmuffin
  76. Output SVG How The Command-Line Can Give You Superpowers —

    @gnufmuffin
  77. Comparing images visually with IM's compare $ compare butterfly.svg optimized.svg

    svgo-compare.png $ compare butterfly.png optimized.png png-diff.png How The Command-Line Can Give You Superpowers — @gnufmuffin
  78. Moving Things Around How The Command-Line Can Give You Superpowers

    — @gnufmuffin
  79. Install APKs wirelessly — APKs can be installed over WiFi

    (see Google's adb over WiFi) — Process seems cumbersome — Why? Some devices block USB loading unless you sign up! How The Command-Line Can Give You Superpowers — @gnufmuffin
  80. qr-filetransfer — Starts a local webserver, with randomized URL hosting

    your file — Generates QR code in terminal that points to that URL — Get it: https://github.com/claudiodangelis/qr- filetransfer How The Command-Line Can Give You Superpowers — @gnufmuffin
  81. qr-filetransfer <file> How The Command-Line Can Give You Superpowers —

    @gnufmuffin
  82. Demo How The Command-Line Can Give You Superpowers — @gnufmuffin

  83. Scripting How The Command-Line Can Give You Superpowers — @gnufmuffin

  84. Automating soft keyboard and touches — adb shell input text

    <text> — Sends keycodes to device — adb shell input tap <x> <y> — Sends touch events to device — Combine these into a shell script How The Command-Line Can Give You Superpowers — @gnufmuffin
  85. Shell script to perform a login function adb_login_admin() { for

    field in "$@" do adb shell input text "$field" adb shell input keyevent KEYCODE_ENTER done adb shell input tap 760 50 } — Simple username/password — Inputs each argument then presses Enter to advance — Taps on top-right corner (login button) How The Command-Line Can Give You Superpowers — @gnufmuffin
  86. Usage $ adb_login_admin username password — Caveats: may need to

    insert sleep <seconds> calls — Exercise: Correctly handle screen density on taps How The Command-Line Can Give You Superpowers — @gnufmuffin
  87. Generating Kotlin/Java code — Automate repetitive editing — Turn CSV,

    list, etc. into code — Doesn't have to be 100% syntactically correct How The Command-Line Can Give You Superpowers — @gnufmuffin
  88. Input CSV CSS font name,Font name,Category,Weight,Style,Friendly name,File name AnonymousPro 400,Anonymous

    Pro,Block,400,normal,Anonymous,AnonymousPro-Regular AnonymousPro 700,Anonymous Pro,Block,700,normal,Anonymous Bold,AnonymousPro-Bold Cousine 400,Cousine,Block,400,normal,Cousine,Cousine-Regular Cousine 700,Cousine,Block,700,normal,Cousine Bold,Cousine-Bold … How The Command-Line Can Give You Superpowers — @gnufmuffin
  89. Parse a CSV using Ruby File.open(font_list_filename, 'r') do |file| font_list

    = CSV.new(file, { :headers => true, :return_headers => false }) while row = font_list.shift slug = row['CSS font name'].slugify friendly_name = row['Friendly Name'] font_names[slug] = friendly_name font_css_names[slug] = row['CSS font name'] end end — Build up two hashes, mapping their identifier to their friendly name and CSS name — e.g. { "font_cousine400" => "Cousine 400" } How The Command-Line Can Give You Superpowers — @gnufmuffin
  90. Generate Kotlin enum def android_kotlin_enum(font_names, font_css_names) val = "enum class

    FontName(val cssName: String, @StringRes val resId: Int) {\n" val += font_names.keys.map {|k| " #{k.upcase}(\"#{font_css_names[k]}\", R.string.font_#{k})" }.join(",\n") val += ";\n" val += <<END companion object { fun find(cssName: String): FontName? = values().find {it.cssName == cssName} } } END puts val end How The Command-Line Can Give You Superpowers — @gnufmuffin
  91. Output enum class FontName(val cssName: String, @StringRes val resId: Int)

    { ANONYMOUSPRO400("AnonymousPro 400", R.string.font_anonymousPro400), ANONYMOUSPRO700("AnonymousPro 700", R.string.font_anonymousPro700), COUSINE400("Cousine 400", R.string.font_cousine400), COUSINE700("Cousine 700", R.string.font_cousine700), … companion object { fun find(cssName: String): FontName? = values().find {it.cssName == cssName} } } — Not a complete Kotlin class file (but very close) — But the laborious parts were automated How The Command-Line Can Give You Superpowers — @gnufmuffin
  92. Idea: Automate generation of Google Play Billing CSV — Get

    products from "source of truth", e.g. Firebase — Convert or parse data (jq, script) — Generate CSV How The Command-Line Can Give You Superpowers — @gnufmuffin
  93. Further Exploration How The Command-Line Can Give You Superpowers —

    @gnufmuffin
  94. Further Exploration — Install more packages from Homebrew (4000+) —

    Read the documentation of your terminal and shell — Learn a keyboard-oriented text editor — Learn the basics of a high-level scripting language How The Command-Line Can Give You Superpowers — @gnufmuffin
  95. End — Email efung@acm.org — Blog code.gnufmuffin.com — Code github.com/efung

    — Social @gnufmuffin — Slides speakerdeck.com/efung How The Command-Line Can Give You Superpowers — @gnufmuffin