Tweet
Tweet

Slide 1

Slide 1 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) NOVEMBER 12, 2020 Rob Hirschfeld, RackN Boost your JQ IQ

Slide 2

Slide 2 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) Why JQ? Not because it’s elegant, fun or easy. JK! While APIs are JSON native, people aren’t. ● Pretty Print ● Validate JSON ● Parse JSON ● Search JSON ● Build JSON

Slide 3

Slide 3 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) A LOT! It’s so critical, we embedded it into our golang CLI so we didn’t have to install it everywhere. If you cp & suffix “drpcli” with “jq” then it becomes a JQ parser! How much does RackN JQ?

Slide 4

Slide 4 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) Distributed Infrastructure as Code with API-driven Provisioning All our API models and events are JSON All our states are JSON All our IaC templates are JSON (ok, we source them in YAML because readability) What does RackN do?

Slide 5

Slide 5 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) The Basics: How does it work?

Slide 6

Slide 6 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) Two Reference JSON files one.json machine object { “Name”: “x”, “Uuid”: “123-XYZ”, “Params: { “foo”: “bar” } } all.json list of machine objects [ { “Name”: “test1” … }, { “Name”: “test2” … }, { “Name”: “test3” … } ]

Slide 7

Slide 7 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) cat one.json | jq . cat one.json | jq .Name cat all.json | jq .[].Name cat all.json | jq .[0].Name cat all.json | jq .[2:4] JSON dot navigation

Slide 8

Slide 8 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) cat one.json | jq .Params.foo cat one.json | jq .Params.gohai-inventory nope!… be careful cat one.json | jq .Params.[gohai-inventory] nope!… be more careful cat one.json | jq ‘.Params.[“gohai-inventory”]’ cat one.json | jq .Params.\“gohai-inventory\” JSON dot navigation

Slide 9

Slide 9 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) JSON math cat all.json | jq length cat all.json | jq keys cat one.json | jq keys cat all.json | jq .[0] cat all.json | jq first cat all.json | jq last.Name

Slide 10

Slide 10 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) JSON pipes cat one.json | jq “.Params | length” cat one.json | jq “.Params | keys” cat one.json | jq “.Params | keys | length” cat one.json | jq “.Params[‘gohai-inventory’] | keys” nope… yikes quoting! cat one.json | jq '.Params["gohai-inventory"] | keys' cat one.json | jq “.Params[\"gohai-inventory\"] | keys”

Slide 11

Slide 11 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) Why do I care about quotes? export gohai=$(cat one.json | jq -r ‘.Params | keys | last’) echo $gohai cat one.json | jq '.Params["$gohai"]' nope! cat one.json | jq “.Params[\"$gohai\"] | keys” yes!

Slide 12

Slide 12 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) JSON finding cat all.json | jq ‘.[] | select(.Name==”test1”)’ Can use lots of logic! Like has, in, >, etc but, be careful of unexpected output! cat all.json | \ jq '.[] | select(.Name | startswith("test"))'

Slide 13

Slide 13 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) Building JSON maps (arrays) this is set operations…. be careful cat all.json | \ jq 'map(.Name | startswith("test"))' not what you think! cat all.json | \ jq 'map(select(.Name | startswith("test")))'

Slide 14

Slide 14 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) JSON flags cat one.json | jq .Endpoint fix that with -n cat one.json | jq -n .Endpoint cat one.json | jq -r .Name handy for bash loops: cat all.json | jq -r '.[].Uuid'

Slide 15

Slide 15 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) RackN Tips & Tricks

Slide 16

Slide 16 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) Bash Loop Examples all=$(cat all.json | jq -r '.[].Uuid') for uuid in $all; do echo “ID: $uuid” cat all.json \ | jq ".[] | select (.Uuid == \"$uuid\") | .Name" done

Slide 17

Slide 17 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) Bleh, Quoting! Help! --arg can be handy to remove \” quotes export name=”test1” cat all.json | \ jq --arg n "$name" \ ‘.[] | select(.Name == $n) | .Uuid’

Slide 18

Slide 18 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) Pipe from variables export all=$(cat all.json) jq .[].Name <<< "$all" We mix in Golang Templates export params=”{{ .Machine.Params }}” jq -r .[“gohai-inventory”] <<< "$params"

Slide 19

Slide 19 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) Building JSON (better than by hand!!) export base='{"foo": "bar"}' adds jq ‘. += {“rob”:”zehicle”}’ <<< $base removes jq 'del(."foo")' <<< "$base"

Slide 20

Slide 20 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) ...And Don’t Go Crazy

Slide 21

Slide 21 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) ...And Don’t Go Crazy In many cases, THE BEST JQ IS NO JQ

Slide 22

Slide 22 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) Thank you! Resources: Online Dev Tooling: https://jqplay.org Reference: https://stedolan.github.com/jq This is stable stuff! Last major release was 2018

Slide 23

Slide 23 text

TRACK: SITE RELIABILITY ENGINEERING SPEAKER: Rob Hirschfeld (@zehicle) THANK YOU TO OUR SPONSORS