How we tamed bash, a mostly, true story

How we as Ruby developers went about building and maintaining 40+ bash scripts with over 3000 unique lines of code.

Video courtesy of engineers.sg: https://www.youtube.com/watch?v=ytU67cLSNRw

This is the Zen of Python that was mentioned:

```
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
```

Transcript

1. How we tamed bash, a mostly, true story Björn Andersson
2. @gaqzi Björn 
 code monkey, devops and
 loud mouth pythonista https://flic.kr/p/b9jGA
3. @gaqzi what is your code base like?
4. @gaqzi do you test…?
5. @gaqzi
6. @gaqzi unit?
7. @gaqzi unit? manual?
8. @gaqzi unit? manual? integration?
9. @gaqzi unit? manual? integration? infrastructure?
10. @gaqzi Scenario
11. @gaqzi Scenario • Big project
12. @gaqzi Scenario • Big project • File transfers
13. @gaqzi Scenario • Big project • File transfers • Integration points managed by different teams
14. @gaqzi Scenario • Big project • File transfers • Integration points managed by different teams • Our main logic is being run on servers we have no influence over
15. @gaqzi Scenario • Big project • File transfers • Integration points managed by different teams • Our main logic is being run on servers we have no influence over • bash and the GNU Coreutils ended up being the only available option
16. @gaqzi I was begging for Perl
17. @gaqzi I was begging for Perl
18. @gaqzi Hm, now then?
19. @gaqzi Hm, now then? bash has… an aged syntax with arcane invocations
20. @gaqzi Hm, now then? bash has… well we ought to do TDD, what are the options?
21. @gaqzi https://github.com/mlafeldt/sharness “Sharness is a portable shell library to write, run, and analyze automated tests for Unix programs.”
22. @gaqzi #!/bin/sh test_description="Show basic features of Sharness" . ./sharness.sh test_expect_success "Success is reported like this" " echo hello world | grep hello " test_expect_success "Commands are chained this way" " test x = 'x' && test 2 -gt 1 && echo success " return_42() { echo "Will return soon" return 42 } test_expect_success "You can test for a specific exit code" " test_expect_code 42 return_42 " test_expect_failure "We expect this to fail" " test 1 = 2 " test_done
23. @gaqzi
24. @gaqzi
25. @gaqzi Okay, then what?
26. @gaqzi Okay, then what? We know the initial state
27. @gaqzi Okay, then what? We know the initial state We know the expected state
28. @gaqzi that’s integration testing, that is!
29. @gaqzi that’s integration testing, that is!
30. @gaqzi
31. @gaqzi The toolchain
32. @gaqzi The toolchain m4
33. @gaqzi The toolchain m4 make
34. @gaqzi The toolchain m4 make GNU Coreutils
35. @gaqzi The toolchain m4 make GNU Coreutils bash
36. @gaqzi The toolchain m4 make GNU Coreutils bash cucumber
37. @gaqzi The toolchain m4 make GNU Coreutils bash cucumber ssh
38. @gaqzi Some stats
39. @gaqzi Some stats • 616 lines of our own shared library (encryption, transfer, packaging, verification, cleaning etc)
40. @gaqzi Some stats • 616 lines of verification, cleaning etc) • 2226 unique lines in scripts, split over ~40 scripts
41. @gaqzi Some stats • 616 lines of verification, cleaning etc) • 2226 unique lines in scripts, split over ~40 scripts • The biggest pipeline contains 15 script runs, spanning multiple data centers and nine different servers
42. @gaqzi Some stats • 616 lines of verification, cleaning etc) • 2226 unique lines in scripts, split over ~40 scripts • The biggest pipeline contains 15 script runs, spanning multiple data centers and nine different servers • The compiled scripts are a total of 16 504 lines
43. @gaqzi Some stats • 616 lines of verification, cleaning etc) • 2226 unique lines in scripts, split over ~40 scripts • The biggest pipeline contains 15 script runs, spanning multiple data centers and nine different servers • The compiled scripts are a total of 16 504 lines • The largest compiled script is 720 lines (142 lines uncompiled)
44. @gaqzi Some stats • 616 lines of verification, cleaning etc) • 2226 unique lines in scripts, split over ~40 scripts • The biggest pipeline contains 15 script runs, spanning multiple data centers and nine different servers • The compiled scripts are a total of 16 504 lines • The largest compiled script is 720 lines (142 lines uncompiled) • 2400 lines of cucumber features, 299 scenarios over a total of 2052 steps
45. @gaqzi Bash that survives you?
46. @gaqzi Bash that survives you? Not wanting your fancy new code ending up being the equivalent of autohell
47. @gaqzi Bash that survives you? Not wanting your fancy new code ending up being the equivalent of autohell Very few people have experience with writing more than a couple of lines of bash
48. @gaqzi Bash that survives you? Not wanting your fancy new code ending up being the equivalent of autohell Very few people have experience with writing more than a couple of lines of bash Keep it easy, clean and in chunks that make sense
49. @gaqzi Bash that survives you? Not wanting your fancy new code ending up being the equivalent of autohell Very few people have experience with writing more than a couple of lines of bash Keep it easy, clean and in chunks that make sense Functions, they’re there
50. @gaqzi Bash that survives you? Not wanting your fancy new code ending up being the equivalent of autohell Very few people have experience with writing more than a couple of lines of bash Keep it easy, clean and in chunks that make sense Functions, they’re there Eschew the shortcuts
51. @gaqzi Bash that survives you? Not wanting your fancy new code ending up being the equivalent of autohell Very few people have experience with writing more than a couple of lines of bash Keep it easy, clean and in chunks that make sense Functions, they’re there Eschew the shortcuts Try to keep it beautiful
52. @gaqzi
53. @gaqzi “Beautiful is better than ugly.
54. @gaqzi “Beautiful is better than ugly. Explicit is better than implicit.
55. @gaqzi “Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex.
56. @gaqzi –Tim Peters, The Zen of Python “Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated.”
57. @gaqzi What does it look like? fetch-files-from $REMOTE_SFTP_IN_SERVER $REMOTE_FTP_OUT_PATH $LOCAL_IN_PATH if-error $? 1 "An error occurred trying to download files from \ ’$REMOTE_SFTP_SERVER:$REMOTE_FTP_OUT_PATH' => '$LOCAL_IN_PATH'" any-files-downloaded-to $LOCAL_IN_PATH if-error $? 2 "No files exists in '$LOCAL_IN_PATH'" add-checksum-to-files-in $LOCAL_IN_PATH if-error $? 3 "Error creating checksum for new files" push-files-to $REMOTE_SFTP_OUT_SERVER $REMOTE_FTP_IN_PATH $LOCAL_OUT_PATH if-error $? 4 "Error sending files to remote server" if [ "$cleanup" = 'true' ] ; then do-cleanup if-error $? 5 "Error cleaning up temporary files" fi
58. @gaqzi if-error $? 3 "Error creating checksum"
59. @gaqzi if-error $? 3 "Error creating checksum" function
60. @gaqzi if-error $? 3 "Error creating checksum" function exit status
61. @gaqzi if-error $? 3 "Error creating checksum" function exit status error code
62. @gaqzi if-error $? 3 "Error creating checksum" function exit status error code message
63. @gaqzi What is the success status code?
64. @gaqzi What is the success status code? 0 (zero), it means “no errors”
65. @gaqzi Feature: File transfer from third party Background: Given I am logged in as "ftpinternal" And I have a clean "third party" environment Scenario: A successful transfer from DMZ to Internal without cleanup Given that "the test file" is available in "/ftp/third-party/project_1234/daily/out" When I execute "dmz-fetch-files-from-third-party.sh nocleanup" successfully Then the file "the test file" in "/internal/ftp/third-party/project_1234/daily/in" is valid with | ff961dc5e8da688fa78540651160b223 | And there is no logfiles in "/dmz/ftp/third-party/project_1234/archive/log/in"
66. @gaqzi Test it where you deploy
67. @gaqzi Test it where you deploy And I’m not talking about testing in production, even if that’s what real hackers do
68. @gaqzi Test it where you deploy And I’m not talking about testing in production, even if that’s what real hackers do If you’re deploying on GNU/Linux, then always test there
69. @gaqzi Test it where you deploy And I’m not talking about testing in production, even if that’s what real hackers do If you’re deploying on GNU/Linux, then always test there Did you know OSX mostly uses the BSD toolchain?
70. @gaqzi Test it where you deploy And I’m not talking about testing in production, even if that’s what real hackers do If you’re deploying on GNU/Linux, then always test there Did you know OSX mostly uses the BSD toolchain? Vagrant is wonderful, Ansible or similar also helps a lot with setting up the test machine
71. @gaqzi Test it where you deploy And I’m not talking about testing in production, even if that’s what real hackers do If you’re deploying on GNU/Linux, then always test there Did you know OSX mostly uses the BSD toolchain? Vagrant is wonderful, Ansible or similar also helps a lot with setting up the test machine To run all scenarios it takes about 10 minutes
72. @gaqzi What did I learn?
73. @gaqzi What did I learn? I can feel confident about a big bash script, even when needing it to change
74. @gaqzi What did I learn? I can feel confident about a big bash script, even when needing it to change Your code will be more readable if it’s less idiomatic, unless everyone is really comfortable with bash then use the overly expressive version
75. @gaqzi What did I learn? I can feel confident about a big bash script, even when needing it to change Your code will be more readable if it’s less idiomatic, unless everyone is really comfortable with bash then use the overly expressive version set -x will echo every command being performed, wonderful when enabled in production
76. @gaqzi What did I learn? I can feel confident about a big bash script, even when needing it to change Your code will be more readable if it’s less idiomatic, unless everyone is really comfortable with bash then use the overly expressive version set -x enabled in production set -u explode when using an unassigned variable, if you didn’t assign then it likely is a typo
77. @gaqzi What did I learn?
78. @gaqzi set -e exits on any error, seems like a great idea but doesn’t allow you to set your own error messages What did I learn?
79. @gaqzi set -e you to set your own error messages : ${SECRET_PASSWORD:=default-pass} is the weirdest syntax ever for default/overridable variables, but it’s worth using What did I learn?
80. @gaqzi set -e you to set your own error messages : ${SECRET_PASSWORD:=default-pass} ever for default/overridable variables, but it’s worth using Keeping a separate configuration file makes it easier to test What did I learn?
81. @gaqzi set -e you to set your own error messages : ${SECRET_PASSWORD:=default-pass} ever for default/overridable variables, but it’s worth using Keeping a separate configuration file makes it easier to test A lot of us devs don’t actually know our small Unix tools very well, look up GNU Coreutils and some basic bash. It’ll save you time! What did I learn?
82. @gaqzi set -e you to set your own error messages : ${SECRET_PASSWORD:=default-pass} ever for default/overridable variables, but it’s worth using Keeping a separate configuration file makes it easier to test A lot of us devs don’t actually know our small Unix tools very well, look up GNU Coreutils and some basic bash. It’ll save you time! It was actually fun doing this ridiculous thing What did I learn?
83. @gaqzi questions, comments?
84. @gaqzi thanks @gaqzi