Script Bash like a Pro: Good Practices and Common Pitfalls

In einer Welt voller Scripting-Tools wie Python, Deno oder Babashka wirkt Bash auf den ersten Blick wie ein verstaubtes Relikt aus vergangenen Zeiten. Doch Bash ist praktisch überall startklar und bleibt der perfekte Klebstoff, um die vielen mächtigen Tools zu verbinden, die längst auf eurem Rechner installiert sind.

In diesem Talk zeigt euch Roman, wie ihr schnelle One-Liner in portable und robuste Skripte verwandelt, die nicht beim ersten Leerzeichen auseinanderfallen. Mit nur wenigen Zeilen Code automatisiert ihr mühselige oder repetitive Aufgaben und schafft euch so Freiraum für die wirklich wichtigen Dinge. Ihr bekommt praxiserprobte Tipps – und lernt, wie ihr eure Skripte debuggt, testet und so baut, dass sie euch Zeit sparen, statt euch in den Fuß zu schießen. Ob Shell-Neuling oder Skript-Veteran: Ein „Ah, so funktioniert das also!"-Moment ist garantiert.

Transcript

2. Modern languages for automation Python, Golang, Deno, Rust ✨ ‣ Linters, type checks, libraries, package managers, tests, etc Then there are also shell scripts " ‣ BASH (Bourne Again Shell) from 1989 is the de-facto standard interpreter Let’s spin up the time machine …
3. Then there is BASH (from 1989 💾) Roman Neß (as old as BASH)
4. ▓▓▓ Basically automates commands issued to your Terminal 1 echo "Hello, world!" # Print text 2 ls # List files 3 pwd # Print current directory 4 whoami # Show current user 5 date # Print the current date and time ———————————————————————————— [finished] ———————————————————————————— Hello, world! presenterm.md presenterm.pdf shebang.sh shebang_pitfall.sh timemachine.gif /Users/ness/customer/techtalks/2025-bash-101/presenterm ness Mon Apr 28 12:02:03 CEST 2025 2 / 21
5. ▓▓▓ No linter. The script must go on... 1 echo "Hello, world!" # Print text 2 ls # List files 3 I'm a cat running over the keyborard§"$%§'%&/ª¯\_😸_/¯ 4 whoami # Show current user 5 date # Print the current date and time ———————————————————————————— [finished] ———————————————————————————— Hello, world! presenterm.md presenterm.pdf shebang.sh shebang_pitfall.sh timemachine.gif /var/folders/z9/xncr2c615514985188cqw9mh0000gp/T/.presentermbl7c2I/scr ipt.sh: line 3: /ª¯_😸_/¯: No such file or directory /var/folders/z9/xncr2c615514985188cqw9mh0000gp/T/.presentermbl7c2I/scr ipt.sh: line 3: Im a cat running over the keyborard§"$%§%: command not found ness Mon Apr 28 12:02:03 CEST 2025 3 / 21
6. ▓▓▓ No type checks Everything is a string (unless it is an array of strings). 1 x="42" # x is a string 2 3 if [[ "$x" -eq "42" ]]; then 4 echo "x is a number and equals 42" 5 fi 6 7 if [[ "$x" == "42" ]]; then 8 echo "x is the string '42'" 9 fi ——————————————————— [finished] ——————————————————— x is a number and equals 42 x is the string '42' 4 / 21
7. ▓▓▓ A lot of weird operators 1 grep "foo" <*.md >/dev/null 2>&1 || exit $? 5 / 21
8. ▓▓▓ No seatbelts 1 dir="build" 2 # one typo can mess everything up 3 echo rm -rf ${dirr}/* # 💣 ——————————————————— [finished] ——————————————————— rm -rf /Applications /Library /System /Users /Volumes /bin /cores /dev /etc /home /opt /private /sbin /tmp /usr /var 6 / 21
9. text="Okay, enough bashing! Let's do a quick Bash syntax refresher." cowsay "${text}" say "${text}" —————————————————————————————— [finished] ——————————————————————————————— ________________________________________ / Okay, enough bashing! Let's do a quick \ \ Bash syntax refresher. / ---------------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || || 7 / 21
10. ▓▓▓ Variables There are only strings and arrays of strings. 1 string="Techtalk" 2 echo "Hello ${string}. unset=${unset}" 3 4 arr=("a" "b" "c") 5 echo "The first element of: ${arr[@]} is ${arr[0]}" ——————————————————————— [finished] ——————————————————————— Hello Techtalk. unset= The first element of: a b c is a 8 / 21
11. ▓▓▓ Command substitution $(cmd) and `cmd` execute in a sub-shell. 1 hostname="$(hostname)" 2 echo "Running on hostname: '${hostname}'" 3 echo "in directory: $(basename "$(pwd)")" 4 echo "at date: `date`" ——————————————————— [finished] ——————————————————— Running on hostname: 'm1' in directory: presenterm at date: Mon Apr 28 12:02:09 CEST 2025 9 / 21
12. ▓▓▓ Arrays & Loops It's possible to iterate over array elements or words in a string. 1 arr=("one" "two" "three") 1 string="one two three" 2 2 3 for elem in "${arr[@]}"; do 3 for word in ${string}; do 4 echo "$elem" 4 echo "$word" 5 done 5 done ————————————— [finished] ————————————— ————————————— [finished] ————————————— one one two two three three 10 / 21
13. ▓▓▓ stdin, stdout, stderr, pipestreams Pipestreams feed the stdout as stdin of the next command. 1 echo "redirect stdout" > 1 ls does-not-exist 2> /tmp/stdout.log /tmp/stderr.log 2 echo "append stdout" >> 2 # feed stdin from file /tmp/stdout.log 3 grep "exist" < /tmp/stderr.log 3 # pipe stdout to other commands 4 cat /tmp/stdout.log | grep "std" | sort ————————————— [finished] ————————————— ls: does-not-exist: No such file or ————————————— [finished] ————————————— directory append stdout redirect stdout 11 / 21
14. ▓▓▓ Exit Codes exit 0 -> true all other codes -> false 1 false # builtin that returns non-zero 2 if [[ "$?" -ne 0 ]]; then 3 echo "previous command failed 🔴" 4 fi 5 6 false || echo "handle failure of previous cmd 🩹" 7 true && echo "previous command was successful 🟢" 8 9 false # exit code of last command is exit code of script ———————————————————— [finished with error] ———————————————————— previous command failed 🔴 handle failure of previous cmd 🩹 previous command was successful 🟢 12 / 21
15. ▓▓▓ Functions Functions are simply named code blocks that set parameters as $1,$2,... Parameters are not validated. 1 function add() { 2 echo $(($1 + $2)) 3 } 4 5 add 3 4 6 echo "'add 3 4' returned with code: $?" 7 8 add 3 foo bar 9 echo "'add 3 foo bar' returned with code: $?" ———————————————————— [finished] ———————————————————— 7 'add 3 4' returned with code: 0 3 'add 3 foo bar' returned with code: 0 13 / 21
16. ▓▓▓ Run processes in the background 1 function sleep_and_log() { 2 sleep 1 3 echo "$1 finished at $(date)." 4 say "done." 5 } 6 7 echo "started at $(date)" 8 # command ended with '&' run in background 9 sleep_and_log 1 & # forks a sub process 10 sleep_and_log 2 & 11 sleep_and_log 3 & 12 wait # wait for all background processes 13 echo "all done at $(date)" ——————————————————— [finished] ——————————————————— started at Mon Apr 28 12:02:10 CEST 2025 3 finished at Mon Apr 28 12:02:11 CEST 2025. 1 finished at Mon Apr 28 12:02:11 CEST 2025. 2 finished at Mon Apr 28 12:02:11 CEST 2025. all done at Mon Apr 28 12:02:12 CEST 2025 14 / 21
17. ▓▓▓ Traps 1 # "event listener" on EXIT event 2 trap 'echo "👋 Exit ($?) on: $BASH_COMMAND" >&2' EXIT 3 4 echo "hello" 5 exit 42 —————————————————— [finished with error] ——————————————————— hello 👋 Exit (42) on: exit 42 15 / 21
18. ▓▓▓ shebang sets the interpreter #!/bin/bash #!/usr/bin/env bash # 👆 uses old version on Mac # ✅ uses first in PATH echo "${BASH_VERSION}" echo "${BASH_VERSION}" 1 # obey shebang 1 ./shebang.sh 2 ./shebang_pitfall.sh 3 # ignore shebang 4 bash ./shebang_pitfall.sh ————————————— [finished] ————————————— 5.2.26(1)-release ————————————— [finished] ————————————— 3.2.57(1)-release 5.2.26(1)-release 16 / 21
19. ▓▓▓ safe exit on errors 💡 Use set -o errexit or set -e. 1 # ⚠ error is ignored 1 set -o errexit 2 cd missing_dir 2 3 echo rm * 💣 3 cd missing_dir # 👋 exit here 4 echo rm * ————————————— [finished] ————————————— ——————— [finished with error] ———————— /var/folders/z9/xncr2c615514985188cqw9 mh0000gp/T/.presentermSOl5lo/script.sh /var/folders/z9/xncr2c615514985188cqw9 : line 2: cd: missing_dir: No such mh0000gp/T/.presentermqCUBSR/script.sh file or directory : line 3: cd: missing_dir: No such rm presenterm.md presenterm.pdf file or directory shebang.sh shebang_pitfall.sh timemachine.gif 💣 17 / 21
20. ▓▓▓ safe exit on unset variables 💡 Use set -o nounset or set -u. 1 dir="build" 1 # ✅ exit on unset variable 2 echo rm -rf "${dirr}/*" # ⚠ typo 2 set -o nounset 3 4 dir="build" ————————————— [finished] ————————————— 5 echo rm -rf "${dirr}/*" # 👋 rm -rf /* ——————— [finished with error] ———————— /var/folders/z9/xncr2c615514985188cqw9 mh0000gp/T/.presentermD6uMIV/script.sh : line 5: dirr: unbound variable 18 / 21
21. ▓▓▓ exit on error in pipestream 💡 Use set -o pipefail. 1 # ⚠ error is ignored 1 set -e 2 cat missingfile.txt | wc -l 2 set -o pipefail # ✅ treat error 3 echo "🟢 file processed" in pipeline as error 3 4 cat missingfile.txt | wc -l # 👋 ————————————— [finished] ————————————— 5 echo "🟢 file processed" cat: missingfile.txt: No such file or directory ——————— [finished with error] ———————— 0 🟢 file processed cat: missingfile.txt: No such file or directory 0 19 / 21
22. cowsay "Let's go back to the future!" ——————————————————— [finished] ——————————————————— ______________________________ < Let's go back to the future! > ------------------------------ \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || || 20 / 21
23. Should I scripts on a shell in 2025? ‣ Do you use a terminal? ‣ Do you work on Dockerfiles? ‣ Do you work on CICD pipelines? ‣ Do you work on package.json files? ‣ What happens if you press run / debug in your IDE? By writing shell scripts you gain better understanding of all of the above _____________________________ < You probably already do it! > ----------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || ||
24. BASH is the duct tape of DevOps ‣ All of the mighty CLI tools interact with the host system via a well defined interface • stdin, stdout, stderr, exit codes, interrupt signals ‣ BASH can orchestrate any cli tool and use the interfaces ./myapp <(file.txt) | ./my-script.sh Redirect stdin ./myapp >/stdout.txt 2>stderr.txt Redirect stdout, stderr ./wait-for-sidecar.sh && ./myapp Delay app startup until the sidecar is online alpine-debug-image.sh bash jq curl Create an image with tools for debugging
25. BASH use cases ‣ Automate repetitive tasks (setups, builds, deployments) ‣ Build tools on top of existing CLI tools (aws-cli, docker-cli, jq, yq, sed) ‣ Quickfix production problems (handle exit codes, delay startup) ‣ Speed up the cycle time on frustrating problems ‣ Create quick pipelines to analyse or modify data (with pipestreams) ➡ Solve typical dev and ops problems in minutes
26. BASH is always* ready ‣ BASH is the de-facto standard interpreter for shell scripting • Already available on your machine (Linux, MacOS, WSL) • Already available in your container (except for busybox) • Using another shell (zsh, fish) and scripting in BASH works well in practice Minor Pitfalls • MacOS is shipped with a BASH version from 2007 due to licensing issues • alpine containers o!en have to install BASH from package manager
27. Glossary for the remaining slides ⚠ Common Pitfall that should be avoided $ Good Practice == A practice we did good experience with ✅ Robust/preferred syntax & Command in this line exits with error
28. $Conditions ‣ test … and [ … ] are the same utility [[ … ]] is Bash syntax with additional features a && b || c is not equal to if-then-else it also executes c if b fails a="foo" b="bar" with_spaces="with spaces" # boolean operators within brackets [[ "$#" == 0 && "$a" != "$b" ]] # can handle word splitting [[ $with_spaces == "with spaces" ]] # regex comparison [[ "$with_spaces" =~ spaces$ ]] true && { echo true; false; } || echo false
29. $Using external commands and tools Check that required tools are available at script start For some commands there are incompatible implementations • https://github.com/mikefarah/yq vs https://github.com/kislyuk/yq • GNU sed (most linux distributions) vs BSD sed (MacOS) if ! command -v aws >/dev/null; then echo "aws-cli not found." exit 1 fi
30. $Logs In longer scripts create log functions Logging to stderr is a good idea if you use stdout for pipestreams succeed silently & fail loudly Use emojis in logs ('&()) printf is more robust than echo, but echo commands are so much faster to type function log_err() { echo -e "[ERR]: $*" >&2 } function log_debug() { [[ "${DEBUG}" = "true" ]] \ && echo -e "[DEBUG]: $*" >&2 }
31. $Command options Use long version of command options unless they are well known Use arrays to configure command options # ❓what does this do? docker ps -a -l -q # ✅ easier to understand docker ps --all --latest --quiet # ✅ arrays are great for command options ls_flags=(-l --color=auto) ls_flags+=(-a) ls "${ls_flags[@]}"
32. $Script Arguments If your scripts accept arguments you should validate them and print correct usage on error Use getopts if you want to parse flags Consider env vars with defaults as an alternative to flags (these can be set by direnv, CICD) usage() { cat << EOF Usage: $0 <STRING>... joins all provided strings with '-' EOF exit 1 } # validate arguments if [[ "$#" == 0 ]]; then usage fi
33. $Functions omit the function keyword for maximum portability (sh, ash) all arguments as array with “$@“ Declare variables in function with local to not expose them globally use return instead of exit to exit a function, unless you want to exit the entire script logAndRun() { echo " # " "$@" if [[ "${dryRun}" = false ]]; then "$@" else echo "+ dry run" fi }
34. $Ask for confirmation avoid accidental execution of critical commands function confirmAndRun() { echo " # " "$@" >&2 read -r -p "(y) run, (s) skip, (n) cancel: " yn </dev/tty >&2 case ${yn} in [yY]* ) "$@" ;; s) echo ", skipped." >&2;; *) echo "- exiting ..." >&2; exit 1 ;; esac } confirmAndRun date confirmAndRun pwd confirmAndRun hostname confirmAndRun uname $ bash ./toolbox/confirm-and-run.sh # date (y) run, (s) skip, (n) cancel: y Thu Apr 24 12:46:15 CEST 2025
35. Testing with BATS ‣ Bash Automated Testing System https://github.com/bats-core/bats-core ‣ Implemented in Bash ‣ Helper libraries for assertions • bats-assert, bats-file ‣ setup() and teardown() hooks. ‣ Can test everything you can execute in Bash hello() { echo "Hello $1!" } @test "hello returns correct output" { run hello "Techtalk" [ "$status" -eq 0 ] [ "$output" = "Hello Techtalk!" ] } @test "intentional failure" { run hello "Techtalk" [ "$status" -eq 0 ] [ "$output" = "Hello World!" ] }
36. How to debug shell scripts ‣ set -x; commands;to;debug; set +x ⇨ trace commands ‣ printf / echo commands (to stderr) ‣ Hack your own library showing errors / exits with traps ‣ Check for Shellcheck warnings / errors ‣ bashdb Debugger
37. Debugging scripts with bashdb ‣ Plugins for JetBrains IDEs (paid) and VS Code ‣ Support for: • Breakpoints • Watch variables and commands • Call stack ‣ Implemented mostly in Bash (with traps) . https://github.com/Trepan-Debuggers/bashdb
38. ⚠ Variables & Assignments Use double quotes on every string to avoid word splitting Use single quotes to suppress expansion Using braces is a good practice string="foo" # ✅ no spaces around equal sign echo string=$string # string=foo echo $string_bar ${string}_bar # ⚠ no expansion w/o braces i= hostname # ⚠ i="" hostname ls =bar # ⚠ ls "=bar" i=foo echo oops # ⚠ i="foo" echo oops i=* # ⚠ globbing expands `*` to all objects in pwd filename="a filename with spaces.txt" # ✅ quote string w/ spaces echo "filename=${filename}" # ✅ quote every string ls ${filename} # ⚠ ls "a" "filename" "with" "spaces.txt echo '${filename}' # ⚠ no expansion in single quotes
39. ⚠ Word Splitting ‣ Words split on spaces, newlines & tabs ‣ Quotes suppress word splitting Quote every string to avoid unintended word splitting Don’t set IFS (internal field separator) unless you know what you’re doing show_words a b 'c d' 'e'"f" # 4 words: <a> <b> <c d> <ef> arr=(a b 'c d' 'e'"f") show_words "${arr[@]}" # array preserves words # 4 words: <a> <b> <c d> <ef> show_words "${arr[*]}" # array to string # 1 words: <a b c d ef> string="a b c" show_words $string # unintended word split? # 3 words: <a> <b> <c> show_words "$string" # 1 words: <a b c>
40. ⚠ set -o errexit Safe exit with set -e is disabled when the expression is part of a boolean condition If a function is part of a boolean condition it must handle errors itself set -e function fail_on_cd() { cd "missing_$1" # & error here echo rm -f "$1" } # ⚠ set -e is disabled in if-condition if fail_on_cd 1; then true; fi # ⚠ set -e is disabled for LHS of `||` and `&&` fail_on_cd 2 && true # ⚠ set -e is disabled for negations with `!` ! fail_on_cd 3 fail_on_cd 4 # ✅ will exit(1) here echo "not reached"
41. ⚠ Command Substitutions Command substitutions run in a subshell and disable set -e and set -o pipefail shopt -s inherit_errexit or $(set -e; cmd) to inherit set -e in subshells Command substitutions in arguments mask return type set -e function fail_on_cd() { cd "missing_$1" # & error here echo rm -f "$1" >&2 } # ⚠ set -e not inherited in subshell a="$(fail_on_cd 4)"; echo "$a" # ⚠ return type is masked, bc cmd is echo “…” echo "$(set -e; fail_on_cd 5)" b="$(set -e; fail_on_cd 6)" # - exit(1) echo "${b} bar"
42. How to avoid pitfalls in Bash scripting? Use ShellCheck (in your IDE, in pre-commit, in CICD) Avoid lengthy “in-line” scripts (CICD, entrypoints) Consult manuals • Of utilities (e.g. man test, docker ps —help) • Of bash syntax (e.g. help if) Create templates for new scripts #!/usr/bin/env bash #shellcheck enable=check-extra-masked-returns #shellcheck enable=check-set-e-suppressed set -euo pipefail shopt -s inherit_errexit #!/usr/bin/env bash set -euo pipefail
43. Learnings ‣ You can probably solve almost every problem with a Bash script ‣ But at some point you should use a more structured language ‣ By using your shell and scripting in it you gain a lot of DevOps knowledge ‣ Speed up cycle times, automate distracting tasks and increase overall quality ‣ You can build your own toolbox with Bash and share it with any other dev
44. ▓▓▓ That's all folks figlet THANKS qrencode -m 2 -t utf8 <<< " https://github.com/RomanNess/techt alk-bash-2025" ————————————— [finished] ————————————— _____ _ _ _ _ _ _ ______ ————————————— [finished] ————————————— |_ _| | | | / \ | \ | | |/ / ___| | | | |_| | / _ \ | \| | ' /\___ \ █████████████████████████████████ | | | _ |/ ___ \| |\ | . \ ___) | ██ ▄▄▄▄▄ █▀█ █▄ █▀▄█ ▄ █ ▄▄▄▄▄ ██ |_| |_| |_/_/ \_\_| \_|_|\_\____/ ██ █ █ █▀▀▀█ ▀▄▄▄▀█▀▄█ █ █ ██ ██ █▄▄▄█ █▀ █▀▀▄▀▀ ▄▄ █ █▄▄▄█ ██ ██▄▄▄▄▄▄▄█▄▀ ▀▄█ ▀▄█ ▀ █▄▄▄▄▄▄▄██ ██ █▄ ▄▀▄ ▄▄▄▀▀▀ ▀▄▀ ▀▄█▄▀██ ███ ▄█ ▄▄▄██▄▀ ▄█ ▀▄▀▀████▄▀█▀███ ███ ▄█▀▄▄▄█ ▄▄ ▄▄▄ ▄▀█ ▀▀▀▀▄▄█▀██ ██ ▄▄ ██▄ ██▄███▀ ███▀ ▀▀ ▄▄▀███ ██ ▀ ▄▄ ▄▀█ ▄▀▀▄ █▀▄█ █ ▀ ▀▄ █▀██ ██ █▄▄█▀▄▀█▄▀▀ ▄▀▄▄▄▄█ ▀ ▄▄█▄▀███ ██▄██▄█▄▄█ ▄▀▀▀ ▄▄▀▄█▄ ▄▄▄ ▀ ██ ██ ▄▄▄▄▄ █▄▀██ ██ ▄▄ █▄█ ▄▄▀███ ██ █ █ █ ▀ ▀▀ █▄▀ ▀ ▄▄▄▄▀ ▄▀██ ██ █▄▄▄█ █ ▀█▀ █ ▀▄█▄ ▄ ▄ ▄ ███ 21 / 21 ██▄▄▄▄▄▄▄█▄█▄██▄█▄▄▄█▄██▄▄▄█▄████