Bash like a marshall

Bash is a programming language design for interact with user. Which have a very flexible syntax to take command from user. But in the bottom, it's just a programming language like any others, it provides variable, function and flow control..., we are going to reintroduce bash syntax and tools, some of you might already know, but is still good to review. In the end of this talk, everyone can write bash script like a marshall. (or at least understand Marshall's)

Transcript

1. BASH LIKE A MARSHALL HOW TO WRITE ADVANCE BASH SCRIPT
2. What is bash? A Programming Language designed for interact with user Load the interpreter when terminal start under unix Flexible and untyped
3. Bourne Again Shell
4. Variable In bash, we can define variable by a=hello and use by echo $a => hello
5. Variable Space will separate parameter a=hello world will be interpret to a=hello, world should use quotation mark a=“hello world”
6. Bash function
7. Variable use variable in string echo “say $a” echo “say ${a}!!!”
8. Variable variable does not have type, so everything can be command a=echo $a $a => echo $a $a $a => echo echo $a $($a $a) => echo
10. Variable check global variable by: export env arithmetic expression: expr 1 + 2 $(( 1 + 2 ))
11. Variable pattern matching ${e#he} => llo ${e/hello/world} => world heredoc a=<<STR … STR
12. Variable PS1: Prompt PS2: Next line PS3: Read Prompt echo $PS1 \h:\W $(parse_git_branch)$(git config user.initials)
13. Function function hello() { } or hello () { }
14. Function $@ => all params $1 => first params $# => number of parameter $? => last function return value
15. Function hello () { return 0 # only return 1/0 } hello () { echo “hello” # return hello }
16. Function require other source file by: source <filename> declare local variable (function scope) by: local a=“hello”
17. Pipeline Pipeline can redirect an output (file descriptor) to another file descriptor echo ‘hello world’ > hello.txt can also redirect error message hello 2> error.txt can redirect thing into blackhole hello &> /dev/null
18. Pipeline - File descriptor ˻ʒʒʒʒʒʒʒʒʒʒ˼ Keyboard ╾──╼┥0 bash STDOUT 1ʯ╾ʒʾʒ╼ Display ! STDERR 2ʯ╾ʒʪ ˾ʒʒʒʒʒʒʒʒʒʒ˽
19. Pipeline - File descriptor ˻ʒʒʒʒʒʒʒʒʒʒ˼ Keyboard ╾──╼┥0 bash STDOUT 1ʯ╾ʒʒʒ╼ Display ! STDERR 2ʯ╾ʒʒʒ╼ error.txt ˾ʒʒʒʒʒʒʒʒʒʒ˽ echo “hello” 2> error.txt
20. Pipeline - File descriptor ˻ʒʒʒʒʒʒʒʒʒʒ˼ Keyboard ╾──╼┥0 bash STDOUT 1ʯ╾ʒʒʒ╼ hello.txt ! STDERR 2ʯ╾ʒʒʒ╼ error.txt ˾ʒʒʒʒʒʒʒʒʒʒ˽ echo “hello” 2> error.txt 1> hello.txt
21. Pipeline - File descriptor echo “hello” 1>&2 2> hello.txt
22. Pipeline - File descriptor ˻ʒʒʒʒʒʒʒʒʒʒ˼ Keyboard ╾──╼┥0 bash STDOUT 1ʯ╾ʒʒʒ╼ display ! STDERR 2ʯ╾ʒʒʒ╼ hello.txt ˾ʒʒʒʒʒʒʒʒʒʒ˽ echo “hello” 1>&2 2> hello.txt
23. Pipeline - File descriptor ˻ʒʒʒʒʒʒʒʒʒʒ˼ Keyboard ╾──╼┥0 bash STDOUT 1ʯ╾ʒ| ! STDERR 2ʯ╾ʒʒʒ╼ hello.txt ˾ʒʒʒʒʒʒʒʒʒʒ˽ echo “hello” 2> hello.txt 1>&2
24. Pipeline we can pipe output as input to another command/function cat names.txt | grep jimmy on function side, the first line can read the piped input from STDIN say_my_name() { read name echo $name } echo “Jimmy” | say_my_name
25. Flow Control KEYWORD <expression> BODY_HEADER BODY_COMMAND BODY_END
26. Flow Control - if if <expression> then … fi
27. Flow Control - for for var in <collection> do echo $var done collection can be spaced string bash automatically separate params, ex: 1 2 3 4 5
28. Flow Control - while while <expression> do … done
29. Flow Control - case case <param> in <pattern>) … ;; *) … ;; esac
30. Test command test command check the parameter and return 0/1 check result: test -z “” # return 0 -z : string is empty test -n “test” # return 0 -n : string is not empty
31. Test command also written as: [ -n $str ] often used in flow control if [ -n $str ]; then echo $str fi
32. Test command - string Use `man test` for all commands! [ -n “test” ] # => not empty [ -z “” ] # => empty [ $str = $str ] # => equal
33. Test command - integer [ $int1 -eq $int2 ] # => equal [ $int -gt $int2 ] # => greater than [ $int -lt $int2 ] # => lesser than
34. Test command - extended Extended test command [[ ]] use a more modern way of comparison: [[ $int1 = $int2 ]] [[ $int1 >= $int2 ]] [[ $int1 <= $int2 ]] It is always prefer to use this rather then [ ]
35. Test command - file [[ -e ./hello.txt ]] # => file exist [[ -f ./hello.txt ]] # => is file [[ -d /usr ]] # => is directory
36. Test command - misc [[ ! -n “test” ]] # => not [[ -n “test” && -n “test2” ]] # => and, -a in test [[ -z “test” || -z “test2 ]] # => or, -o in test
37. Unix commands Unix command is the essential part of bash Library in bash or bash perform as a glue for those commands. man is your good friend cat,grep,sort,uniq,comm,cut,perl,xargs,find,head, tail
38. Unix commands - cut cut can extract string from input cut -d <delimiter> -f <field> echo ‘jimmy, php, js, ruby’ | cut -d , -f 1,3- => jimmy js ruby
39. Unix commands - grep grep print line if pattern is matched cat names.txt | grep marshall => print line with marshall cat names.txt | grep -v marshall => print line is not marshall egrep can use regex as pattern
40. Unix commands - tr replace string from input echo “time” | tr “t” “l” => lime
41. Unix commands cat - output file cat name.txt head - print first n lines of input head name.txt -n 10 tail - print last n lines of input tail name.txt -n 5
42. Unix commands wc - count character, word or line of file cat name.txt | wc -l sort - sort input by alphabetical order cat name.txt | sort uniq - don’t print continue non-unique line, cat name.txt | uniq
43. Unix commands diff - show different lines in 2 files diff file1 file2 comm - show same lines in 2 files comm file1 file2 find - recursive find files by file name or content find ./ -name name.txt
44. Unix commands xargs - construct argument list from input echo ‘-al’ | xargs ls => ls -al extra extra…
45. AWK and SED Text process language read text file line by line detect pattern process line
46. SED Most useful SED syntax is replace, just works as vim replace syntax cat name | sed s/jimmy/james/
47. AWK <pattern> { <action> } cat name.txt | awk ‘{ print $1 }’ cat name.txt | /jimmy/i ‘{ print $0 }’ cat name.txt | awk ‘\ BEGIN { print ‘Start awk’} { print $0 } END { print ‘End awk’}
48. Perl… perl -pe ‘<perl code>’ expend to a loop, read input line by line and execute for each line. http://www.catonmat.net/blog/perl-one-liners-explained-part- one/
49. Bash like a Marshall print all environments git tag | egrep '/.*/' | rev | cut -d / -f 2- | rev | sort | uniq
50. Bash like a Marshall Run all spec that does not require rails find spec -name '*_spec.rb' | xargs egrep -L "require\\s+['\"]support\/rails['\"]" | egrep -v '^spec/nanny/' | grep -v does_boot_spec | xargs ruby bin/rspork
51. Bash like a marshall Print all story numbers from commits story_regex() { local story_number="$1" [[ -z "$story_number" ]] && story_number='\d+' echo '\[ *(\w+ *)?[#0-9, ]*(#'"$story_number"'\b)[#0-9, ]* *]' } stories_in() { git log --format='%b' "$1" | egrep -o "$(story_regex)" | egrep -o '(\d+)' | sort -n | uniq }
52. Bash like a Marshall find longest commit message and rank them git log —format="¶%H¬%B" | tr "\n" ' ' | tr -s ' ' | perl -pe 's/¶/\n/g' | grep -v Conflicts | grep -v 'diff --git' | perl -e 'print sort { length($b) <=> length($a) } <>' | head -25 | cut -d¬ -f 1 | xargs -n1 git log -1 --format='%h %an' | cat -n