Slide 1

Slide 1 text

Shell Scripting Lessons from a year administering remote machines

Slide 2

Slide 2 text

Basic Concepts

Slide 3

Slide 3 text

Shell Anatomy 101 • time php test.php 2>&1 | grep $NEEDLE > results.txt Shell Builtin Executables Arguments Redirection Pipe Environment Variable output

Slide 4

Slide 4 text

Process Pipes, a review Process stdin stdout stderr 2 1

Slide 5

Slide 5 text

2>&1 Process stdin stdout stderr 2 1

Slide 6

Slide 6 text

Shell Anatomy 101 • time php test.php 2>&1 | grep $NEEDLE > results.txt Shell Builtin Executables Arguments Redirection Pipe Environment Variable output

Slide 7

Slide 7 text

Tools to Know About

Slide 8

Slide 8 text

man • read the manual page for a given command • $ man grep

Slide 9

Slide 9 text

cat • Puts the contents of a file onto stdout • $ cat “hello.txt”
 Hello, world

Slide 10

Slide 10 text

head/tail • Puts the beginning or end of a file on stdout • great for dealing with logs, especially with -f • $ tail -f /var/log/system.log
 Hello, world

Slide 11

Slide 11 text

ps • list information about processes • $ ps axu | grep grep
 bnicholas 4521 0.0 0.0 2453264 692 s000 S+ 10:26PM 0:00.00 grep grep

Slide 12

Slide 12 text

kill/killall • send a signal to processes by pid or name • $ kill -SIGUSR1 2048 • $ killall perl

Slide 13

Slide 13 text

sort • Takes lines from stdin and sorts them on stdout • $ cat file.txt | sort • $ cat numbers.txt | sort -n

Slide 14

Slide 14 text

uniq • Removes duplicate lines from stdin to stdout • $ cat file.txt | sort | uniq • $ cat ~/history | sort | uniq -c | sort -n

Slide 15

Slide 15 text

grep • Search for a regular expression • can be used on stdin or files • $ cat /var/log/product.log | grep -i “error”

Slide 16

Slide 16 text

tr • “trim” text, replacing characters with others • $ cat sentence.txt | tr ‘ ‘ ‘\n’
 My
 name
 is
 ben.

Slide 17

Slide 17 text

sed • stream editor: a mini text editor • Similar to vim find+replace syntax, but so much more too • $ echo “hello world!” | sed -e “s/ w.*d/bn.u/“
 hello bn.u!

Slide 18

Slide 18 text

find • A file finder with extra • $ find . -name “*.orig” -print - delete • $ find . -type f —mtime -1 -print

Slide 19

Slide 19 text

awk • a small programming language for dealing with columns • ps aux | grep grep | awk ‘{print $2}’
 6845

Slide 20

Slide 20 text

Control Flow

Slide 21

Slide 21 text

conditionals • if [ $VAR = “foo” ]
 then
 echo “bar”
 else
 echo “baz”
 fi

Slide 22

Slide 22 text

Better conditionals • Use && and || when possible • cat a.txt | grep “hello” && echo “world”

Slide 23

Slide 23 text

For loops • for FILE in `ls`
 do
 cat $FILE
 done

Slide 24

Slide 24 text

SSH The magic glue that holds things together

Slide 25

Slide 25 text

Key-based auth • More secure and convenient than passwords • ssh-copy-id @ • Last time you’ll need to type a password • Also applies to git, scp, sftp, and others

Slide 26

Slide 26 text

~/.ssh/config • Set up usernames across computers • Keep connections alive longer • Set up tunnels

Slide 27

Slide 27 text

Great Tricks/Tools

Slide 28

Slide 28 text

autocomplete

Slide 29

Slide 29 text

fish

Slide 30

Slide 30 text

ack http://beyondgrep.com

Slide 31

Slide 31 text

tmux

Slide 32

Slide 32 text

vim

Slide 33

Slide 33 text

dsh

Slide 34

Slide 34 text

Some example problems