Slide 1

Slide 1 text

Antediluvian Unix a guide to unix fundamentals revision 1

Slide 2

Slide 2 text

Why is UNIX so hard?

Slide 3

Slide 3 text

Why is UNIX so hard? - UNIX is made of tiny little pieces, all alike.

Slide 4

Slide 4 text

Why is UNIX so hard? - UNIX is made of tiny little pieces, all alike. - They can be put together in many ways.

Slide 5

Slide 5 text

Why is UNIX so hard? - UNIX is made of tiny little pieces, all alike. - They can be put together in many ways. - The pieces have manuals, but UNIX doesn’t.

Slide 6

Slide 6 text

Why is UNIX so hard? - UNIX is made of tiny little pieces, all alike. - They can be put together in many ways. - The pieces have manuals, but UNIX doesn’t. - The Mysteries of UNIX were handed down in an oral tradition.

Slide 7

Slide 7 text

Why is UNIX so hard? - UNIX is made of tiny little pieces, all alike. - They can be put together in many ways. - The pieces have manuals, but UNIX doesn’t. - The Mysteries of UNIX were handed down in an oral tradition. - Then the GNU/Linux flood happened.

Slide 8

Slide 8 text

The Deluge

Slide 9

Slide 9 text

The Deluge - Users adopted Linux as their first unix, learning everything from HOWTOs.

Slide 10

Slide 10 text

The Deluge - Users adopted Linux as their first unix, learning everything from HOWTOs. - The GNU system combined toolkit pieces.

Slide 11

Slide 11 text

The Deluge - Users adopted Linux as their first unix, learning everything from HOWTOs. - The GNU system combined toolkit pieces. - Distributions with X11 out of the box further obfuscated the shell.

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

We’re going to talk about antediluvian UNIX.

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

...but we’re going to use modern tools.

Slide 16

Slide 16 text

...but we’re going to use modern tools. Because being a fundamentalist does not mean being a relic.

Slide 17

Slide 17 text

UNIX at Rest the unix filesystem

Slide 18

Slide 18 text

The Time Before UNIX

Slide 19

Slide 19 text

The Time Before UNIX - Every file had a structure: a list of similar records.

Slide 20

Slide 20 text

The Age of UNIX

Slide 21

Slide 21 text

The Age of UNIX - Forcing a record structure on every file was a hassle.

Slide 22

Slide 22 text

The Age of UNIX - Forcing a record structure on every file was a hassle. - Plain text was easier to read and edit without special tools.

Slide 23

Slide 23 text

The Age of UNIX - Forcing a record structure on every file was a hassle. - Plain text was easier to read and edit without special tools. - So records were stored in the natural plain text way: columns and lines.

Slide 24

Slide 24 text

The Age of UNIX - Forcing a record structure on every file was a hassle. - Plain text was easier to read and edit without special tools. - So records were stored in the natural plain text way: columns and lines. - Every file is just a bytestream.

Slide 25

Slide 25 text

The Flat File 127.0.0.1 localhost 66.92.232.136 cheshirecat 66.92.232.135 redking 192.168.200.1 airport

Slide 26

Slide 26 text

The Flat File 127.0.0.1 localhost 66.92.232.136 cheshirecat 66.92.232.135 redking 192.168.200.1 airport FS = space RS = newline

Slide 27

Slide 27 text

The Flat File root:x:0:0::/root:/bin/sh rjbs:x:101:100:RJBS:/home/rjbs:/bin/zsh

Slide 28

Slide 28 text

The Flat File root:x:0:0::/root:/bin/sh rjbs:x:101:100:RJBS:/home/rjbs:/bin/zsh FS = colon RS = newline

Slide 29

Slide 29 text

Special Files

Slide 30

Slide 30 text

Special Files - Files that contain data are plain files.

Slide 31

Slide 31 text

Special Files - Files that contain data are plain files. - Other (special) files perform system magic.

Slide 32

Slide 32 text

Special Files - Files that contain data are plain files. - Other (special) files perform system magic. - directories

Slide 33

Slide 33 text

Special Files - Files that contain data are plain files. - Other (special) files perform system magic. - directories - symbolic links

Slide 34

Slide 34 text

Special Files - Files that contain data are plain files. - Other (special) files perform system magic. - directories - symbolic links - named pipes

Slide 35

Slide 35 text

Special Files - Files that contain data are plain files. - Other (special) files perform system magic. - directories - symbolic links - named pipes - sockets

Slide 36

Slide 36 text

Special Files - Files that contain data are plain files. - Other (special) files perform system magic. - directories - symbolic links - named pipes - sockets - devices

Slide 37

Slide 37 text

Permissions

Slide 38

Slide 38 text

Permissions - Every file has associated permissions.

Slide 39

Slide 39 text

Permissions - Every file has associated permissions. - Permissions govern reading, writing, and execution.

Slide 40

Slide 40 text

Permissions - Every file has associated permissions. - Permissions govern reading, writing, and execution. - And then there are the special permissions: s?id and sticky.

Slide 41

Slide 41 text

Permissions - Every file has associated permissions. - Permissions govern reading, writing, and execution. - And then there are the special permissions: s?id and sticky. ----------

Slide 42

Slide 42 text

Magic Numbers

Slide 43

Slide 43 text

Magic Numbers - UNIX supports many executable file types.

Slide 44

Slide 44 text

Magic Numbers - UNIX supports many executable file types. - Roughly 65,535, at the minimum.

Slide 45

Slide 45 text

Magic Numbers - UNIX supports many executable file types. - Roughly 65,535, at the minimum. - Magic numbers identify the file type.

Slide 46

Slide 46 text

Magic Numbers - UNIX supports many executable file types. - Roughly 65,535, at the minimum. - Magic numbers identify the file type. - They’re the first two bytes of the file.

Slide 47

Slide 47 text

The Magic Magic Number

Slide 48

Slide 48 text

The Magic Magic Number - One magic number, 0x2321, is common in modern unices.

Slide 49

Slide 49 text

The Magic Magic Number - One magic number, 0x2321, is common in modern unices. - In ASCII, it’s #!

Slide 50

Slide 50 text

The Magic Magic Number - One magic number, 0x2321, is common in modern unices. - In ASCII, it’s #! - In English, it’s “shebang!”

Slide 51

Slide 51 text

The Magic Magic Number - One magic number, 0x2321, is common in modern unices. - In ASCII, it’s #! - In English, it’s “shebang!” - The kernel will run the program named after #! with the given args, plus the filename.

Slide 52

Slide 52 text

#! in Action

Slide 53

Slide 53 text

#! in Action #!/bin/sh echo Good morning, $USER! ~/bin/hello

Slide 54

Slide 54 text

#! in Action #!/bin/sh echo Good morning, $USER! ~/bin/hello knave!rjbs% hello

Slide 55

Slide 55 text

#! in Action #!/bin/sh echo Good morning, $USER! ~/bin/hello execve( ”/bin/sh”, “/home/rjbs/hello” ); knave!rjbs% hello

Slide 56

Slide 56 text

#! in Action

Slide 57

Slide 57 text

#!/usr/bin/awk -f BEGIN { FS=”:”; } {} ($7 != “”) && ($5 !=””) { {print “Hello, “ $1 “!”;} END { print “Buh-bye!” } ~/bin/hi_all #! in Action

Slide 58

Slide 58 text

#!/usr/bin/awk -f BEGIN { FS=”:”; } {} ($7 != “”) && ($5 !=””) { {print “Hello, “ $1 “!”;} END { print “Buh-bye!” } ~/bin/hi_all #! in Action

Slide 59

Slide 59 text

#!/usr/bin/awk -f BEGIN { FS=”:”; } {} ($7 != “”) && ($5 !=””) { {print “Hello, “ $1 “!”;} END { print “Buh-bye!” } ~/bin/hi_all #! in Action

Slide 60

Slide 60 text

#!/usr/bin/awk -f BEGIN { FS=”:”; } {} ($7 != “”) && ($5 !=””) { {print “Hello, “ $1 “!”;} END { print “Buh-bye!” } ~/bin/hi_all #! in Action knave!rjbs% hi_all /etc/passwd

Slide 61

Slide 61 text

Hello, sync! Hello, shutdown! Hello, halt! Hello, operator! Hello, gdm! Hello, samael! Hello, rjbs! Hello, calliope! Hello, solios! Hello, photon! Buh-bye!

Slide 62

Slide 62 text

UNIX in Motion unix program execution

Slide 63

Slide 63 text

The Process Tree

Slide 64

Slide 64 text

The Process Tree - Every running process is the child of the process that ran it (except init).

Slide 65

Slide 65 text

The Process Tree - Every running process is the child of the process that ran it (except init). - A child can’t live without its parent.

Slide 66

Slide 66 text

The Process Tree - Every running process is the child of the process that ran it (except init). - A child can’t live without its parent. - A child starts life with a copy of its parent’s environment.

Slide 67

Slide 67 text

Environment

Slide 68

Slide 68 text

Environment - An environment is a set of data communicated from parent to child.

Slide 69

Slide 69 text

Environment - An environment is a set of data communicated from parent to child. - Parents teach children, but children can never teach parents.

Slide 70

Slide 70 text

Environment - An environment is a set of data communicated from parent to child. - Parents teach children, but children can never teach parents. - This is important.

Slide 71

Slide 71 text

Environment - An environment is a set of data communicated from parent to child. - Parents teach children, but children can never teach parents. - This is important. - Really important.

Slide 72

Slide 72 text

MAIL=/var/spoołmaiłrjbs MAILCHECK=60 IRCSERVER=us.slashnet.org IRCNICK=rjbs LESS=-M init -+ |-zsh

Slide 73

Slide 73 text

knave!rjbs% IRCNICK=rsignes irssi MAIL=/var/spoołmaiłrjbs MAILCHECK=60 IRCSERVER=us.slashnet.org IRCNICK=rjbs LESS=-M init -+ |-zsh

Slide 74

Slide 74 text

knave!rjbs% IRCNICK=rsignes irssi MAIL=/var/spoołmaiłrjbs MAILCHECK=60 IRCSERVER=us.slashnet.org LESS=-M init -+ |-zsh

Slide 75

Slide 75 text

knave!rjbs% IRCNICK=rsignes irssi MAIL=/var/spoołmaiłrjbs MAILCHECK=60 IRCSERVER=us.slashnet.org LESS=-M IRCNICK=rsignes init -+ |-zsh

Slide 76

Slide 76 text

knave!rjbs% IRCNICK=rsignes irssi MAIL=/var/spoołmaiłrjbs MAILCHECK=60 IRCSERVER=us.slashnet.org LESS=-M IRCNICK=rsignes init -+ |-zsh -irssi

Slide 77

Slide 77 text

MAIL=/var/spoołmaiłrjbs MAILCHECK=60 IRCSERVER=us.slashnet.org LESS=-M IRCNICK=rsignes init -+ |-zsh -irssi

Slide 78

Slide 78 text

MAIL=/var/spoołmaiłrjbs MAILCHECK=60 IRCSERVER=us.slashnet.org LESS=-M IRCNICK=rsignes init -+ |-zsh

Slide 79

Slide 79 text

MAIL=/var/spoołmaiłrjbs MAILCHECK=60 IRCSERVER=us.slashnet.org LESS=-M init -+ |-zsh

Slide 80

Slide 80 text

MAIL=/var/spoołmaiłrjbs MAILCHECK=60 IRCSERVER=us.slashnet.org IRCNICK=rjbs LESS=-M init -+ |-zsh

Slide 81

Slide 81 text

Signals

Slide 82

Slide 82 text

Signals - Processes mind their own business and try to run to completion.

Slide 83

Slide 83 text

Signals - Processes mind their own business and try to run to completion. - But they can be sent signals to make them do unusual things.

Slide 84

Slide 84 text

Signals - Processes mind their own business and try to run to completion. - But they can be sent signals to make them do unusual things. - Mostly, “stop running.”

Slide 85

Slide 85 text

Some Common Signals

Slide 86

Slide 86 text

Some Common Signals SIGHUP

Slide 87

Slide 87 text

Some Common Signals SIGHUP SIGINT

Slide 88

Slide 88 text

Some Common Signals SIGHUP SIGQUIT SIGINT

Slide 89

Slide 89 text

Some Common Signals SIGHUP SIGILL SIGQUIT SIGINT

Slide 90

Slide 90 text

Some Common Signals SIGHUP SIGILL SIGTRAP SIGQUIT SIGINT

Slide 91

Slide 91 text

Some Common Signals SIGHUP SIGILL SIGTRAP SIGABRT SIGQUIT SIGINT

Slide 92

Slide 92 text

Some Common Signals SIGHUP SIGILL SIGBUS SIGTRAP SIGABRT SIGQUIT SIGINT

Slide 93

Slide 93 text

Some Common Signals SIGHUP SIGILL SIGBUS SIGTRAP SIGABRT SIGKILL SIGQUIT SIGINT

Slide 94

Slide 94 text

Some Common Signals SIGHUP SIGILL SIGSEGV SIGBUS SIGTRAP SIGABRT SIGKILL SIGQUIT SIGINT

Slide 95

Slide 95 text

Some Common Signals SIGHUP SIGILL SIGSEGV SIGBUS SIGTRAP SIGABRT SIGKILL SIGQUIT SIGINT SIGTERM

Slide 96

Slide 96 text

Some Common Signals SIGHUP SIGSTOP SIGILL SIGSEGV SIGBUS SIGTRAP SIGABRT SIGKILL SIGQUIT SIGINT SIGTERM

Slide 97

Slide 97 text

Some Common Signals SIGHUP SIGCONT SIGSTOP SIGILL SIGSEGV SIGBUS SIGTRAP SIGABRT SIGKILL SIGQUIT SIGINT SIGTERM

Slide 98

Slide 98 text

Some Common Signals SIGHUP SIGCONT SIGSTOP SIGILL SIGSEGV SIGBUS SIGTRAP SIGABRT SIGKILL SIGQUIT SIGINT SIGTERM SIGPWR

Slide 99

Slide 99 text

The Time Before UNIX

Slide 100

Slide 100 text

The Time Before UNIX - Your computer ran one program.

Slide 101

Slide 101 text

The Time Before UNIX - Your computer ran one program. - When that program was done, it ran the next.

Slide 102

Slide 102 text

The Time Before UNIX - Your computer ran one program. - When that program was done, it ran the next. - Lather, rinse, repeat.

Slide 103

Slide 103 text

The Time Before UNIX - Your computer ran one program. - When that program was done, it ran the next. - Lather, rinse, repeat. - Batch processing.

Slide 104

Slide 104 text

The Age of UNIX

Slide 105

Slide 105 text

The Age of UNIX - Many programs run at once.

Slide 106

Slide 106 text

The Age of UNIX - Many programs run at once. - Interactive programs became possible.

Slide 107

Slide 107 text

The Age of UNIX - Many programs run at once. - Interactive programs became possible. - ...but batch processing is not dead.

Slide 108

Slide 108 text

Job Control

Slide 109

Slide 109 text

Job Control - A system for handling batch processing in UNIX.

Slide 110

Slide 110 text

Job Control - A system for handling batch processing in UNIX. - Jobs can be

Slide 111

Slide 111 text

Job Control - A system for handling batch processing in UNIX. - Jobs can be - paused

Slide 112

Slide 112 text

Job Control - A system for handling batch processing in UNIX. - Jobs can be - paused - run in the background

Slide 113

Slide 113 text

Job Control - A system for handling batch processing in UNIX. - Jobs can be - paused - run in the background - brought to the foreground

Slide 114

Slide 114 text

Job Control knave!rjbs%

Slide 115

Slide 115 text

Job Control knave!rjbs% jobs [1]+ Running make bzImage& [2] Running backup& [3]- Suspended wget -rnp http... [4] Running pdflatex thesis.tex knave!rjbs%

Slide 116

Slide 116 text

Job Control knave!rjbs% jobs [1]+ Running make bzImage& [2] Running backup& [3]- Suspended wget -rnp http... [4] Running pdflatex thesis.tex knave!rjbs% mutt

Slide 117

Slide 117 text

Job Control knave!rjbs% jobs [1]+ Running make bzImage& [2] Running backup& [3]- Suspended wget -rnp http... [4] Running pdflatex thesis.tex knave!rjbs% mutt [2] Done backup& [4] Done pdflatex thesis.tex

Slide 118

Slide 118 text

#!/bin/sh for i in 1 2 3 4 5 6 7 8 9 10 do sleep $i done ~/bin/waiter Job Control in Action How long does this take to run?

Slide 119

Slide 119 text

~/bin/waiter Job Control in Action How long does this take to run? #!/bin/sh for i in 1 2 3 4 5 6 7 8 9 10 do (sleep $i)& done wait

Slide 120

Slide 120 text

Job Control

Slide 121

Slide 121 text

Job Control - Ctrl-Z

Slide 122

Slide 122 text

Job Control - Ctrl-Z - bg and fg

Slide 123

Slide 123 text

Job Control - Ctrl-Z - bg and fg - &

Slide 124

Slide 124 text

Job Control - Ctrl-Z - bg and fg - & - jobs

Slide 125

Slide 125 text

Job Control - Ctrl-Z - bg and fg - & - jobs - wait

Slide 126

Slide 126 text

Bytestreams

Slide 127

Slide 127 text

Bytestreams - Files, you remember, are just bytestreams.

Slide 128

Slide 128 text

Bytestreams - Files, you remember, are just bytestreams. - A bytestream is list of bytes, in order.

Slide 129

Slide 129 text

Bytestreams - Files, you remember, are just bytestreams. - A bytestream is list of bytes, in order. - But it’s a stream: it flows

Slide 130

Slide 130 text

Bytestreams - Files, you remember, are just bytestreams. - A bytestream is list of bytes, in order. - But it’s a stream: it flows - in and out of programs

Slide 131

Slide 131 text

Bytestreams - Files, you remember, are just bytestreams. - A bytestream is list of bytes, in order. - But it’s a stream: it flows - in and out of programs - through pipes

Slide 132

Slide 132 text

Bytestreams

Slide 133

Slide 133 text

Bytestreams - Programs read and write from bytestreams all the time.

Slide 134

Slide 134 text

Bytestreams - Programs read and write from bytestreams all the time. - But a UNIX program’s favorite bytestreams are the standard IO streams.

Slide 135

Slide 135 text

Bytestreams - Programs read and write from bytestreams all the time. - But a UNIX program’s favorite bytestreams are the standard IO streams. - stdin - standard input

Slide 136

Slide 136 text

Bytestreams - Programs read and write from bytestreams all the time. - But a UNIX program’s favorite bytestreams are the standard IO streams. - stdin - standard input - stdout - standard output (1)

Slide 137

Slide 137 text

Bytestreams - Programs read and write from bytestreams all the time. - But a UNIX program’s favorite bytestreams are the standard IO streams. - stdin - standard input - stdout - standard output (1) - stderr - standard err (2)

Slide 138

Slide 138 text

Standard I/O

Slide 139

Slide 139 text

Standard I/O - The standard I/O streams are usually connected to the terminal.

Slide 140

Slide 140 text

Standard I/O - The standard I/O streams are usually connected to the terminal. - But you can redirect them.

Slide 141

Slide 141 text

Standard I/O - The standard I/O streams are usually connected to the terminal. - But you can redirect them. - Understanding I/O redirection is fundamental to using UNIX effectively.

Slide 142

Slide 142 text

#!/bin/sh alias > .bash_tmp_1 cat .bash_alias .bash_tmp1 \ > .bash_tmp_2 sort < .bash_tmp_2 > .bash_tmp_1 uniq < .bash_tmp_1> .bash_tmp_2 mv .bash_tmp_2 .bash_alias rm .bash_tmp_1 ~/bin/savalias Redirection in Action

Slide 143

Slide 143 text

#!/bin/sh alias \ | cat - .bash_alias \ | sort | uniq > .bash_alias ~/bin/savalias Redirection in Action

Slide 144

Slide 144 text

#!/bin/sh netstat -n \ | grep ESTABLISHED \ | cut -c 45-65 \ | awk ‘BEGIN{FS=”:”}{print $1}’ \ | sort -n \ | uniq ~/bin/connected Redirection in Action

Slide 145

Slide 145 text

Redirection

Slide 146

Slide 146 text

Redirection - > - >> - < - | - `` or $()

Slide 147

Slide 147 text

Arguments

Slide 148

Slide 148 text

Arguments - Programs are passed a list of arguments, given on the command line.

Slide 149

Slide 149 text

Arguments - Programs are passed a list of arguments, given on the command line. - These are usually switches that change how the program will work.

Slide 150

Slide 150 text

Arguments - Programs are passed a list of arguments, given on the command line. - These are usually switches that change how the program will work. - In lieu of stdio, arguments may name files on which to operate.

Slide 151

Slide 151 text

Return Values

Slide 152

Slide 152 text

Return Values - When a job finishes, its status is reported by a single integer.

Slide 153

Slide 153 text

Return Values - When a job finishes, its status is reported by a single integer. - This “return value” indicates success, failure, or other information.

Slide 154

Slide 154 text

Process stdin stdout stderr arguments return value environment other processes signals

Slide 155

Slide 155 text

The UNIX Toolkit elementary programs

Slide 156

Slide 156 text

The Toolkit

Slide 157

Slide 157 text

The Toolkit - Little utilities.

Slide 158

Slide 158 text

The Toolkit - Little utilities. - Found almost everywhere.

Slide 159

Slide 159 text

The Toolkit - Little utilities. - Found almost everywhere. - Work the same almost everywhere.

Slide 160

Slide 160 text

Some Toolkit Programs

Slide 161

Slide 161 text

Some Toolkit Programs cd

Slide 162

Slide 162 text

Some Toolkit Programs cd ls

Slide 163

Slide 163 text

Some Toolkit Programs cd cp ls

Slide 164

Slide 164 text

Some Toolkit Programs cd mv cp ls

Slide 165

Slide 165 text

Some Toolkit Programs cd mv rm cp ls

Slide 166

Slide 166 text

Some Toolkit Programs cd mv rm mkdir cp ls

Slide 167

Slide 167 text

Some Toolkit Programs cd mv chmod rm mkdir cp ls

Slide 168

Slide 168 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls

Slide 169

Slide 169 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls sort

Slide 170

Slide 170 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls sort uniq

Slide 171

Slide 171 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls sort cat uniq

Slide 172

Slide 172 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls sort test cat uniq

Slide 173

Slide 173 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls sort test basename cat uniq

Slide 174

Slide 174 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls sort test basename cut cat uniq

Slide 175

Slide 175 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls sort test tr basename cut cat uniq

Slide 176

Slide 176 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls sort test tr basename cut grep cat uniq

Slide 177

Slide 177 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls sort test tr basename cut grep cat uniq seq

Slide 178

Slide 178 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls sort test tr basename cut grep cat uniq seq yes

Slide 179

Slide 179 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls sort test tr basename cut grep cat uniq seq true yes

Slide 180

Slide 180 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls sort test tr basename cut grep cat uniq seq false true yes

Slide 181

Slide 181 text

Some Toolkit Programs cd mv chmod rm mkdir chown cp ls sort test tr basename cut grep cat uniq seq false true yes

Slide 182

Slide 182 text

#!/bin/sh cat /etc/passwd /sbin/armageddon The Toolkit in Action

Slide 183

Slide 183 text

#!/bin/sh cat /etc/passwd \ | grep :100: /sbin/armageddon The Toolkit in Action

Slide 184

Slide 184 text

#!/bin/sh cat /etc/passwd \ | grep :100: \ | cut -d: -f6 /sbin/armageddon The Toolkit in Action

Slide 185

Slide 185 text

#!/bin/sh rm -R $(cat /etc/passwd \ | grep :100: \ | cut -d: -f6) /sbin/armageddon The Toolkit in Action

Slide 186

Slide 186 text

#!/bin/sh yes | rm -R $(cat /etc/passwd \ | grep :100: \ | cut -d: -f6) \ echo Users obliterated! /sbin/armageddon The Toolkit in Action

Slide 187

Slide 187 text

The Shell

Slide 188

Slide 188 text

The Shell - Just another piece of the toolkit.

Slide 189

Slide 189 text

The Shell - Just another piece of the toolkit. - But the first among equals.

Slide 190

Slide 190 text

The Shell - Just another piece of the toolkit. - But the first among equals. - Provides flow control for processes.

Slide 191

Slide 191 text

The Shell - Just another piece of the toolkit. - But the first among equals. - Provides flow control for processes. - Also, built-in utilities.

Slide 192

Slide 192 text

The Shell - Just another piece of the toolkit. - But the first among equals. - Provides flow control for processes. - Also, built-in utilities. - ...but most of these are extraneous.

Slide 193

Slide 193 text

Flow Control

Slide 194

Slide 194 text

Flow Control - Controls:

Slide 195

Slide 195 text

Flow Control - Controls: - whether a process will run

Slide 196

Slide 196 text

Flow Control - Controls: - whether a process will run - when a process will run

Slide 197

Slide 197 text

Flow Control - Controls: - whether a process will run - when a process will run - process iteration over a list of data

Slide 198

Slide 198 text

#!/bin/sh if [ $UID = 0 ]; then echo You’re root! else echo You’re a luser! fi Flow Control

Slide 199

Slide 199 text

#!/bin/sh if [ $UID = 0 ]; then echo You’re root! else echo You’re a luser! fi Flow Control So, how does sh compare $UID and zero?

Slide 200

Slide 200 text

Process stdin stdout stderr arguments return value environment other processes signals

Slide 201

Slide 201 text

#!/bin/sh if test $UID = 0; then echo You’re root! else echo You’re a luser! fi if

Slide 202

Slide 202 text

#!/bin/sh if cp $@ $HOME; then echo Everything’s fine! else echo WE’RE ALL GONNA DIE! fi if

Slide 203

Slide 203 text

#!/bin/sh if [ $UID = 0 ]; then echo You’re root! else echo You’re a luser! fi Flow Control

Slide 204

Slide 204 text

#!/bin/sh for dir in ~/code/*; do cd $dir; make done for ~/bin/makeall

Slide 205

Slide 205 text

#!/bin/sh for dir in ~/code/*; do \ if [ -d $dir -a -r $dir/Makefile ]; then cd $dir; make fi done for and if ~/bin/makeall

Slide 206

Slide 206 text

#!/bin/sh for dir in ~/code/*; do \ if [ -d $dir -a -r $dir/Makefile ]; then (cd $dir; make)& fi done wait for, if, and job control ~/bin/makeall

Slide 207

Slide 207 text

#!/bin/sh echo $$ > ~/unique_pid while [ $$ = $(cat ~/unique_pid) ] do sleep 1 done echo Replaced by $(cat ~/unique_pid) while ~/bin/unique

Slide 208

Slide 208 text

#!/bin/sh echo $$ > ~/unique_pid until [ $$ != $(cat ~/unique_pid) ] do sleep 1 done echo Replaced by $(cat ~/unique_pid) until ~/bin/unique

Slide 209

Slide 209 text

#!/bin/sh if [ -r ~/unique_pid ]; then kill $(cat ~/unique_pid) rm ~/unique_pid fi echo $$ > ~/unique_pid while true do rm -R $1/* > /dev/null done while and if ~/bin/rmcont

Slide 210

Slide 210 text

#!/bin/sh select user in $(who|cut -f1 -d” “|uniq) do if [ $user ]; then talk $user; fi break done select ~/bin/chat

Slide 211

Slide 211 text

#!/bin/sh select user in $(who|cut -f1 -d” “|uniq) do if [ $user ]; then talk $user; fi break done select ~/bin/chat 1) rjbs 2) calliope #?

Slide 212

Slide 212 text

#!/bin/sh case “$1” in *.tgz|*.tar.gz) tar zxvf $1;; *.tar.bz2) tar jxvf $1;; *.gz) gunzip $1;; *.tar) tar xvf $1;; *.zip) unzip $1;; *.shar.gz) cat $1 | gunzip | unshar > \ $(basename $1 .shar.gz) ;; *) echo Unknown archive type! esac case ~/bin/unarc

Slide 213

Slide 213 text

Redefining Your UNIX customizing the interactive experience

Slide 214

Slide 214 text

Why script?

Slide 215

Slide 215 text

Why script? - Any shell command can be entered directly at the command line.

Slide 216

Slide 216 text

Why script? - Any shell command can be entered directly at the command line. - So why script?

Slide 217

Slide 217 text

Why script? - Any shell command can be entered directly at the command line. - So why script? - Scripting lets you remember useful recipes.

Slide 218

Slide 218 text

Why script? - Any shell command can be entered directly at the command line. - So why script? - Scripting lets you remember useful recipes. - And abstract them.

Slide 219

Slide 219 text

My Favorite Scripts

Slide 220

Slide 220 text

My Favorite Scripts - ...are incredibly simple.

Slide 221

Slide 221 text

My Favorite Scripts - ...are incredibly simple. - They just eliminate some keystrokes.

Slide 222

Slide 222 text

My Favorite Scripts - ...are incredibly simple. - They just eliminate some keystrokes. #!/bin/sh screen -x $@ ~/bin/sx

Slide 223

Slide 223 text

My Favorite Scripts - ...are incredibly simple. - They just eliminate some keystrokes. #!/bin/sh screen -x $@ ~/bin/sx #!/bin/sh module-starter --author=”Ricardo Signes \ --email=”[email protected]” $@ ~/bin/n2pm

Slide 224

Slide 224 text

Functions

Slide 225

Slide 225 text

Functions - These don’t need to be scripts.

Slide 226

Slide 226 text

Functions - These don’t need to be scripts. - You can do this with functions.

Slide 227

Slide 227 text

Functions - These don’t need to be scripts. - You can do this with functions. knave!rjbs% sx() { screen -x $@ }

Slide 228

Slide 228 text

Functions - These don’t need to be scripts. - You can do this with functions. knave!rjbs% cs() { cp $2 $2.bak; cp $1 $2 } knave!rjbs% sx() { screen -x $@ }

Slide 229

Slide 229 text

Functions - These don’t need to be scripts. - You can do this with functions. knave!rjbs% cs() { cp $2 $2.bak; cp $1 $2 } knave!rjbs% ch() { awk ‘FNR==1{for(i=1;i<=NF;i++)print $i}{next}’ $@ } knave!rjbs% sx() { screen -x $@ }

Slide 230

Slide 230 text

No content

Slide 231

Slide 231 text

Aliases

Slide 232

Slide 232 text

Aliases - Aliases can do much of what functions can.

Slide 233

Slide 233 text

Aliases - Aliases can do much of what functions can. - Quoting makes them hard to write.

Slide 234

Slide 234 text

Aliases - Aliases can do much of what functions can. - Quoting makes them hard to write. - Single-lining makes them harder.

Slide 235

Slide 235 text

Aliases - Aliases can do much of what functions can. - Quoting makes them hard to write. - Single-lining makes them harder. - Those can be worked around, but why bother?

Slide 236

Slide 236 text

Aliases

Slide 237

Slide 237 text

Aliases - There are three reasons to bother:

Slide 238

Slide 238 text

Aliases - There are three reasons to bother: - better recursion protection

Slide 239

Slide 239 text

Aliases - There are three reasons to bother: - better recursion protection - non-command aliases (the space hack)

Slide 240

Slide 240 text

Aliases - There are three reasons to bother: - better recursion protection - non-command aliases (the space hack) - higher precedence

Slide 241

Slide 241 text

Aliases - There are three reasons to bother: - better recursion protection - non-command aliases (the space hack) - higher precedence - I still don’t bother.

Slide 242

Slide 242 text

Functions

Slide 243

Slide 243 text

Functions - More reasons to use functions:

Slide 244

Slide 244 text

Functions - More reasons to use functions: - they’re already in memory

Slide 245

Slide 245 text

Functions - More reasons to use functions: - they’re already in memory - they run in the current process

Slide 246

Slide 246 text

Functions - More reasons to use functions: - they’re already in memory - they run in the current process - they produce scope

Slide 247

Slide 247 text

Scope

Slide 248

Slide 248 text

Scope - Scope is the place where something is visible.

Slide 249

Slide 249 text

Scope - Scope is the place where something is visible. - Using scope wisely makes life easy.

Slide 250

Slide 250 text

Scope - Scope is the place where something is visible. - Using scope wisely makes life easy. - The file hierarchy is a kind of scope.

Slide 251

Slide 251 text

Scope - Scope is the place where something is visible. - Using scope wisely makes life easy. - The file hierarchy is a kind of scope. - So is the process environment.

Slide 252

Slide 252 text

Scope cvsroot() { if [ “$1” != “” ]; then if [ -f $CVSROOT_DIR/$1 ]; then CVSROOT=$(cat $CVSROOT_DIR/$1) else echo cvsroot: $CVSROOT_DIR/$1 is invalid fi else echo cvsroot: currently $CVSROOT fi } Functions can alter the running shell. (It’s in scope.)

Slide 253

Slide 253 text

Scope mown() { local filename = $1 local owner = $(ls -l $1|cut -c21-29) mail $owner } Function variables can be made local (scope-limited).

Slide 254

Slide 254 text

Scope #!/bin/sh hypotenuses() { pythagoras() { echo $(dc -e “$1 2 ^ $2 2 ^ v f”) } for $side in $*; do pythagoras $side; done } Nested functions are scoped to the enclosing function. The pythagoras() function is not visible outside of hypotenuses().

Slide 255

Slide 255 text

Scope #!/bin/sh hypotenuse() { echo $(dc -e “$1 2 ^ $2 2 ^ v f”) } for side do hypotenuse $(dc -e “$side 2 / f”) Functions are scoped to their enclosing script. The special-purpose hypotenuse() function will go away when the script is done running.

Slide 256

Slide 256 text

Scope for func in ~/.bash/functions/*; do . $i done This is how I load my functions. (it’s in my profile)

Slide 257

Slide 257 text

Scripts

Slide 258

Slide 258 text

- When are scripts useful? Scripts

Slide 259

Slide 259 text

- When are scripts useful? - programs not run under a login Scripts

Slide 260

Slide 260 text

- When are scripts useful? - programs not run under a login - programs too large to leave in memory Scripts

Slide 261

Slide 261 text

- When are scripts useful? - programs not run under a login - programs too large to leave in memory - rarely-used programs Scripts

Slide 262

Slide 262 text

- When are scripts useful? - programs not run under a login - programs too large to leave in memory - rarely-used programs - In almost no case do I write sh scripts. Scripts

Slide 263

Slide 263 text

Scripts

Slide 264

Slide 264 text

Scripts - ...but I write plenty of other scripts.

Slide 265

Slide 265 text

Scripts - ...but I write plenty of other scripts. - Shell scripts lose utility because I’m always in the shell.

Slide 266

Slide 266 text

Scripts - ...but I write plenty of other scripts. - Shell scripts lose utility because I’m always in the shell. - But I’m not limited to shell scripts! I have the shebang!

Slide 267

Slide 267 text

#!/usr/bin/whatever

Slide 268

Slide 268 text

#!/usr/bin/whatever - Almost any interpreted data file can be run.

Slide 269

Slide 269 text

#!/usr/bin/whatever - Almost any interpreted data file can be run. - It needs a shebang and +x permissions.

Slide 270

Slide 270 text

#!/usr/bin/whatever - Almost any interpreted data file can be run. - It needs a shebang and +x permissions. - The key is knowing the right tool.

Slide 271

Slide 271 text

#!/usr/bin/awk -f BEGIN { FS=”:”; } {} ($7 != “”) && ($5 !=””) { {print “Hello, “ $1 “!”;} END { print “Buh-bye!” } ~/bin/hi_all #! in Action

Slide 272

Slide 272 text

#!/usr/bin/awk -f BEGIN { FS=”:” } ($7 != “”) && ($5 !=””) {print “Hello, “ $1 “!”;} {next} END { print “Buh-bye!” } ~/bin/hi_all #!/usr/bin/whatever

Slide 273

Slide 273 text

#!/usr/bin/whatever #!/usr/bin/make -f RSYNC_SW=”--delete -ave ssh” default: web gnupg logs mirrors: ifarchive minicpan all: default mirrors web: rsync ${RSYNC_SW} ${HOME}/public_htmł cheshire:~/public_htmł gnupg: rsync ${RSYNC_SW} cheshire:~/.gnupg/ ${HOME}/backup/gnupg/ logs: rsync ${RSYNC_SW} cheshire:/var/www/ ${HOME}/backup/www/ rsync ${RSYNC_SW} cheshire:~/cvs/ ${HOME}/backup/cvs/ rsync ${RSYNC_SW} cheshire:~/log/ ${HOME}/backup/log/ ifarchive: rsync ${RSYNC_SW} cheshire:~/ifarchive/ ${HOME}/mirrors/ifarchive/ ~/bin/backup

Slide 274

Slide 274 text

#!/usr/bin/mpg123 -z -@ /usr/bin/music/outkast/ms.jackson.mp3 /usr/bin/music/devøshout.mp3 /usr/bin/music/talking_heads/television_man.mp3 ... ~/bin/bestsongs #!/usr/bin/whatever

Slide 275

Slide 275 text

#!/usr/bin/mpg123 -z -@ /usr/bin/music/outkast/ms.jackson.mp3 /usr/bin/music/devøshout.mp3 /usr/bin/music/talking_heads/television_man.mp3 ... ~/bin/bestsongs #!/usr/bin/whatever addsong() { if [ “$1” != “” -a -r “$1” ]; then echo $1 >> ~/bin/bestsongs; else echo can’t add file }

Slide 276

Slide 276 text

#!/usr/bin/perl #!/usr/bin/whatever #!/usr/bin/python #!/usr/bin/ruby #!/usr/bin/clisp

Slide 277

Slide 277 text

More Slides things that deserve more time

Slide 278

Slide 278 text

Globbing

Slide 279

Slide 279 text

Globbing - * and ?

Slide 280

Slide 280 text

Globbing - * and ? - [ABC]

Slide 281

Slide 281 text

Globbing - * and ? - [ABC] - {xyz}

Slide 282

Slide 282 text

Globbing - * and ? - [ABC] - {xyz} - zsh globbing

Slide 283

Slide 283 text

Environment Variables

Slide 284

Slide 284 text

Environment Variables - shell vars v. environment vars

Slide 285

Slide 285 text

Environment Variables - shell vars v. environment vars - $x and ${x}

Slide 286

Slide 286 text

Environment Variables - shell vars v. environment vars - $x and ${x} - ${x:-default}

Slide 287

Slide 287 text

Environment Variables - shell vars v. environment vars - $x and ${x} - ${x:-default} - ${x:=default}

Slide 288

Slide 288 text

Environment Variables - shell vars v. environment vars - $x and ${x} - ${x:-default} - ${x:=default} - ${x:?epitaph}

Slide 289

Slide 289 text

Environment Variables - shell vars v. environment vars - $x and ${x} - ${x:-default} - ${x:=default} - ${x:?epitaph} - ${x:+iftrue}

Slide 290

Slide 290 text

Environment Variables - shell vars v. environment vars - $x and ${x} - ${x:-default} - ${x:=default} - ${x:?epitaph} - ${x:+iftrue} - ${x:offset:length}

Slide 291

Slide 291 text

STDIO

Slide 292

Slide 292 text

STDIO - `` and $()

Slide 293

Slide 293 text

STDIO - `` and $() - >&

Slide 294

Slide 294 text

STDIO - `` and $() - >& - << and <<-

Slide 295

Slide 295 text

STDIO - `` and $() - >& - << and <<- - command blocks { }

Slide 296

Slide 296 text

STDIO - `` and $() - >& - << and <<- - command blocks { } - the read builtin

Slide 297

Slide 297 text

Magic Nonsense

Slide 298

Slide 298 text

Magic Nonsense - (job&) # job into void

Slide 299

Slide 299 text

Magic Nonsense - (job&) # job into void - $((2+2))

Slide 300

Slide 300 text

Magic Nonsense - (job&) # job into void - $((2+2)) - “” versus ‘’

Slide 301

Slide 301 text

Magic Nonsense - (job&) # job into void - $((2+2)) - “” versus ‘’ - ugh! ‘abc’\’’def’

Slide 302

Slide 302 text

Magic Nonsense - (job&) # job into void - $((2+2)) - “” versus ‘’ - ugh! ‘abc’\’’def’ - eval

Slide 303

Slide 303 text

Magic Nonsense - (job&) # job into void - $((2+2)) - “” versus ‘’ - ugh! ‘abc’\’’def’ - eval - source

Slide 304

Slide 304 text

Precedence Hackery - command - builtin - enable