LANGUAGES
JavaScript
Python
Perl
Java/C#
Clojure/Erlang/Haskell/Scala
PHP
COBOL?
Slide 13
Slide 13 text
LINGUA FRANCA
Slide 14
Slide 14 text
SHELL!
Slide 15
Slide 15 text
BASH
Slide 16
Slide 16 text
THE MYTH
"Bash isn't Webscale"
Slide 17
Slide 17 text
THE BUILD
A single-page web chat application, backed by a Bash script
Slide 18
Slide 18 text
https://github.com/avdi/walrus
Slide 19
Slide 19 text
HIGHLIGHTS
Slide 20
Slide 20 text
SERVING HTTP
Slide 21
Slide 21 text
NETCAT
Like cat(1) for TCP/UDP sockets.
Slide 22
Slide 22 text
A SIMPLE HTTP SERVER
$ p
r
i
n
t
f "
H
T
T
P
/
1
.
1 2
0
0 O
K
\
r
\
n
\
r
\
n
H
e
l
l
o w
o
r
l
d
\
n
" |
n
c -
l 8
0
8
0
Slide 23
Slide 23 text
CLIENT
$ c
u
r
l l
o
c
a
l
h
o
s
t
:
8
0
8
0
H
e
l
l
o w
o
r
l
d
Slide 24
Slide 24 text
NETCAT OUTPUT
$ p
r
i
n
t
f "
H
T
T
P
/
1
.
1 2
0
0 O
K
\
r
\
n
\
r
\
n
H
e
l
l
o w
o
r
l
d
\
n
" |
n
c -
l 8
0
8
0
G
E
T / H
T
T
P
/
1
.
1
U
s
e
r
-
A
g
e
n
t
: c
u
r
l
/
7
.
3
2
.
0
H
o
s
t
: l
o
c
a
l
h
o
s
t
:
8
0
8
0
A
c
c
e
p
t
: *
/
*
Slide 25
Slide 25 text
SERVING PAGES
DYNAMICALLY
Slide 26
Slide 26 text
NETCAT IS READ/WRITE
No callbacks.
Slide 27
Slide 27 text
DYNAMIC NETCAT
Branching on input.
Slide 28
Slide 28 text
COPROCESSES
Slide 29
Slide 29 text
c
o
p
r
o
c N
A
M
E C
O
M
M
A
N
D
Slide 30
Slide 30 text
ROT13 COPROCESS
$ c
o
p
r
o
c r
o
t
1
3 { s
t
d
b
u
f -
o
L t
r '
[
A
-
Z
a
-
z
]
' '
[
N
-
Z
A
-
M
n
-
z
a
-
m
]
'
;
[
1
] 2
6
8
1
2
Slide 31
Slide 31 text
COPROCESS FILEHANDLES
$ e
c
h
o $
{
r
o
t
1
3
[
0
]
}
6
3
$ e
c
h
o $
{
r
o
t
1
3
[
1
]
}
6
0
Slide 32
Slide 32 text
WRITING AND READING
$ e
c
h
o "
h
e
l
l
o
, w
o
r
l
d
" >
&
$
{
r
o
t
1
3
[
1
]
}
$ e
c
h
o "
t
h
i
s i
s a s
e
c
r
e
t
" >
&
$
{
r
o
t
1
3
[
1
]
}
$ r
e
a
d l
i
n
e <
&
$
{
r
o
t
1
3
[
0
]
}
$ e
c
h
o $
l
i
n
e
u
r
y
y
b
, j
b
e
y
q
$ r
e
a
d l
i
n
e <
&
$
{
r
o
t
1
3
[
0
]
}
$ e
c
h
o $
l
i
n
e
g
u
v
f v
f n f
r
p
e
r
g
Slide 33
Slide 33 text
ENDING A COPROCESS
THE RUDE WAY
$ k
i
l
l $
{
r
o
t
1
3
_
P
I
D
}
Slide 34
Slide 34 text
ENDING A COPROCESS
THE POLITE WAY
Closing a filehandle
s
o
m
e
c
o
m
m
a
n
d 1
&
>
-
Closing a filehandle without a command
e
x
e
c 6
0
&
>
-
Slide 35
Slide 35 text
Interpolating the coprocess STDIN
e
x
e
c $
{
r
o
t
1
3
[
1
]
}
&
>
-
(Doesn't parse correctly)
Slide 36
Slide 36 text
Using e
v
a
l
e
v
a
l "
e
x
e
c $
{
r
o
t
1
3
[
1
]
}
&
>
-
"
Simple, right?
NETCAT AS A COPROCESS
# S
t
a
r
t n
e
t
c
a
t
c
o
p
r
o
c n
c ( n
c -
l 8
0
8
0 )
# P
l
u
g i
t i
n
t
o a r
e
q
u
e
s
t h
a
n
d
l
e
r
h
a
n
d
l
e
_
r
e
q
u
e
s
t $
{
n
c
_
P
I
D
} <
&
$
{
n
c
[
0
]
} >
&
$
{
n
c
[
1
]
}
# C
l
o
s
e f
i
l
e
h
a
n
d
l
e
s
e
v
a
l "
e
x
e
c $
{
n
c
[
1
]
}
>
&
-
"
e
v
a
l "
e
x
e
c $
{
n
c
[
0
]
}
<
&
-
"
Slide 39
Slide 39 text
ORGANIZING THE
PROGRAM
Slide 40
Slide 40 text
SHELL FUNCTIONS
Slide 41
Slide 41 text
MINIATURE SHELL SCRIPTS
Positional params ($
1
, $
2
, $
*
, etc.)
Redirectable STDIN, STDOUT, and STDERR
Return an integer exit status
Slide 42
Slide 42 text
NAIVE ROT13 FUNCTION
r
o
t
1
3
(
) {
e
c
h
o "
$
1
" | s
t
d
b
u
f -
o
L t
r '
[
A
-
Z
a
-
z
]
' '
[
N
-
Z
A
-
M
n
-
z
a
-
m
]
'
}
$ r
o
t
1
3 "
p
s
s
s
t
"
c
f
f
f
g
Slide 43
Slide 43 text
PIPELINE ROT13 FUNCTION
r
o
t
1
3
(
) {
s
t
d
b
u
f -
o
L t
r '
[
A
-
Z
a
-
z
]
' '
[
N
-
Z
A
-
M
n
-
z
a
-
m
]
'
}
$ e
c
h
o "
p
s
s
s
t
" | r
o
t
1
3
c
f
f
f
g
Slide 44
Slide 44 text
LOCAL VARIABLES
f
o
o
(
) {
b
a
r
=
4
2 # g
l
o
b
a
l
!
}
$ f
o
o
$ e
c
h
o $
{
b
a
r
}
4
2
f
o
o
(
) {
l
o
c
a
l b
a
r
=
4
2 # l
o
c
a
l
}
$ f
o
o
$ e
c
h
o $
{
b
a
r
} # n
o o
u
t
p
u
t
Slide 45
Slide 45 text
DECLARING LOCALS BEFORE
ASSIGNMENT
g
r
e
e
t
(
) {
l
o
c
a
l n
a
m
e
r
e
a
d n
a
m
e
e
c
h
o "
H
e
l
l
o
, $
n
a
m
e
"
}
Slide 46
Slide 46 text
SERVING STATIC FILES
H
T
T
P
/
1
.
1 G
E
T /
i
n
d
e
x
.
h
t
m
l
Slide 47
Slide 47 text
CONTENT TYPE
t
e
x
t
/
h
t
m
l
, a
p
p
l
i
c
a
t
i
o
n
/
j
s
o
n
, etc.
Slide 48
Slide 48 text
ASSOCIATIVE ARRAYS
(AKA Map, Dictionary, Hash)
d
e
c
l
a
r
e -
A c
o
n
t
e
n
t
_
t
y
p
e
s
=
(
[
j
s
]
=
t
e
x
t
/
j
a
v
a
s
c
r
i
p
t
[
h
t
m
l
]
=
t
e
x
t
/
h
t
m
l
)
Slide 49
Slide 49 text
FIND CONTENT TYPE FOR EXTENSION
$ e
c
h
o $
{
c
o
n
t
e
n
t
_
t
y
p
e
s
[
h
t
m
l
]
}
t
e
x
t
/
h
t
m
l
With default:
$ e
c
h
o $
{
c
o
n
t
e
n
t
_
t
y
p
e
s
[
f
o
o
]
-
t
e
x
t
/
p
l
a
i
n
}
t
e
x
t
/
p
l
a
i
n
Slide 50
Slide 50 text
GET FILE EXTENSION
$ f
i
l
e
=
p
u
b
l
i
c
/
i
n
d
e
x
.
h
t
m
l
$ e
c
h
o $
{
f
i
l
e
#
#
*
.
}
h
t
m
l
Slide 51
Slide 51 text
SERVING A FILE
l
o
c
a
l f
i
l
e
=
"
p
u
b
l
i
c
/
$
{
p
a
t
h
}
"
i
f [ -
f "
$
{
f
i
l
e
}
" ]
; d
o
l
o
c
a
l e
x
t
=
"
$
{
f
i
l
e
#
#
*
.
}
"
l
o
c
a
l t
y
p
e
=
"
$
{
c
o
n
t
e
n
t
_
t
y
p
e
s
[
$
{
e
x
t
}
]
}
"
p
r
i
n
t
f "
H
T
T
P
/
1
.
1 2
0
0 O
K
\
r
\
n
"
p
r
i
n
t
f "
C
o
n
t
e
n
t
-
T
y
p
e
: $
{
t
y
p
e
}
\
r
\
n
\
r
\
n
"
c
a
t "
$
{
f
i
l
e
}
"
f
i
Slide 52
Slide 52 text
ACTOR MODEL
1. Asynchronous process
2. Queue for communication
Slide 53
Slide 53 text
But this is Bash...
Slide 54
Slide 54 text
No content
Slide 55
Slide 55 text
JOHN AGAR
B-List Actor
Slide 56
Slide 56 text
No content
Slide 57
Slide 57 text
No content
Slide 58
Slide 58 text
B‑MOVIE ACTOR MODEL
1. Asynchronous process: a subshell
2. Queue for communication: a named pipe (FIFO)
Slide 59
Slide 59 text
CREATE AN ACTOR
a
g
a
r
(
) {
# .
.
.
}
Slide 60
Slide 60 text
GET ACTOR NAME AND ARGS
l
o
c
a
l n
a
m
e
=
$
1
l
o
c
a
l a
r
g
s
=
$
{
@
:
1
}
Slide 61
Slide 61 text
CREATE PIPE
m
k
d
i
r -
p f
i
f
o
s
l
o
c
a
l f
i
f
o
n
a
m
e
=
f
i
f
o
s
/
$
{
n
a
m
e
}
m
k
f
i
f
o $
{
f
i
f
o
n
a
m
e
}
Slide 62
Slide 62 text
SPAWN ACTOR
{
q
u
e
u
e
=
$
{
f
i
f
o
n
a
m
e
} $
{
n
a
m
e
} $
{
a
r
g
s
[
@
]
}
r
m -
f $
{
f
i
f
o
n
a
m
e
}
} &
Slide 63
Slide 63 text
"SEND" HELPER
s
e
n
d
(
) {
l
o
c
a
l d
e
s
t
=
$
1
l
o
c
a
l m
e
s
s
a
g
e
=
$
{
*
:
1
}
e
c
h
o "
$
{
m
e
s
s
a
g
e
}
" > f
i
f
o
s
/
$
{
d
e
s
t
} &
}
Slide 64
Slide 64 text
CREATE AN ACTOR
a
g
a
r m
a
i
n
Slide 65
Slide 65 text
"MAIN" FUNCTION
m
a
i
n
(
) {
w
h
i
l
e t
r
u
e
; d
o
s
e
r
v
e
_
w
i
t
h
_
c
o
p
r
o
c &
r
e
a
d < $
{
q
u
e
u
e
}
d
o
n
e
}
Slide 66
Slide 66 text
SIGNALING MAIN
# .
.
.
r
e
a
d r
e
q
_
l
i
n
e
s
e
n
d m
a
i
n "
c
o
n
t
i
n
u
e
"
# .
.
.
CLIENT SIDE
v
a
r s
o
u
r
c
e = n
e
w E
v
e
n
t
S
o
u
r
c
e
(
'
/
u
p
d
a
t
e
s
'
)
;
s
o
u
r
c
e
.
o
n
m
e
s
s
a
g
e = f
u
n
c
t
i
o
n (
e
v
e
n
t
) {
a
l
e
r
t
(
e
v
e
n
t
.
d
a
t
a
)
;
}
;
Slide 71
Slide 71 text
SERVER SIDE
H
T
T
P
/
1
.
1 2
0
0 O
K
C
o
n
t
e
n
t
-
T
y
p
e
: t
e
x
t
/
e
v
e
n
t
-
s
t
r
e
a
m
d
a
t
a
: T
h
i
s i
s m
e
s
s
a
g
e 1
d
a
t
a
: T
h
i
s i
s m
e
s
s
a
g
e 2
.
.
.
Slide 72
Slide 72 text
Hooray for plain text!
Slide 73
Slide 73 text
w
h
i
l
e r
e
a
d d
a
t
a < f
i
f
o
/
u
p
d
a
t
e
s
; d
o
p
r
i
n
t
f "
d
a
t
a
: $
{
d
a
t
a
}
\
n
\
n
"
d
o
n
e
Slide 74
Slide 74 text
THE MYTH
"Bash isn't Webscale"
Slide 75
Slide 75 text
I got it working... barely.
Slide 76
Slide 76 text
ANALYSIS
Slow
Unreliable
Only works in Firefox (?!)
Leaks Processes
FIFOs are a pain to work with
So is netcat
Slide 77
Slide 77 text
This was a terrible idea!
Slide 78
Slide 78 text
MYTH: CONFIRMED
Slide 79
Slide 79 text
EXPERIMENTATION, NOT
DEMONSTRATION
Slide 80
Slide 80 text
When we experiment—when we try things,
and we fail—we start to ask why, and that’s
when we learn.
–Jamie Hyneman
Slide 81
Slide 81 text
A story to tell
Slide 82
Slide 82 text
Abstractions stripped away
SQL
HTTP
Slide 83
Slide 83 text
Greater familiarity with tools
Netcat
psql
Slide 84
Slide 84 text
Increased comfort with Bash scripting
Slide 85
Slide 85 text
It was fun
Slide 86
Slide 86 text
EXPERIMENT!
Slide 87
Slide 87 text
THANK YOU!
Code: https://github.com/avdi/walrus
I make screencasts: RubyTapas.com
@avdi / [email protected]