MythBashers: An Adventure in Overlooked Technologies - Avdi Grimm

MythBashers: An Adventure in Overlooked Technologies - Avdi Grimm

A846fc46522b396026adcb62e162b7dc?s=128

Joy of Coding

March 07, 2014
Tweet

Transcript

  1. MYTHBASHERS AVDI GRIMM 0

  2. None
  3. JAMIE HYNEMAN

  4. TAKE A COMMON MYTH

  5. DEVISE AN EXPERIMENT

  6. BUILD A TEST RIG

  7. TEST THE MYTH!

  8. MAKE A DETERMINATION Confirmed Plausible Busted

  9. COOL TOYS

  10. ME TOO!

  11. RUBY

  12. LANGUAGES JavaScript Python Perl Java/C# Clojure/Erlang/Haskell/Scala PHP COBOL?

  13. LINGUA FRANCA

  14. SHELL!

  15. BASH

  16. THE MYTH "Bash isn't Webscale"

  17. THE BUILD A single-page web chat application, backed by a

    Bash script
  18. https://github.com/avdi/walrus

  19. HIGHLIGHTS

  20. SERVING HTTP

  21. NETCAT Like cat(1) for TCP/UDP sockets.

  22. A SIMPLE HTTP SERVER $ p r i n t

    f " H T T P / 1 . 1 2 0 0 O K \ r \ n \ r \ n H e l l o w o r l d \ n " | n c - l 8 0 8 0
  23. CLIENT $ c u r l l o c a

    l h o s t : 8 0 8 0 H e l l o w o r l d
  24. NETCAT OUTPUT $ p r i n t f "

    H T T P / 1 . 1 2 0 0 O K \ r \ n \ r \ n H e l l o w o r l d \ n " | n c - l 8 0 8 0 G E T / H T T P / 1 . 1 U s e r - A g e n t : c u r l / 7 . 3 2 . 0 H o s t : l o c a l h o s t : 8 0 8 0 A c c e p t : * / *
  25. SERVING PAGES DYNAMICALLY

  26. NETCAT IS READ/WRITE No callbacks.

  27. DYNAMIC NETCAT Branching on input.

  28. COPROCESSES

  29. c o p r o c N A M E

    C O M M A N D
  30. ROT13 COPROCESS $ c o p r o c r

    o t 1 3 { s t d b u f - o L t r ' [ A - Z a - z ] ' ' [ N - Z A - M n - z a - m ] ' ; [ 1 ] 2 6 8 1 2
  31. COPROCESS FILEHANDLES $ e c h o $ { r

    o t 1 3 [ 0 ] } 6 3 $ e c h o $ { r o t 1 3 [ 1 ] } 6 0
  32. WRITING AND READING $ e c h o " h

    e l l o , w o r l d " > & $ { r o t 1 3 [ 1 ] } $ e c h o " t h i s i s a s e c r e t " > & $ { r o t 1 3 [ 1 ] } $ r e a d l i n e < & $ { r o t 1 3 [ 0 ] } $ e c h o $ l i n e u r y y b , j b e y q $ r e a d l i n e < & $ { r o t 1 3 [ 0 ] } $ e c h o $ l i n e g u v f v f n f r p e r g
  33. ENDING A COPROCESS THE RUDE WAY $ k i l

    l $ { r o t 1 3 _ P I D }
  34. ENDING A COPROCESS THE POLITE WAY Closing a filehandle s

    o m e c o m m a n d 1 & > - Closing a filehandle without a command e x e c 6 0 & > -
  35. Interpolating the coprocess STDIN e x e c $ {

    r o t 1 3 [ 1 ] } & > - (Doesn't parse correctly)
  36. Using e v a l e v a l "

    e x e c $ { r o t 1 3 [ 1 ] } & > - " Simple, right?
  37. IÄ! IÄ! CTHULHU FHTAGN! Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn!

  38. NETCAT AS A COPROCESS # S t a r t

    n e t c a t c o p r o c n c ( n c - l 8 0 8 0 ) # P l u g i t i n t o a r e q u e s t h a n d l e r h a n d l e _ r e q u e s t $ { n c _ P I D } < & $ { n c [ 0 ] } > & $ { n c [ 1 ] } # C l o s e f i l e h a n d l e s e v a l " e x e c $ { n c [ 1 ] } > & - " e v a l " e x e c $ { n c [ 0 ] } < & - "
  39. ORGANIZING THE PROGRAM

  40. SHELL FUNCTIONS

  41. MINIATURE SHELL SCRIPTS Positional params ($ 1 , $ 2

    , $ * , etc.) Redirectable STDIN, STDOUT, and STDERR Return an integer exit status
  42. NAIVE ROT13 FUNCTION r o t 1 3 ( )

    { e c h o " $ 1 " | s t d b u f - o L t r ' [ A - Z a - z ] ' ' [ N - Z A - M n - z a - m ] ' } $ r o t 1 3 " p s s s t " c f f f g
  43. PIPELINE ROT13 FUNCTION r o t 1 3 ( )

    { s t d b u f - o L t r ' [ A - Z a - z ] ' ' [ N - Z A - M n - z a - m ] ' } $ e c h o " p s s s t " | r o t 1 3 c f f f g
  44. LOCAL VARIABLES f o o ( ) { b a

    r = 4 2 # g l o b a l ! } $ f o o $ e c h o $ { b a r } 4 2 f o o ( ) { l o c a l b a r = 4 2 # l o c a l } $ f o o $ e c h o $ { b a r } # n o o u t p u t
  45. DECLARING LOCALS BEFORE ASSIGNMENT g r e e t (

    ) { l o c a l n a m e r e a d n a m e e c h o " H e l l o , $ n a m e " }
  46. SERVING STATIC FILES H T T P / 1 .

    1 G E T / i n d e x . h t m l
  47. CONTENT TYPE t e x t / h t m

    l , a p p l i c a t i o n / j s o n , etc.
  48. ASSOCIATIVE ARRAYS (AKA Map, Dictionary, Hash) d e c l

    a r e - A c o n t e n t _ t y p e s = ( [ j s ] = t e x t / j a v a s c r i p t [ h t m l ] = t e x t / h t m l )
  49. FIND CONTENT TYPE FOR EXTENSION $ e c h o

    $ { c o n t e n t _ t y p e s [ h t m l ] } t e x t / h t m l With default: $ e c h o $ { c o n t e n t _ t y p e s [ f o o ] - t e x t / p l a i n } t e x t / p l a i n
  50. GET FILE EXTENSION $ f i l e = p

    u b l i c / i n d e x . h t m l $ e c h o $ { f i l e # # * . } h t m l
  51. SERVING A FILE l o c a l f i

    l e = " p u b l i c / $ { p a t h } " i f [ - f " $ { f i l e } " ] ; d o l o c a l e x t = " $ { f i l e # # * . } " l o c a l t y p e = " $ { c o n t e n t _ t y p e s [ $ { e x t } ] } " p r i n t f " H T T P / 1 . 1 2 0 0 O K \ r \ n " p r i n t f " C o n t e n t - T y p e : $ { t y p e } \ r \ n \ r \ n " c a t " $ { f i l e } " f i
  52. ACTOR MODEL 1. Asynchronous process 2. Queue for communication

  53. But this is Bash...

  54. None
  55. JOHN AGAR B-List Actor

  56. None
  57. None
  58. B‑MOVIE ACTOR MODEL 1. Asynchronous process: a subshell 2. Queue

    for communication: a named pipe (FIFO)
  59. CREATE AN ACTOR a g a r ( ) {

    # . . . }
  60. GET ACTOR NAME AND ARGS l o c a l

    n a m e = $ 1 l o c a l a r g s = $ { @ : 1 }
  61. CREATE PIPE m k d i r - p f

    i f o s l o c a l f i f o n a m e = f i f o s / $ { n a m e } m k f i f o $ { f i f o n a m e }
  62. SPAWN ACTOR { q u e u e = $

    { f i f o n a m e } $ { n a m e } $ { a r g s [ @ ] } r m - f $ { f i f o n a m e } } &
  63. "SEND" HELPER s e n d ( ) { l

    o c a l d e s t = $ 1 l o c a l m e s s a g e = $ { * : 1 } e c h o " $ { m e s s a g e } " > f i f o s / $ { d e s t } & }
  64. CREATE AN ACTOR a g a r m a i

    n
  65. "MAIN" FUNCTION m a i n ( ) { w

    h i l e t r u e ; d o s e r v e _ w i t h _ c o p r o c & r e a d < $ { q u e u e } d o n e }
  66. SIGNALING MAIN # . . . r e a d

    r e q _ l i n e s e n d m a i n " c o n t i n u e " # . . .
  67. TAKE THAT, ERLANG!

  68. REALTIME UPDATES

  69. SERVER‑SENT EVENTS (SSE) W3C Standards Track Supported: Firefox/Chrome/Safari/Opera

  70. CLIENT SIDE v a r s o u r c

    e = n e w E v e n t S o u r c e ( ' / u p d a t e s ' ) ; s o u r c e . o n m e s s a g e = f u n c t i o n ( e v e n t ) { a l e r t ( e v e n t . d a t a ) ; } ;
  71. SERVER SIDE H T T P / 1 . 1

    2 0 0 O K C o n t e n t - T y p e : t e x t / e v e n t - s t r e a m d a t a : T h i s i s m e s s a g e 1 d a t a : T h i s i s m e s s a g e 2 . . .
  72. Hooray for plain text!

  73. w h i l e r e a d d

    a t a < f i f o / u p d a t e s ; d o p r i n t f " d a t a : $ { d a t a } \ n \ n " d o n e
  74. THE MYTH "Bash isn't Webscale"

  75. I got it working... barely.

  76. ANALYSIS Slow Unreliable Only works in Firefox (?!) Leaks Processes

    FIFOs are a pain to work with So is netcat
  77. This was a terrible idea!

  78. MYTH: CONFIRMED

  79. EXPERIMENTATION, NOT DEMONSTRATION

  80. When we experiment—when we try things, and we fail—we start

    to ask why, and that’s when we learn. –Jamie Hyneman
  81. A story to tell

  82. Abstractions stripped away SQL HTTP

  83. Greater familiarity with tools Netcat psql

  84. Increased comfort with Bash scripting

  85. It was fun

  86. EXPERIMENT!

  87. THANK YOU! Code: https://github.com/avdi/walrus I make screencasts: RubyTapas.com @avdi /

    avdi@avdi.org
  88. MythBusters: © 2014 Discovery Communications, LLC. Photo of Tom Cruise

    by Gareth Cattermole © 2010 Getty Images "The Mole People" poster © Universal Pictures All others either unknown or in the public domain