In the Back of Your Mind

Slide 1

Slide 1 text

IN THE BACK OF YOUR MIND beanstalkd and 0MQ through the Dominic System

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

THE DOMINIC SYSTEM

Slide 4

Slide 4 text

LEARN HOW TO IMPROVE YOUR MIND A book I am currently reading

Slide 5

Slide 5 text

MEMORY TECHNIQUES

Slide 6

Slide 6 text

MEMORY TECHNIQUES • The book includes various techniques for improving memory

Slide 7

Slide 7 text

MEMORY TECHNIQUES • The book includes various techniques for improving memory • It helps to translate abstract data (the brain is weak with) into vivid images (a strength of the brain)

Slide 8

Slide 8 text

MEMORY TECHNIQUES • The book includes various techniques for improving memory • It helps to translate abstract data (the brain is weak with) into vivid images (a strength of the brain) • The Dominic System is used to translate numbers into characters and actions • It can help you memorize large numbers

Slide 9

Slide 9 text

SINGLE DIGITS

Slide 10

Slide 10 text

SINGLE DIGITS • Just count through the alphabet • But replace F with S • And skip to N when you hit 9

Slide 11

Slide 11 text

SINGLE DIGITS • Just count through the alphabet • But replace F with S • And skip to N when you hit 9 1 2 3 4 5 6 7 8 9 0 A B C D E S G H N O

Slide 12

Slide 12 text

DOUBLE DIGITS

Slide 13

Slide 13 text

DOUBLE DIGITS • Convert digit pairs into letters • 15 = AE, 23 = BC, 27 = BG, 39 = CN, 80 = HO, 80 = HA

Slide 14

Slide 14 text

Slide 15

Slide 15 text

DOUBLE DIGITS • Convert digit pairs into letters • 15 = AE, 23 = BC, 27 = BG, 39 = CN, 80 = HO, 80 = HA • Use the letters as initials for a character • AE = Albert Einstein, BG = Bill Gates, CN = Chuck Norris • Or use letters to indicate characters in other ways • BC = Jesus Christ, HO = Santa Claus, HA = Julia Sweeney

Slide 16

Slide 16 text

IMAGERY IS KEY

Slide 17

Slide 17 text

IMAGERY IS KEY • Each digit pair should create an image in your mind

Slide 18

Slide 18 text

IMAGERY IS KEY • Each digit pair should create an image in your mind • The image need to be a character doing their signature action • Chuck Norris performing a roundhouse kick • Santa Claus delivering presents • Julia Sweeney dressing asexually

Slide 19

Slide 19 text

FOUR DIGIT NUMBERS

Slide 20

Slide 20 text

FOUR DIGIT NUMBERS • To handle four digit numbers, use the ﬁrst two for the character and the second two as the action • 2339 = BCCN = Jesus Christ performing a roundhouse kick • 8081 = HOHA = Santa Claus dressing asexually

Slide 21

Slide 21 text

Slide 22

Slide 22 text

LARGER NUMBERS

Slide 23

Slide 23 text

LARGER NUMBERS • “Chunk” up bigger numbers into two and four digit groups

Slide 24

Slide 24 text

LARGER NUMBERS • “Chunk” up bigger numbers into two and four digit groups • Build wacky stories using those characters and actions • 27808039 = 2780 8039 = BGHO HOCN = Bill Gates started delivering presents, but this angered Santa Claus who gave him a roundhouse kick.

Slide 25

Slide 25 text

A CAST OF CHARACTERS

Slide 26

Slide 26 text

A CAST OF CHARACTERS http://ron.ludism.org/mnemonics_public.txt … 40 = DO = Fred Flintstone, breaking rocks in the quarry (Yabba dabba DOo!) 41 = DA = Douglas Adams, putting a Babelfish into his ear (_The Hitch Hiker's Guide to the Galaxy_) 42 = DB = PERSONAL 43 = DC = George Washington, chopping down cherry tree (Washington DC) 44 = DD = Danny Dunn (fictional character), descending in a bathysphere 45 = DE = PERSONAL 46 = DS = Ivan Stang (a.k.a. Doug Smith), ranting on a soapbox 47 = DG = PERSONAL 48 = DH = Douglas Hofstadter, recursively holding a picture of himself 49 = DN = Dan (_Roseanne_ TV show), wearing sunglasses to switch on Christmas lights …

Slide 27

Slide 27 text

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

BEANSTALKD (VIA STALKER)

Slide 30

Slide 30 text

BEANSTALKD

Slide 31

Slide 31 text

BEANSTALKD • A simple but highly effective message queue

Slide 32

Slide 32 text

BEANSTALKD • A simple but highly effective message queue • I’ve used it many times, including to do moves in Go vs Go

Slide 33

Slide 33 text

BEANSTALKD • A simple but highly effective message queue • I’ve used it many times, including to do moves in Go vs Go • It’s deﬁnitely one of my favorite tools for background jobs

Slide 34

Slide 34 text

CHARACTER LOOKUPS

Slide 35

Slide 35 text

CHARACTER LOOKUPS require "open-uri" def lookup_table return @lookup_table if defined? @lookup_table @lookup_table = { } open("http://ron.ludism.org/mnemonics_public.txt") do |page| page.each do |line| if line =~ /\A(\d{2})\s*=\s*[A-Z]{2}\s*=\s*(.+)/ @lookup_table[$1] = $2 elsif line =~ /\A\s+(.+)/ @lookup_table.values.last << " #{$1}" end end end @lookup_table end

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

ENCODING CHUNKS

Slide 39

Slide 39 text

ENCODING CHUNKS def encode(n, as = :both) if n.size == 4 and as == :both "#{encode(n[0..1], :character)} #{encode(n[2..3], :action)}." elsif as == :both "#{encode(n, :character)} #{encode(n, :action)}." elsif as == :character lookup_table[n][/\A\s*([^,]+(?:$[^)]+$)?)/, 1] elsif as == :action lookup_table[n][/\A\s*[^,]+(?:$[^)]+$)?,\s*(.+?)\s*\z/, 1] end end

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Slide 42

Slide 42 text

ENCODING NUMBERS

Slide 43

Slide 43 text

ENCODING NUMBERS job "number.encode" do |args| number = args["number"].to_s.delete("^0-9") number = "0#{number}" if (number.size % 2).nonzero? chunks = [4] * (number.size / 4) + [2] * (number.size % 4 / 2) encodings = [ ] chunks.permutation.to_a.uniq.each do |grouping| encodings << number.match(/\A#{grouping.map { |g| "(.{#{g}})" }.join}\z/) .captures .map { |n| encode(n) } .join(" ") .tr("_", '"') end Stalker.enqueue( "number.display", number: args["number"], encodings: encodings ) end

Slide 44

Slide 44 text

Slide 45

Slide 45 text

Slide 46

Slide 46 text

Slide 47

Slide 47 text

DISPLAYING ENCODINGS

Slide 48

Slide 48 text

DISPLAYING ENCODINGS job "number.display" do |args| Array(args["encodings"]).each do |encoded| puts "#{args['number']}:" puts encoded puts end end

Slide 49

Slide 49 text

QUEUING NUMBERS

Slide 50

Slide 50 text

QUEUING NUMBERS abort "USAGE: #{$PROGRAM_NAME} NUMBER" if ARGV.empty? require "stalker" Stalker.enqueue("number.encode", number: ARGV.shift)

Slide 51

Slide 51 text

RUNNING JOBS

Slide 52

Slide 52 text

RUNNING JOBS $ beanstalkd

Slide 53

Slide 53 text

RUNNING JOBS $ beanstalkd $ ruby encode_number.rb 405-285-0536

Slide 54

Slide 54 text

Slide 55

Slide 55 text

Slide 56

Slide 56 text

Slide 57

Slide 57 text

Slide 58

Slide 58 text

Slide 59

Slide 59 text

Slide 60

Slide 60 text

RUNNING JOBS $ beanstalkd $ ruby encode_number.rb 405-285-0536 $ rvm 1.9.2 $ RUBYLIB=. stalk jobs.rb number.encode [2010-12-08 13:07:24 -0600] Working 1 jobs: [ number.encode ] [2010-12-08 13:11:18 -0600] -> number.encode (number=405-285-0536) [2010-12-08 13:11:19 -0600] -> number.encode finished in 510ms $ RUBYLIB=. stalk jobs.rb number.display … 405-285-0536: Fred Flintstone breaking rocks in the quarry (Yabba dabba DOo!). Stuart Little (tiny fictional mouse wearing street clothes) releasing helium balloons (Newtonmas, HE HE HE!). J.R.R. Tolkien stepping into the Wardrobe. …

Slide 61

Slide 61 text

FOR MORE INFORMATION See Railscasts #243

Slide 62

Slide 62 text

FOR MORE INFORMATION See Railscasts #243

Slide 63

Slide 63 text

BEANSTALKD PROS

Slide 64

Slide 64 text

BEANSTALKD PROS • Extremely lightweight (especially if you drop the Rails stack)

Slide 65

Slide 65 text

BEANSTALKD PROS • Extremely lightweight (especially if you drop the Rails stack) • Ridiculously fast

Slide 66

Slide 66 text

BEANSTALKD PROS • Extremely lightweight (especially if you drop the Rails stack) • Ridiculously fast • Designed to be a queue

Slide 67

Slide 67 text

BEANSTALKD PROS • Extremely lightweight (especially if you drop the Rails stack) • Ridiculously fast • Designed to be a queue • “Tubes” make it trivial to divide work or pipeline jobs

Slide 68

Slide 68 text

BEANSTALKD PROS • Extremely lightweight (especially if you drop the Rails stack) • Ridiculously fast • Designed to be a queue • “Tubes” make it trivial to divide work or pipeline jobs • Really easy to spread across ruby versions and environments

Slide 69

Slide 69 text

BEANSTALKD CONS

Slide 70

Slide 70 text

BEANSTALKD CONS • Not ideal for durable queuing needs (though it does have a binlog)

Slide 71

Slide 71 text

BEANSTALKD CONS • Not ideal for durable queuing needs (though it does have a binlog) • Missing niceties like logging (though Stalker adds logging above beanstalkd)

Slide 72

Slide 72 text

BEANSTALKD CONS • Not ideal for durable queuing needs (though it does have a binlog) • Missing niceties like logging (though Stalker adds logging above beanstalkd) • Needs job introspection

Slide 73

Slide 73 text

No content

Slide 74

Slide 74 text

0MQ (AKA ZEROMQ OR ZMQ)

Slide 75

Slide 75 text

0MQ

Slide 76

Slide 76 text

0MQ • A high-level socket API over an asynchronous message queue • Request and reply, publish subscribe, and pipeline models

Slide 77

Slide 77 text

0MQ • A high-level socket API over an asynchronous message queue • Request and reply, publish subscribe, and pipeline models • This was my ﬁrst time using it (I was curious about it) • I don’t think it’s well suited to this task

Slide 78

Slide 78 text

THE END OF THE PIPELINE

Slide 79

Slide 79 text

THE END OF THE PIPELINE require "zmq" zmq = ZMQ::Context.new up = zmq.socket(ZMQ::UPSTREAM) up.bind("tcp://127.0.0.1:5000") while message = up.recv puts message end

Slide 80

Slide 80 text

THE END OF THE PIPELINE require "zmq" zmq = ZMQ::Context.new up = zmq.socket(ZMQ::UPSTREAM) up.bind("tcp://127.0.0.1:5000") while message = up.recv puts message end

Slide 81

Slide 81 text

THE END OF THE PIPELINE require "zmq" zmq = ZMQ::Context.new up = zmq.socket(ZMQ::UPSTREAM) up.bind("tcp://127.0.0.1:5000") while message = up.recv puts message end

Slide 82

Slide 82 text

THE END OF THE PIPELINE require "zmq" zmq = ZMQ::Context.new up = zmq.socket(ZMQ::UPSTREAM) up.bind("tcp://127.0.0.1:5000") while message = up.recv puts message end

Slide 83

Slide 83 text

NUMBER LOOKUPS

Slide 84

Slide 84 text

NUMBER LOOKUPS # same lookup_table() as before… def lookup lookup_table.each do |number, character_and_action| return number if yield character_and_action end nil end def character_number(msg) lookup { |ca| msg.start_with? ca[/\A\s*([^,]+(?:$[^)]+$)?)/, 1] } end def action_number(msg) lookup { |ca| msg.end_with? ca[/\A\s*[^,]+(?:$[^)]+$)?,\s*(.+?)\s*\z/, 1] } end

Slide 85

Slide 85 text

Slide 86

Slide 86 text

A DECODER

Slide 87

Slide 87 text

A DECODER def decode(message) message.to_s.strip.split(/\.(?:\s{2}|\n|\z)/).map { |chunk| chunk.gsub!(/\s+/, " ") character = character_number(chunk) action = action_number(chunk) character == action ? character : "#{character}#{action}" }.join end

Slide 88

Slide 88 text

THE MIDDLE PIPE

Slide 89

Slide 89 text

THE MIDDLE PIPE require "zmq" zmq = ZMQ::Context.new up = zmq.socket(ZMQ::UPSTREAM) down = zmq.socket(ZMQ::DOWNSTREAM) up.bind("tcp://127.0.0.1:5001") down.connect("tcp://127.0.0.1:5000") while message = up.recv down.send(decode(message)) end

Slide 90

Slide 90 text

Slide 91

Slide 91 text

Slide 92

Slide 92 text

Slide 93

Slide 93 text

THE PIPELINE OPENING

Slide 94

Slide 94 text

THE PIPELINE OPENING require "zmq" zmq = ZMQ::Context.new down = zmq.socket(ZMQ::DOWNSTREAM) down.connect("tcp://127.0.0.1:5001") down.send(ARGF.read)

Slide 95

Slide 95 text

A DATA PIPELINE IN ACTION

Slide 96

Slide 96 text

A DATA PIPELINE IN ACTION $ ruby queue_message.rb Fred Flintstone breaking rocks in the quarry (Yabba dabba DOo!). Stuart Little (tiny fictional mouse wearing street clothes) releasing helium balloons (Newtonmas, HE HE HE!). J.R.R. Tolkien stepping into the Wardrobe. ^d

Slide 97

Slide 97 text

Slide 98

Slide 98 text

Slide 99

Slide 99 text

0MQ PROS

Slide 100

Slide 100 text

0MQ PROS • Asynchronous messaging primitives for constructing networks

Slide 101

Slide 101 text

0MQ PROS • Asynchronous messaging primitives for constructing networks • Extremely versatile (can be made to ﬁt most any model)

Slide 102

Slide 102 text

0MQ PROS • Asynchronous messaging primitives for constructing networks • Extremely versatile (can be made to ﬁt most any model) • Supports many to many connections

Slide 103

Slide 103 text

0MQ PROS • Asynchronous messaging primitives for constructing networks • Extremely versatile (can be made to ﬁt most any model) • Supports many to many connections • Automatically load balanced

Slide 104

Slide 104 text

0MQ CONS

Slide 105

Slide 105 text

0MQ CONS • Very low-level (really just a platform to build on) • Missing all the conveniences: logging, message formats, etc.

Slide 106

Slide 106 text

0MQ CONS • Very low-level (really just a platform to build on) • Missing all the conveniences: logging, message formats, etc. • Best for asynchronous networks you design and build