Threads & JRuby: the simple alternative to evented

Slide 1

Slide 1 text

Threads & JRuby the simple alternative to evented david.copeland@livingsocial.com @davetron5000 www.naildrivin5.com www.awesomecommandlineapps.com Thursday, July 19, 12

Slide 2

Slide 2 text

{ :me => { :twitter => “@davetron5000”, :author => “Build Awesome Command-Line Applications in Ruby”, } } Thursday, July 19, 12

Slide 3

Slide 3 text

Thursday, July 19, 12

Slide 4

Slide 4 text

{ :me => { :twitter => “@davetron5000”, :author => “Build Awesome Command Line Applications in Ruby”, :techlead => “LivingSocial”, } } Thursday, July 19, 12

Slide 5

Slide 5 text

First Assignment Thursday, July 19, 12

Slide 6

Slide 6 text

Command Line App Write a Thursday, July 19, 12

Slide 7

Slide 7 text

Command Line App Write a To charge credit cards faster Thursday, July 19, 12

Slide 8

Slide 8 text

NO NEW HARDWARE Thursday, July 19, 12

Slide 9

Slide 9 text

Thursday, July 19, 12

Slide 10

Slide 10 text

THREADS! Thursday, July 19, 12

Slide 11

Slide 11 text

Are we maximizing our resources? Thursday, July 19, 12

Slide 12

Slide 12 text

Are we maximizing our resources? Thursday, July 19, 12

Slide 13

Slide 13 text

What problem are we solving? Thursday, July 19, 12

Slide 14

Slide 14 text

Do some I/O Compute Stuff Thursday, July 19, 12

Slide 15

Slide 15 text

Do some I/O Thursday, July 19, 12

Slide 16

Slide 16 text

Do some I/O Can I do I/O? Read/Write Block YES NO Thursday, July 19, 12

Slide 17

Slide 17 text

Do some I/O Block Thursday, July 19, 12

Slide 18

Slide 18 text

Block Thursday, July 19, 12

Slide 19

Slide 19 text

Let someone else work Block Thursday, July 19, 12

Slide 20

Slide 20 text

Maximize Resources Block Thursday, July 19, 12

Slide 21

Slide 21 text

Thursday, July 19, 12

Slide 22

Slide 22 text

Will our code be Thursday, July 19, 12

Slide 23

Slide 23 text

Will our code be easy to write? Thursday, July 19, 12

Slide 24

Slide 24 text

easy to understand? Will our code be easy to write? Thursday, July 19, 12

Slide 25

Slide 25 text

easy to understand? easy to test? Will our code be easy to write? Thursday, July 19, 12

Slide 26

Slide 26 text

Are we maximizing our resources? Are we maximizing our resources? Thursday, July 19, 12

Slide 27

Slide 27 text

Run lots of processes Thursday, July 19, 12

Slide 28

Slide 28 text

Thursday, July 19, 12

Slide 29

Slide 29 text

fork { # your code } Thursday, July 19, 12

Slide 30

Slide 30 text

fork { # your code } not available on JVM or in JavaScript Thursday, July 19, 12

Slide 31

Slide 31 text

Thursday, July 19, 12

Slide 32

Slide 32 text

for i in {1..10} do ./your_app done Thursday, July 19, 12

Slide 33

Slide 33 text

easy to understand Thursday, July 19, 12

Slide 34

Slide 34 text

doesn’t maximize resources Thursday, July 19, 12

Slide 35

Slide 35 text

Parent Thursday, July 19, 12

Slide 36

Slide 36 text

Parent Parent’s Memory Thursday, July 19, 12

Slide 37

Slide 37 text

Parent Parent’s Memory Child fork { Parent’s Memory (copy of) Thursday, July 19, 12

Slide 38

Slide 38 text

Event-based I/O Thursday, July 19, 12

Slide 39

Slide 39 text

var http = require('http'); http.createServer(function (req, res) { res.writeHead(200, {'Content-Type': 'text/plain'}); res.end('Hello World\n'); }).listen(1337, '127.0.0.1'); Thursday, July 19, 12

Slide 40

Slide 40 text

// bring in special classes and setup eventedThing.whenIOIsReady(function(results) { // do something with results }).startEventLoop(); Thursday, July 19, 12

Slide 41

Slide 41 text

Need to do I/O again? Thursday, July 19, 12

Slide 42

Slide 42 text

// bring in special classes and setup eventedThing.whenIOIsReady(function(results) { // do something with results otherEventedThing.whenMoreIOIsReady(function(moreResults) { // do something with more I/O }); }).startEventLoop(); Thursday, July 19, 12

Slide 43

Slide 43 text

Slide 44

Slide 44 text

public class EchoServer { public static void main(String[] args) throws Exception { ServerBootstrap bootstrap = new ServerBootstrap( new NioServerSocketChannelFactory( Executors.newCachedThreadPool(), Executors.newCachedThreadPool())); bootstrap.setPipelineFactory(new ChannelPipelineFactory() { public ChannelPipeline getPipeline() throws Exception { return Channels.pipeline(new EchoServerHandler()); } }); bootstrap.bind(new InetSocketAddress(8080)); } } Thursday, July 19, 12

Slide 45

Slide 45 text

Slide 46

Slide 46 text

public class EchoServerHandler extends SimpleChannelUpstreamHandler { @Override public void messageReceived(ChannelHandlerContext ctx, MessageEvent e) { e.getChannel().write(e.getMessage()); } @Override public void exceptionCaught(ChannelHandlerContext ctx, ExceptionEvent e) { e.getChannel().close(); } } Thursday, July 19, 12

Slide 47

Slide 47 text

NOT easy to understand Thursday, July 19, 12

Slide 48

Slide 48 text

kinda maximizes resources Thursday, July 19, 12

Slide 49

Slide 49 text

Entire call chain must be evented Thursday, July 19, 12

Slide 50

Slide 50 text

Not true parallelism Thursday, July 19, 12

Slide 51

Slide 51 text

Not true parallelism unless you run lots of processes Thursday, July 19, 12

Slide 52

Slide 52 text

NOT easy to understand kinda maximizes resources Thursday, July 19, 12

Slide 53

Slide 53 text

Threads Thursday, July 19, 12

Slide 54

Slide 54 text

Thursday, July 19, 12

Slide 55

Slide 55 text

Thread.new { # your code } Thursday, July 19, 12

Slide 56

Slide 56 text

fork { # your code } Thursday, July 19, 12

Slide 57

Slide 57 text

Thread.new { # your code } Thursday, July 19, 12

Slide 58

Slide 58 text

new Thread(new Runnable() { public void run() { // your code } }).start(); Thursday, July 19, 12

Slide 59

Slide 59 text

Need to do I/O? Thursday, July 19, 12

Slide 60

Slide 60 text

Need to do I/O? Not a problem Thursday, July 19, 12

Slide 61

Slide 61 text

Need to do I/O THREE TIMES? Thursday, July 19, 12

Slide 62

Slide 62 text

Need to do I/O THREE TIMES? Not a problem Thursday, July 19, 12

Slide 63

Slide 63 text

What if I need to calculate π to the 2,345,123rd decimal place? Thursday, July 19, 12

Slide 64

Slide 64 text

NOT A PROBLEM Thursday, July 19, 12

Slide 65

Slide 65 text

Threads achieve true parallelism Thursday, July 19, 12

Slide 66

Slide 66 text

easy to understand Thursday, July 19, 12

Slide 67

Slide 67 text

maximizes resources Thursday, July 19, 12

Slide 68

Slide 68 text

JRuby Thursday, July 19, 12

Slide 69

Slide 69 text

Ruby runtime… Thursday, July 19, 12

Slide 70

Slide 70 text

Ruby runtime… …running on the JVM Thursday, July 19, 12

Slide 71

Slide 71 text

new Thread(new Runnable() { public void run() { List people = queryPeople(); for (Person p: people) { if (p.getAge() < 18) { // do one thing for minors } else { // do another thing for adults } } } }).start(); Thursday, July 19, 12

Slide 72

Slide 72 text

Thread.new { query_people.each do |p| if p.get_age < 18 # do one thing for minors else # do another thing for adults end end } Thursday, July 19, 12

Slide 73

Slide 73 text

Thread.new { query_people.each do |p| if p.get_age < 18 # do one thing for minors else # do another thing for adults end end } Thursday, July 19, 12

Slide 74

Slide 74 text

Thread.new { query_people.each do |p| if p.get_age < 18 # do one thing for minors else # do another thing for adults end end } Thursday, July 19, 12

Slide 75

Slide 75 text

Fast, powerful dynamic language… Thursday, July 19, 12

Slide 76

Slide 76 text

Fast, powerful dynamic language… …full power of the JVM Thursday, July 19, 12

Slide 77

Slide 77 text

Other Rubies? MRI 1.8 Green Threads No Parallelism MRI 1.9 Real Threads No Parallelism Rubinius Real Threads, no GIL True Parallelism Thursday, July 19, 12

Slide 78

Slide 78 text

Other Rubies? MRI 1.8 Green Threads No Parallelism MRI 1.9 Real Threads No Parallelism Rubinius Real Threads, no GIL True Parallelism Thursday, July 19, 12

Slide 79

Slide 79 text

So, you’ve decided to use Threads… Thursday, July 19, 12

Slide 80

Slide 80 text

You need to know four things Thursday, July 19, 12

Slide 81

Slide 81 text

Thursday, July 19, 12

Slide 82

Slide 82 text

Start & Manage Thursday, July 19, 12

Slide 83

Slide 83 text

Start & Manage (Dealing with) Shared State Thursday, July 19, 12

Slide 84

Slide 84 text

Start & Manage (Dealing with) Shared State Using Third Party Libraries Thursday, July 19, 12

Slide 85

Slide 85 text

Start & Manage (Dealing with) Shared State Using Third Party Libraries Context Switching Thursday, July 19, 12

Slide 86

Slide 86 text

Start & Manage Threads Thursday, July 19, 12

Slide 87

Slide 87 text

Chaos Thursday, July 19, 12

Slide 88

Slide 88 text

Thread.new { # your code } Thread.new { # moar code } # etc Thursday, July 19, 12

Slide 89

Slide 89 text

Hand-Roll Thursday, July 19, 12

Slide 90

Slide 90 text

threads = [] threads << Thread.new { # your code } threads << Thread.new { # moar code } # etc threads.each(&:join) # All threads have completed exit 0 Thursday, July 19, 12

Slide 91

Slide 91 text

java.util.concurrent Thursday, July 19, 12

Slide 92

Slide 92 text

java_import java.util.concurrent service = Executors.new_fixed_thread_pool(100) service.execute { # your code } service.execute { # some other code } service.shutdown service.await_termination(10,SECONDS) service.shutdown_now Thursday, July 19, 12

Slide 93

Slide 93 text

Slide 94

Slide 94 text

Slide 95

Slide 95 text

Slide 96

Slide 96 text

Slide 97

Slide 97 text

Slide 98

Slide 98 text

service = Executors.new_fixed_thread_pool(10) tcp_server = TCPServer.new("127.0.0.1",8080) Signal.trap('SIGINT') { service.shutdown } loop do { s = tcp_server.accept service.execute { s.puts calculate_pi() s.close } break if service.is_shutdown } service.await_termination(10,TimeUnit.SECONDS) service.shutdown_now Thursday, July 19, 12

Slide 99

Slide 99 text

So much more Thursday, July 19, 12

Slide 100

Slide 100 text

ScheduledExecutorService So much more Thursday, July 19, 12

Slide 101

Slide 101 text

ScheduledExecutorService So much more ThreadFactory Thursday, July 19, 12

Slide 102

Slide 102 text

Read the javadocs ScheduledExecutorService So much more ThreadFactory Thursday, July 19, 12

Slide 103

Slide 103 text

Shared State Thursday, July 19, 12

Slide 104

Slide 104 text

results = [] Thread.new { results << some_result() } Thread.new { results << other_result() } Thursday, July 19, 12

Slide 105

Slide 105 text

mutex = Mutex.new results = [] Thread.new { mutex.synchronize { results << some_result() } } Thread.new { mutex.synchronize { results << other_result() } } Thursday, July 19, 12

Slide 106

Slide 106 text

Avoid explicit locking/sync Thursday, July 19, 12

Slide 107

Slide 107 text

results = ConcurrentLinkedQueue.new Thread.new { results.offer(some_result()) } Thread.new { results.offer(some_result()) } Thursday, July 19, 12

Slide 108

Slide 108 text

So much more Thursday, July 19, 12

Slide 109

Slide 109 text

ConcurrentHashMap So much more Thursday, July 19, 12

Slide 110

Slide 110 text

ConcurrentHashMap So much more AtomicReference Thursday, July 19, 12

Slide 111

Slide 111 text

Read the javadocs ConcurrentHashMap So much more AtomicReference Thursday, July 19, 12

Slide 112

Slide 112 text

Third Party Libraries Thursday, July 19, 12

Slide 113

Slide 113 text

Are they thread safe? Thursday, July 19, 12

Slide 114

Slide 114 text

Probably Thursday, July 19, 12

Slide 115

Slide 115 text

Probably …but to be sure Thursday, July 19, 12

Slide 116

Slide 116 text

variables Thursday, July 19, 12

Slide 117

Slide 117 text

variables Global Thursday, July 19, 12

Slide 118

Slide 118 text

variables Global Class Thursday, July 19, 12

Slide 119

Slide 119 text

Most are fine Thursday, July 19, 12

Slide 120

Slide 120 text

Context Switching Thursday, July 19, 12

Slide 121

Slide 121 text

Amdahl’s Law Thursday, July 19, 12

Slide 122

Slide 122 text

Performance Gains Increase in # of Threads Speedup Thursday, July 19, 12

Slide 123

Slide 123 text

Performance Gains with Context Switching Increase in # of Threads Speedup Cost of Context Switching Takes Over Thursday, July 19, 12

Slide 124

Slide 124 text

Thread Pools Thursday, July 19, 12

Slide 125

Slide 125 text

Your Code Your Code Your Code Your Code Thread Thread Thread Thread Pool Your Code Your Code Your Code Your Code Thursday, July 19, 12

Slide 126

Slide 126 text

Your Code Your Code Your Code Your Code Thread Thread Thread Thread Pool Your Code Your Code Your Code Your Code Thursday, July 19, 12

Slide 127

Slide 127 text

Your Code Your Code Your Code Your Code Thread Thread Thread Thread Pool Your Code Your Code Your Code Thursday, July 19, 12

Slide 128

Slide 128 text

Thread.new { # your code } Thursday, July 19, 12

Slide 129

Slide 129 text

service.execute { # your code } Thursday, July 19, 12

Slide 130

Slide 130 text

Not usually a concern Thursday, July 19, 12

Slide 131

Slide 131 text

Always use Threads, right? Thursday, July 19, 12

Slide 132

Slide 132 text

Thread UNsafe Libraries Thursday, July 19, 12

Slide 133

Slide 133 text

Evented Thursday, July 19, 12

Slide 134

Slide 134 text

Cannot use JRuby/JVM Thursday, July 19, 12

Slide 135

Slide 135 text

Evented Thursday, July 19, 12

Slide 136

Slide 136 text

Otherwise, Threads will simplify your code and maximize your resources Thursday, July 19, 12

Slide 137

Slide 137 text

Ask “why shouldn’t I use threads?” Thursday, July 19, 12

Slide 138

Slide 138 text

Exercises Thursday, July 19, 12

Slide 139

Slide 139 text

Echo Server •Listen on a port •Respond to each request in a new Thread •Extra Credit: Record stats on requests in a shared data structure Thursday, July 19, 12

Slide 140

Slide 140 text

Connection Pool •Allow N clients to access X shared instances of, say, Redis (where N > X) •Clients “check out” a connection and get exclusive access •Clients “check in” when done •Instances get re-used Thursday, July 19, 12

Slide 141

Slide 141 text

THANKS! david.copeland@livingsocial.com @davetron5000 www.naildrivin5.com www.awesomecommandlineapps.com Thursday, July 19, 12

Slide 142

Slide 142 text

THANKS! david.copeland@livingsocial.com @davetron5000 www.naildrivin5.com www.awesomecommandlineapps.com We’re Hiring! Thursday, July 19, 12