Slide 1

Slide 1 text

Framework Development

Slide 2

Slide 2 text

Application Development

Slide 3

Slide 3 text

Spectrum

Slide 4

Slide 4 text

Spectrum First App

Slide 5

Slide 5 text

Spectrum First App Rails

Slide 6

Slide 6 text

Spectrum First App Procore Rails

Slide 7

Slide 7 text

Spectrum First App Procore Strobe Rails

Slide 8

Slide 8 text

Apps Often Include Frameworks

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

The difference between app and framework development is actually a state of mind.

Slide 11

Slide 11 text

My Code

Slide 12

Slide 12 text

Upstream My Code

Slide 13

Slide 13 text

Upstream My Code Downstream

Slide 14

Slide 14 text

Upstream My Code App Development

Slide 15

Slide 15 text

Upstream My Code Downstream Framework Development

Slide 16

Slide 16 text

Upstream

Slide 17

Slide 17 text

Understand Your Upstream

Slide 18

Slide 18 text

Ruby Rails Downstream Framework Development

Slide 19

Slide 19 text

Ruby Rails Downstream Framework Development When doing framework development, go beyond the abstractions presented.

Slide 20

Slide 20 text

Ruby Threads

Slide 21

Slide 21 text

"Threads Are Slow"

Slide 22

Slide 22 text

"Events Are Hot"

Slide 23

Slide 23 text

"YUNO Use Fibers"

Slide 24

Slide 24 text

"YUNO Use Goliath"

Slide 25

Slide 25 text

def hello(y) y += 1 sleep y puts y end x = 1 hello(x) puts "done"

Slide 26

Slide 26 text

__script__ def hello(y) y += 1 sleep y puts y end x = 1 hello(x) puts "done" Name Value x

Slide 27

Slide 27 text

__script__ Name Value x def hello(y) y += 1 sleep y puts y end x = 1 hello(x) puts "done"

Slide 28

Slide 28 text

__script__ Name Value x 1 def hello(y) y += 1 sleep y puts y end x = 1 hello(x) puts "done"

Slide 29

Slide 29 text

hello __script__ Name Value y 1 def hello(y) y += 1 sleep y puts y end x = 1 hello(x) puts "done"

Slide 30

Slide 30 text

hello __script__ Name Value y 2 def hello(y) y += 1 sleep y puts y end x = 1 hello(x) puts "done"

Slide 31

Slide 31 text

hello __script__ Name Value y 2 def hello(y) y += 1 sleep y puts y end x = 1 hello(x) puts "done" Name Value x 1 "the stack"

Slide 32

Slide 32 text

Scheduler "resume this in 2 seconds" hello __script__ Name Value y 2 def hello(y) y += 1 sleep y puts y end x = 1 hello(x) puts "done" Name Value x 1

Slide 33

Slide 33 text

hello __script__ Name Value y 2 def hello(y) y += 1 sleep y puts y end x = 1 hello(x) puts "done"

Slide 34

Slide 34 text

__script__ def hello(y) y += 1 sleep y puts y end x = 1 hello(x) puts "done" Name Value x 1

Slide 35

Slide 35 text

def hello(y) y += 1 sleep y puts y end x = 1 hello(x) puts "done"

Slide 36

Slide 36 text

Stacks Give Us Stack Traces

Slide 37

Slide 37 text

Stacks Give Us Deferred Return

Slide 38

Slide 38 text

__SCRIPT__ def read(file) File.read(file) end file = "README" body = read(file) puts body Name Value file body

Slide 39

Slide 39 text

__SCRIPT__ def read(file) File.read(file) end file = "README" body = read(file) puts body Name Value file "README" body

Slide 40

Slide 40 text

read __SCRIPT__ def read(file) File.read(file) end file = "README" body = read(file) puts body Name Value file "README"

Slide 41

Slide 41 text

read __SCRIPT__ def read(file) File.read(file) end file = "README" body = read(file) puts body Name Value file "README" Name Value file "README" body

Slide 42

Slide 42 text

Scheduler "resume this when the file is ready to read" read __SCRIPT__ def read(file) File.read(file) end file = "README" body = read(file) puts body Name Value file "README" Name Value file "README" body

Slide 43

Slide 43 text

Scheduler "resume this when the file is ready to read" read __SCRIPT__ def read(file) File.read(file) end file = "README" body = read(file) puts body Name Value file "README" Name Value file "README" body The File.read method takes care of returning the actual value once the thread resumes.

Slide 44

Slide 44 text

read __SCRIPT__ def read(file) File.read(file) end file = "README" body = read(file) puts body Name Value file "README"

Slide 45

Slide 45 text

__SCRIPT__ def read(file) File.read(file) end file = "README" body = read(file) puts body Name Value file "README" body <file text>

Slide 46

Slide 46 text

def read(file) File.read(file) end file = "README" body = read(file) puts body

Slide 47

Slide 47 text

Scheduler

Slide 48

Slide 48 text

Manages Multiple Stacks

Slide 49

Slide 49 text

def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__

Slide 50

Slide 50 text

__SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value main

Slide 51

Slide 51 text

__SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ main t1

Slide 52

Slide 52 text

Scheduler "resume this when you have a second" def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ t1 Name Value d "/Users"

Slide 53

Slide 53 text

__SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ main t1 __SCRIPT__ t2

Slide 54

Slide 54 text

Scheduler "resume this when you have a second" def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ t2 Name Value d "/Users"

Slide 55

Slide 55 text

Scheduler def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ t2 Name Value d "/Users" def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ t1 Name Value d "/Users" __SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value main Name Value d "/Users" active thread inactive threads

Slide 56

Slide 56 text

__SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ main t1 __SCRIPT__ t2

Slide 57

Slide 57 text

Scheduler "resume this when t1 finishes" __SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value main Name Value d "/Users"

Slide 58

Slide 58 text

Scheduler __SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value main Name Value d "/Users" active thread inactive threads def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ t1 Name Value d "/Users" def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ t2 Name Value d "/Users"

Slide 59

Slide 59 text

Scheduler def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ t1 Name Value d "/Users" active thread inactive threads def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ t2 Name Value d "/Users" __SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value main Name Value d "/Users"

Slide 60

Slide 60 text

__SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ main t1 __SCRIPT__ t2

Slide 61

Slide 61 text

__SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ main t1 __SCRIPT__ t2

Slide 62

Slide 62 text

__SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value read __SCRIPT__ main t1 __SCRIPT__ t2

Slide 63

Slide 63 text

Scheduler "resume this when the file is ready to read" def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value read __SCRIPT__ t1 Name Value d "/Users" Name Value file "/Users/ README"

Slide 64

Slide 64 text

Scheduler active thread inactive threads def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value __SCRIPT__ t2 Name Value d "/Users" __SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value main Name Value d "/Users" def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value read __SCRIPT__ t1 Name Value d "/Users" Name Value file "/Users/ README"

Slide 65

Slide 65 text

__SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value read __SCRIPT__ main t1 __SCRIPT__ t2

Slide 66

Slide 66 text

__SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value read __SCRIPT__ main t1 __SCRIPT__ t2

Slide 67

Slide 67 text

__SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value read __SCRIPT__ main t1 read __SCRIPT__ t2

Slide 68

Slide 68 text

Scheduler "resume this when the file is ready to read" def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value body __SCRIPT__ t2 Name Value d "/Users" Name Value file "/Users/ README"

Slide 69

Slide 69 text

Scheduler inactive threads __SCRIPT__ def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value main Name Value d "/Users" def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value body __SCRIPT__ t2 Name Value d "/Users" Name Value file "/Users/ README" def read(file) File.read(file) end d = "/Users" t1 = Thread.new do f = "#{d}/README" read(file) end t2 = Thread.new do f = "#{d}/.vimrc" read(file) end puts t1.value puts t2.value read __SCRIPT__ t1 Name Value d "/Users" Name Value file "/Users/ README" waiting for IO to be ready waiting for t1 to finish waiting for IO to be ready

Slide 70

Slide 70 text

Threads Abstract Callback Spaghetti

Slide 71

Slide 71 text

What's in That Stack State?

Slide 72

Slide 72 text

What's in That Stack State? Remember that the stack gives us debuggers, backtraces, and deferred return

Slide 73

Slide 73 text

Fibers Are Small Threads

Slide 74

Slide 74 text

Without Automatic I/O Rules

Slide 75

Slide 75 text

BYOS

Slide 76

Slide 76 text

Bring Your Own Scheduler

Slide 77

Slide 77 text

Threads: 8MB Fibers: 4kb Callbacks: 1 frame

Slide 78

Slide 78 text

In Ruby: Threads: 8MB Fibers: 4kb Callbacks: 1 frame

Slide 79

Slide 79 text

In Node: Callbacks: 1 frame

Slide 80

Slide 80 text

Where Was I?

Slide 81

Slide 81 text

Upstream My Code App Development

Slide 82

Slide 82 text

Upstream My Code App Development As an app developer, use the abstractions provided. Don't try to swap out parts of the stack 'for performance' or any other reason without UNDERSTANDING it.

Slide 83

Slide 83 text

Bug in Upstream

Slide 84

Slide 84 text

Upstream My Code App Development report the bug it's not my problem

Slide 85

Slide 85 text

Framework Developer

Slide 86

Slide 86 text

Framework Developer

Slide 87

Slide 87 text

Framework Developer

Slide 88

Slide 88 text

If You Want Better Upstream

Slide 89

Slide 89 text

Upstream My Code App Development wait complain

Slide 90

Slide 90 text

Framework Development

Slide 91

Slide 91 text

Framework Development

Slide 92

Slide 92 text

Your Own Code

Slide 93

Slide 93 text

Learn By Reading Similar Code

Slide 94

Slide 94 text

CakePHP: copycat

Slide 95

Slide 95 text

CakePHP: copycat Worth skimming to find useful mutations in the copy.

Slide 96

Slide 96 text

CakePHP: copycat Struts: different

Slide 97

Slide 97 text

CakePHP: copycat Struts: different Worth skimming to find useful approaches that can be merged.

Slide 98

Slide 98 text

CakePHP: copycat Struts: different Django: similar

Slide 99

Slide 99 text

CakePHP: copycat Struts: different Django: similar Worth reading in depth. Form Models are very interesting.

Slide 100

Slide 100 text

Downstream

Slide 101

Slide 101 text

Downstream Package up your code for other people who will look at you as *their* upstream.

Slide 102

Slide 102 text

Interfaces

Slide 103

Slide 103 text

Downstream Interfaces ActiveModel ActionPack

Slide 104

Slide 104 text

Downstream Interfaces ActiveModel ActionPack Rails Users

Slide 105

Slide 105 text

App Developer Your App

Slide 106

Slide 106 text

Framework Developer Your App Your Tests

Slide 107

Slide 107 text

Framework Developer Your App Your Tests

Slide 108

Slide 108 text

Framework Developer Internal Libraries Your App Your Tests

Slide 109

Slide 109 text

Encapsulate Interfaces

Slide 110

Slide 110 text

No content

Slide 111

Slide 111 text

Where are the Boundaries?

Slide 112

Slide 112 text

Where are the Boundaries? Corey Haines talked about this a bit yesterday.

Slide 113

Slide 113 text

Framework Development is All About Boundaries

Slide 114

Slide 114 text

Coupling Django Style

Slide 115

Slide 115 text

“ Mark Ramm Turbogears I’m hard on the Django folks here because I think the “tightly integrated/loosely coupled” buzz phrase is actually detrimental to understanding how the trade-off’s work. And there are trade-offs and those trade-offs mean that there isn’t and will never be one perfect web-framework which somehow magically isn’t subject to the down-side of any of the constraints and design trade-offs that we all have to deal with every day.

Slide 116

Slide 116 text

If You Isolate Concerns, Define Interfaces

Slide 117

Slide 117 text

If You Isolate Concerns, Define Interfaces Specifically, think about how the isolated bits communicate with each other

Slide 118

Slide 118 text

What's the Escape Velocity?

Slide 119

Slide 119 text

What's the Escape Velocity? Complexity is pushed up the abstraction chain. Single systems shove it all into one place. Isolating concerns moves it to the links between the concerns. Too many isolated concerns at the same level of abstraction reintroduce complexity.

Slide 120

Slide 120 text

big complex program

Slide 121

Slide 121 text

No content

Slide 122

Slide 122 text

Now You Have 2 Problems

Slide 123

Slide 123 text

No content

Slide 124

Slide 124 text

The Tradeoff is Thinking About the Arrows

Slide 125

Slide 125 text

Framework Developer ActiveModel ActionPack Rails Users ?

Slide 126

Slide 126 text

Framework Developer Libraries Your App Your Tests

Slide 127

Slide 127 text

Packaging for Downstream

Slide 128

Slide 128 text

Document Interfaces

Slide 129

Slide 129 text

Internally

Slide 130

Slide 130 text

No content

Slide 131

Slide 131 text

Publicly

Slide 132

Slide 132 text

No content

Slide 133

Slide 133 text

You Should

Slide 134

Slide 134 text

Pick a Side, We're at War

Slide 135

Slide 135 text

Pick a Side, We're at War App Developer Framework Developer Upstream Pure Abstraction Participant Your Code You Live Here Learn from Similar Codebases Downstream N/A Package Abstractions and Interfaces

Slide 136

Slide 136 text

You Should: Go All the Way

Slide 137

Slide 137 text

͋Γ͕ͱ͏ʂ @wycats profiles.google.com/wycats http://yehudakatz.com

Slide 138

Slide 138 text

Questions?