Price Aware Scheduling - Speaker Deck

Slide 1

Slide 1 text

G PRICE AWARE SCHEDULING

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

task

Slide 4

Slide 4 text

task 12 hours

Slide 5

Slide 5 text

2 x cpu task 12 hours

Slide 6

Slide 6 text

2 x cpu task 12 hours 128 gb disk

Slide 7

Slide 7 text

2 x cpu task 4gb memory 12 hours 128 gb disk

Slide 8

Slide 8 text

2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours

Slide 9

Slide 9 text

2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?

Slide 10

Slide 10 text

resource

Slide 11

Slide 11 text

resource 4 x cpu

Slide 12

Slide 12 text

resource 4 x cpu 8gb memory

Slide 13

Slide 13 text

resource 4 x cpu 240 gb disk 8gb memory

Slide 14

Slide 14 text

resource $0.40 / hr 4 x cpu 240 gb disk 8gb memory

Slide 15

Slide 15 text

resource $0.40 / hr 4 x cpu 240 gb disk 1Gb network 8gb memory

Slide 16

Slide 16 text

resource $0.40 / hr 4 x cpu 240 gb disk 1Gb network 4m boot-time 8gb memory

Slide 17

Slide 17 text

scheduler

Slide 18

Slide 18 text

scheduler

Slide 19

Slide 19 text

heterogeneous tasks

Slide 20

Slide 20 text

heterogeneous resources

Slide 21

Slide 21 text

access to large pool of resources

Slide 22

Slide 22 text

pay only when used

Slide 23

Slide 23 text

variable price

Slide 24

Slide 24 text

variable availability

Slide 25

Slide 25 text

coarse grained billing unit

Slide 26

Slide 26 text

control scaling of resources and scheduling of tasks for optimal cost within required sla

Slide 27

Slide 27 text

control scaling of resources and scheduling of tasks for optimal cost within required sla

Slide 28

Slide 28 text

control scaling of resources and scheduling of tasks for optimal cost within required sla

Slide 29

Slide 29 text

control scaling of resources and scheduling of tasks for optimal cost within required sla

Slide 30

Slide 30 text

WHY DO I CARE?

Slide 31

Slide 31 text

“We are in the business of turning technical debt into monetary debt” - Old Ambiata Proverb

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

ONE Resource Utilisation

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

knapsack problem

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

$1 $7 $11

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

meet constraints

Slide 43

Slide 43 text

maximise $$$ meet constraints

Slide 44

Slide 44 text

bin-packing problem

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

1cpu 8cpu 2cpu

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

1cpu 8cpu 2cpu

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

meet constraints

Slide 53

Slide 53 text

meet constraints maximise utilisation

Slide 54

Slide 54 text

np-hard (with ptas)

Slide 55

Slide 55 text

vector-packing problem

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

No content

Slide 58

Slide 58 text

1cpu, 4gb 8cpu, 16gb 2cpu, 1gb

Slide 59

Slide 59 text

No content

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

meet all constraints

Slide 62

Slide 62 text

meet all constraints maximise utilisation

Slide 63

Slide 63 text

meet all constraints balance all dimensions maximise utilisation

Slide 64

Slide 64 text

vector-scheduling problem

Slide 65

Slide 65 text

meet all constraints balance all dimensions minimise makespan

Slide 66

Slide 66 text

WHY NOT? SO

Slide 67

Slide 67 text

No content

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

no fixed resources

Slide 70

Slide 70 text

variable capability means i can fit resources to tasks rather than tasks to resources

Slide 71

Slide 71 text

TWO Auto Scaling

Slide 72

Slide 72 text

how do we combine auto scaling with scheduling?

Slide 73

Slide 73 text

few public or genuine attempts

Slide 74

Slide 74 text

No content

Slide 75

Slide 75 text

Slide 76

Slide 76 text

trade urgency off against fitness

Slide 77

Slide 77 text

trade urgency off against fitness yet another packing problem

Slide 78

Slide 78 text

No content

Slide 79

Slide 79 text

decouple scaling and scheduling questions

Slide 80

Slide 80 text

No content

Slide 81

Slide 81 text

trigger scaling based on queue

Slide 82

Slide 82 text

how not to do it

Slide 83

Slide 83 text

packing very important for scale down

Slide 84

Slide 84 text

scheduler can’t scale down

Slide 85

Slide 85 text

2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?

Slide 86

Slide 86 text

2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?

Slide 87

Slide 87 text

WHY NOT? SO

Slide 88

Slide 88 text

No content

Slide 89

Slide 89 text

No content

Slide 90

Slide 90 text

optimised for latency

Slide 91

Slide 91 text

not optimised for cost

Slide 92

Slide 92 text

doesn’t / can’t leverage knowledge specific to our workloads

Slide 93

Slide 93 text

dependence on co-ordination

Slide 94

Slide 94 text

urgency is a stretch

Slide 95

Slide 95 text

THREE Cost Optimisation

Slide 96

Slide 96 text

2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?

Slide 97

Slide 97 text

2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?

Slide 98

Slide 98 text

2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?

Slide 99

Slide 99 text

2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?

Slide 100

Slide 100 text

resource $0.40 / hr 4 x cpu 240 gb disk 1Gb network 4m boot-time 8gb memory

Slide 101

Slide 101 text

resource $0.40 / hr 4 x cpu 240 gb disk 1Gb network 4m boot-time 8gb memory

Slide 102

Slide 102 text

resource $0.04 / hr 4 x cpu 240 gb disk 1Gb network 4m boot-time 8gb memory

Slide 103

Slide 103 text

No content

Slide 104

Slide 104 text

No content

Slide 105

Slide 105 text

FOUR The Scheduler

Slide 106

Slide 106 text

2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?

Slide 107

Slide 107 text

1 newtype TaskId = 2 TaskId Text 3 4 data Timeliness = 5 Interactive 6 | Lazy 7 | FinishBy UTCTime 8 9 data Requirement = 10 Requirement { 11 memory :: Memory 12 , disk :: Disk 13 , network :: Network 14 , timeliness :: Timeliness 15 } deriving (Eq, Show) 16 17 data Task = 18 Task { 19 taskId :: !TaskId 20 , taskRequirement :: !Requirement 21 } deriving (Eq, Show)

Slide 108

Slide 108 text

No content

Slide 109

Slide 109 text

1 newtype ResourceId = 2 ResourceId Text 3 4 data Resource = 5 Resource { 6 memory :: Memory 7 , disk :: Disk 8 , network :: Network 9 , cost :: Price 10 } 11 12 data Allocation = 13 Allocated ResourceId Task 14 | Unallocated ResourceId 15 16 data Capability = 17 Capability { 18 capabilityResource :: Resource 19 , capabilityAllocated :: [Allocation] 20 }

Slide 110

Slide 110 text

1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision] 2 submit now capabiities tasks = 3 schedule now capabiities <$> 4 sortBy (compare `on` urgency now) tasks

Slide 111

Slide 111 text

first-fit decreasing

Slide 112

Slide 112 text

1 urgency :: UTCTime -> Task -> Double 2 urgency now t = 3 case timelinessOf t of 4 Interactive -> 5 1.0 6 Lazy -> 7 0.0 8 FinishBy when -> 9 distributionOver (statisticsOf t) when

Slide 113

Slide 113 text

99th percentile 1.0 0.o

Slide 114

Slide 114 text

1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision] 2 submit now capabiities tasks = 3 schedule now capabiities <$> 4 sortBy (compare `on` urgency now) tasks

Slide 115

Slide 115 text

1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision] 2 submit now capabiities tasks = 3 schedule now capabiities <$> 4 sortBy (compare `on` urgency now) tasks 5 6 schedule :: UTCTime -> [Capability] -> Task -> Decision 7 schedule now capabilities task = 8 decide $ 9 evaluate now task <$> capabilities

Slide 116

Slide 116 text

1 data Score = 2 Free Resource [Double] 3 | Scale Capability [Double] 4 | Delay Capability [Double] 5 6 data Scores = 7 Scores Task [Score] 8 9 evaluate :: UTCTime -> Task -> Capability -> Scores 10 evaluate now task capability = 11 merge [ 12 evaluateFree now task capability 13 , evaluateRunning now task capability 14 , evaluateScaled now task capability 15 ] 16 17 evaluateFree :: UTCTime -> Task -> Capability -> Scores 18 19 evaluateRunning :: UTCTime -> Task -> Capability -> Scores 20 21 evaluateScaled :: UTCTime -> Task -> Capability -> Scores

Slide 117

Slide 117 text

1 memory resource allocated required = 2 max 0 $ required / resource - allocated 3 4 cpu resource allocated required = 5 max 0 $ required / resource - allocated 6 7 network resource allocated required = 8 max 0 $ required / resource - allocated 9 10 timeliness remaining average variance = 11 (average + variance) / remaining

Slide 118

Slide 118 text

Slide 119

Slide 119 text

1 data Decision = 2 Decision Task Outcome 3 4 data Outcome = 5 Assign Resource 6 | Delay Resource 7 | Scale Capability 8 | Preempt Capability 9 | Reject 10 11 decide :: [Scores] -> Decision 12 decide scores = 13 let Scores t ss = merge score in Decision t $ 14 case last . sortBy (compare `on` weight) $ ss of 15 Nothing -> 16 Reject 17 Just (Free resource _) -> 18 Assign resource 19 Just (Scale capability _) -> 20 Scale capability 21 Just (Delay capability _) -> 22 Preempt capability

Slide 120

Slide 120 text

No content

Slide 121

Slide 121 text

No content

Slide 122

Slide 122 text

missing bits

Slide 123

Slide 123 text

real world

Slide 124

Slide 124 text

EPILOGUE Scheduling Fairness

Slide 125

Slide 125 text

No content

Slide 126

Slide 126 text

No content

Slide 127

Slide 127 text

No content

Slide 128

Slide 128 text

No content

Slide 129

Slide 129 text

$ $

Slide 130

Slide 130 text

Slide 131

Slide 131 text

2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?

Slide 132

Slide 132 text

2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?

Slide 133

Slide 133 text

2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours x second batch time

Slide 134

Slide 134 text

impacts time based bin packing

Slide 135

Slide 135 text

CONCLUSION IN

Slide 136

Slide 136 text

control scaling of resources and scheduling of tasks for optimal cost within required sla

Slide 137

Slide 137 text

control scaling of resources and scheduling of tasks for optimal cost within required sla

Slide 138

Slide 138 text

control scaling of resources and scheduling of tasks for optimal cost within required sla

Slide 139

Slide 139 text

control scaling of resources and scheduling of tasks for optimal cost within required sla