Price Aware Scheduling

A look at an algorithm in haskell that I am working on for doing fair
and less-fair-but-actually-kind-of efficient scheduling of tasks
across across multiple machines. The goal is to take existing work
scheduling approaches that tend to be ok at optimising for resource
utilisation across multiple dimensions (IO, Memory, CPU) for a fixed
set of resources, and introduce variably available / variable price
resources. Using budgets (min/max concurrent resources, cost of
execution, timelines of completion) I hope to arbitrate between
allocation of existing and new resources without sacrificing
utilisation.

Transcript

1. G PRICE AWARE SCHEDULING
3. task
4. task 12 hours
5. 2 x cpu task 12 hours
6. 2 x cpu task 12 hours 128 gb disk
7. 2 x cpu task 4gb memory 12 hours 128 gb disk
8. 2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours
9. 2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?
10. resource
11. resource 4 x cpu
12. resource 4 x cpu 8gb memory
13. resource 4 x cpu 240 gb disk 8gb memory
14. resource $0.40 / hr 4 x cpu 240 gb disk 8gb memory
15. resource $0.40 / hr 4 x cpu 240 gb disk 1Gb network 8gb memory
16. resource $0.40 / hr 4 x cpu 240 gb disk 1Gb network 4m boot-time 8gb memory
17. scheduler
18. scheduler
19. heterogeneous tasks
20. heterogeneous resources
21. access to large pool of resources
22. pay only when used
23. variable price
24. variable availability
25. coarse grained billing unit
26. control scaling of resources and scheduling of tasks for optimal cost within required sla
27. control scaling of resources and scheduling of tasks for optimal cost within required sla
28. control scaling of resources and scheduling of tasks for optimal cost within required sla
29. control scaling of resources and scheduling of tasks for optimal cost within required sla
30. WHY DO I CARE?
31. “We are in the business of turning technical debt into monetary debt” - Old Ambiata Proverb
35. ONE Resource Utilisation
37. knapsack problem
39. $1 $7 $11
42. meet constraints
43. maximise $$$ meet constraints
44. bin-packing problem
46. 1cpu 8cpu 2cpu
50. 1cpu 8cpu 2cpu
52. meet constraints
53. meet constraints maximise utilisation
54. np-hard (with ptas)
55. vector-packing problem
58. 1cpu, 4gb 8cpu, 16gb 2cpu, 1gb
61. meet all constraints
62. meet all constraints maximise utilisation
63. meet all constraints balance all dimensions maximise utilisation
64. vector-scheduling problem
65. meet all constraints balance all dimensions minimise makespan
66. WHY NOT? SO
69. no fixed resources
70. variable capability means i can fit resources to tasks rather than tasks to resources
71. TWO Auto Scaling
72. how do we combine auto scaling with scheduling?
73. few public or genuine attempts
76. trade urgency off against fitness
77. trade urgency off against fitness yet another packing problem
79. decouple scaling and scheduling questions
81. trigger scaling based on queue
82. how not to do it
83. packing very important for scale down
84. scheduler can’t scale down
85. 2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?
86. 2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?
87. WHY NOT? SO
90. optimised for latency
91. not optimised for cost
92. doesn’t / can’t leverage knowledge specific to our workloads
93. dependence on co-ordination
94. urgency is a stretch
95. THREE Cost Optimisation
96. 2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?
97. 2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?
98. 2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?
99. 2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?
100. resource $0.40 / hr 4 x cpu 240 gb disk 1Gb network 4m boot-time 8gb memory
101. resource $0.40 / hr 4 x cpu 240 gb disk 1Gb network 4m boot-time 8gb memory
102. resource $0.04 / hr 4 x cpu 240 gb disk 1Gb network 4m boot-time 8gb memory
105. FOUR The Scheduler
106. 2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?
107. 1 newtype TaskId = 2 TaskId Text 3 4 data Timeliness = 5 Interactive 6 | Lazy 7 | FinishBy UTCTime 8 9 data Requirement = 10 Requirement { 11 memory :: Memory 12 , disk :: Disk 13 , network :: Network 14 , timeliness :: Timeliness 15 } deriving (Eq, Show) 16 17 data Task = 18 Task { 19 taskId :: !TaskId 20 , taskRequirement :: !Requirement 21 } deriving (Eq, Show)
109. 1 newtype ResourceId = 2 ResourceId Text 3 4 data Resource = 5 Resource { 6 memory :: Memory 7 , disk :: Disk 8 , network :: Network 9 , cost :: Price 10 } 11 12 data Allocation = 13 Allocated ResourceId Task 14 | Unallocated ResourceId 15 16 data Capability = 17 Capability { 18 capabilityResource :: Resource 19 , capabilityAllocated :: [Allocation] 20 }
110. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision] 2 submit now capabiities tasks = 3 schedule now capabiities <$> 4 sortBy (compare `on` urgency now) tasks
111. first-fit decreasing
112. 1 urgency :: UTCTime -> Task -> Double 2 urgency now t = 3 case timelinessOf t of 4 Interactive -> 5 1.0 6 Lazy -> 7 0.0 8 FinishBy when -> 9 distributionOver (statisticsOf t) when
113. 99th percentile 1.0 0.o
114. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision] 2 submit now capabiities tasks = 3 schedule now capabiities <$> 4 sortBy (compare `on` urgency now) tasks
115. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision] 2 submit now capabiities tasks = 3 schedule now capabiities <$> 4 sortBy (compare `on` urgency now) tasks 5 6 schedule :: UTCTime -> [Capability] -> Task -> Decision 7 schedule now capabilities task = 8 decide $ 9 evaluate now task <$> capabilities
116. 1 data Score = 2 Free Resource [Double] 3 | Scale Capability [Double] 4 | Delay Capability [Double] 5 6 data Scores = 7 Scores Task [Score] 8 9 evaluate :: UTCTime -> Task -> Capability -> Scores 10 evaluate now task capability = 11 merge [ 12 evaluateFree now task capability 13 , evaluateRunning now task capability 14 , evaluateScaled now task capability 15 ] 16 17 evaluateFree :: UTCTime -> Task -> Capability -> Scores 18 19 evaluateRunning :: UTCTime -> Task -> Capability -> Scores 20 21 evaluateScaled :: UTCTime -> Task -> Capability -> Scores
117. 1 memory resource allocated required = 2 max 0 $ required / resource - allocated 3 4 cpu resource allocated required = 5 max 0 $ required / resource - allocated 6 7 network resource allocated required = 8 max 0 $ required / resource - allocated 9 10 timeliness remaining average variance = 11 (average + variance) / remaining
118. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision] 2 submit now capabiities tasks = 3 schedule now capabiities <$> 4 sortBy (compare `on` urgency now) tasks 5 6 schedule :: UTCTime -> [Capability] -> Task -> Decision 7 schedule now capabilities task = 8 decide $ 9 evaluate now task <$> capabilities
119. 1 data Decision = 2 Decision Task Outcome 3 4 data Outcome = 5 Assign Resource 6 | Delay Resource 7 | Scale Capability 8 | Preempt Capability 9 | Reject 10 11 decide :: [Scores] -> Decision 12 decide scores = 13 let Scores t ss = merge score in Decision t $ 14 case last . sortBy (compare `on` weight) $ ss of 15 Nothing -> 16 Reject 17 Just (Free resource _) -> 18 Assign resource 19 Just (Scale capability _) -> 20 Scale capability 21 Just (Delay capability _) -> 22 Preempt capability
122. missing bits
123. real world
124. EPILOGUE Scheduling Fairness
129. $ $
130. $
131. 2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?
132. 2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours preemptible y/n?
133. 2 x cpu task 4gb memory 12 hours 128 gb disk σ 1.1 hours, μ 4hours x second batch time
134. impacts time based bin packing
135. CONCLUSION IN
136. control scaling of resources and scheduling of tasks for optimal cost within required sla
137. control scaling of resources and scheduling of tasks for optimal cost within required sla
138. control scaling of resources and scheduling of tasks for optimal cost within required sla
139. control scaling of resources and scheduling of tasks for optimal cost within required sla
140. prior knowledge wins
141. layered bin packing
142. layers prioritise conflicting goals
143. G PRICE AWARE SCHEDULING