Price Aware Scheduling

Price Aware Scheduling

A look at an algorithm in haskell that I am working on for doing fair
and less-fair-but-actually-kind-of efficient scheduling of tasks
across across multiple machines. The goal is to take existing work
scheduling approaches that tend to be ok at optimising for resource
utilisation across multiple dimensions (IO, Memory, CPU) for a fixed
set of resources, and introduce variably available / variable price
resources. Using budgets (min/max concurrent resources, cost of
execution, timelines of completion) I hope to arbitrate between
allocation of existing and new resources without sacrificing
utilisation.

42d9867a0fee0fa6de6534e9df0f1e9b?s=128

Mark Hibberd

May 25, 2016
Tweet

Transcript

  1. G PRICE AWARE SCHEDULING

  2. None
  3. task

  4. task 12 hours

  5. 2 x cpu task 12 hours

  6. 2 x cpu task 12 hours 128 gb disk

  7. 2 x cpu task 4gb memory 12 hours 128 gb

    disk
  8. 2 x cpu task 4gb memory 12 hours 128 gb

    disk σ 1.1 hours, μ 4hours
  9. 2 x cpu task 4gb memory 12 hours 128 gb

    disk σ 1.1 hours, μ 4hours preemptible y/n?
  10. resource

  11. resource 4 x cpu

  12. resource 4 x cpu 8gb memory

  13. resource 4 x cpu 240 gb disk 8gb memory

  14. resource $0.40 / hr 4 x cpu 240 gb disk

    8gb memory
  15. resource $0.40 / hr 4 x cpu 240 gb disk

    1Gb network 8gb memory
  16. resource $0.40 / hr 4 x cpu 240 gb disk

    1Gb network 4m boot-time 8gb memory
  17. scheduler

  18. scheduler

  19. heterogeneous tasks

  20. heterogeneous resources

  21. access to large pool of resources

  22. pay only when used

  23. variable price

  24. variable availability

  25. coarse grained billing unit

  26. control scaling of resources and scheduling of tasks for optimal

    cost within required sla
  27. control scaling of resources and scheduling of tasks for optimal

    cost within required sla
  28. control scaling of resources and scheduling of tasks for optimal

    cost within required sla
  29. control scaling of resources and scheduling of tasks for optimal

    cost within required sla
  30. WHY DO I CARE?

  31. “We are in the business of turning technical debt into

    monetary debt” - Old Ambiata Proverb
  32. None
  33. None
  34. None
  35. ONE Resource Utilisation

  36. None
  37. knapsack problem

  38. None
  39. $1 $7 $11

  40. None
  41. None
  42. meet constraints

  43. maximise $$$ meet constraints

  44. bin-packing problem

  45. None
  46. 1cpu 8cpu 2cpu

  47. None
  48. None
  49. None
  50. 1cpu 8cpu 2cpu

  51. None
  52. meet constraints

  53. meet constraints maximise utilisation

  54. np-hard (with ptas)

  55. vector-packing problem

  56. None
  57. None
  58. 1cpu, 4gb 8cpu, 16gb 2cpu, 1gb

  59. None
  60. None
  61. meet all constraints

  62. meet all constraints maximise utilisation

  63. meet all constraints balance all dimensions maximise utilisation

  64. vector-scheduling problem

  65. meet all constraints balance all dimensions minimise makespan

  66. WHY NOT? SO

  67. None
  68. None
  69. no fixed resources

  70. variable capability means i can fit resources to tasks rather

    than tasks to resources
  71. TWO Auto Scaling

  72. how do we combine auto scaling with scheduling?

  73. few public or genuine attempts

  74. None
  75. 1

  76. trade urgency off against fitness

  77. trade urgency off against fitness yet another packing problem

  78. None
  79. decouple scaling and scheduling questions

  80. None
  81. trigger scaling based on queue

  82. how not to do it

  83. packing very important for scale down

  84. scheduler can’t scale down

  85. 2 x cpu task 4gb memory 12 hours 128 gb

    disk σ 1.1 hours, μ 4hours preemptible y/n?
  86. 2 x cpu task 4gb memory 12 hours 128 gb

    disk σ 1.1 hours, μ 4hours preemptible y/n?
  87. WHY NOT? SO

  88. None
  89. None
  90. optimised for latency

  91. not optimised for cost

  92. doesn’t / can’t leverage knowledge specific to our workloads

  93. dependence on co-ordination

  94. urgency is a stretch

  95. THREE Cost Optimisation

  96. 2 x cpu task 4gb memory 12 hours 128 gb

    disk σ 1.1 hours, μ 4hours preemptible y/n?
  97. 2 x cpu task 4gb memory 12 hours 128 gb

    disk σ 1.1 hours, μ 4hours preemptible y/n?
  98. 2 x cpu task 4gb memory 12 hours 128 gb

    disk σ 1.1 hours, μ 4hours preemptible y/n?
  99. 2 x cpu task 4gb memory 12 hours 128 gb

    disk σ 1.1 hours, μ 4hours preemptible y/n?
  100. resource $0.40 / hr 4 x cpu 240 gb disk

    1Gb network 4m boot-time 8gb memory
  101. resource $0.40 / hr 4 x cpu 240 gb disk

    1Gb network 4m boot-time 8gb memory
  102. resource $0.04 / hr 4 x cpu 240 gb disk

    1Gb network 4m boot-time 8gb memory
  103. None
  104. None
  105. FOUR The Scheduler

  106. 2 x cpu task 4gb memory 12 hours 128 gb

    disk σ 1.1 hours, μ 4hours preemptible y/n?
  107. 1 newtype TaskId = 2 TaskId Text 3 4 data

    Timeliness = 5 Interactive 6 | Lazy 7 | FinishBy UTCTime 8 9 data Requirement = 10 Requirement { 11 memory :: Memory 12 , disk :: Disk 13 , network :: Network 14 , timeliness :: Timeliness 15 } deriving (Eq, Show) 16 17 data Task = 18 Task { 19 taskId :: !TaskId 20 , taskRequirement :: !Requirement 21 } deriving (Eq, Show)
  108. None
  109. 1 newtype ResourceId = 2 ResourceId Text 3 4 data

    Resource = 5 Resource { 6 memory :: Memory 7 , disk :: Disk 8 , network :: Network 9 , cost :: Price 10 } 11 12 data Allocation = 13 Allocated ResourceId Task 14 | Unallocated ResourceId 15 16 data Capability = 17 Capability { 18 capabilityResource :: Resource 19 , capabilityAllocated :: [Allocation] 20 }
  110. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision]

    2 submit now capabiities tasks = 3 schedule now capabiities <$> 4 sortBy (compare `on` urgency now) tasks
  111. first-fit decreasing

  112. 1 urgency :: UTCTime -> Task -> Double 2 urgency

    now t = 3 case timelinessOf t of 4 Interactive -> 5 1.0 6 Lazy -> 7 0.0 8 FinishBy when -> 9 distributionOver (statisticsOf t) when
  113. 99th percentile 1.0 0.o

  114. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision]

    2 submit now capabiities tasks = 3 schedule now capabiities <$> 4 sortBy (compare `on` urgency now) tasks
  115. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision]

    2 submit now capabiities tasks = 3 schedule now capabiities <$> 4 sortBy (compare `on` urgency now) tasks 5 6 schedule :: UTCTime -> [Capability] -> Task -> Decision 7 schedule now capabilities task = 8 decide $ 9 evaluate now task <$> capabilities
  116. 1 data Score = 2 Free Resource [Double] 3 |

    Scale Capability [Double] 4 | Delay Capability [Double] 5 6 data Scores = 7 Scores Task [Score] 8 9 evaluate :: UTCTime -> Task -> Capability -> Scores 10 evaluate now task capability = 11 merge [ 12 evaluateFree now task capability 13 , evaluateRunning now task capability 14 , evaluateScaled now task capability 15 ] 16 17 evaluateFree :: UTCTime -> Task -> Capability -> Scores 18 19 evaluateRunning :: UTCTime -> Task -> Capability -> Scores 20 21 evaluateScaled :: UTCTime -> Task -> Capability -> Scores
  117. 1 memory resource allocated required = 2 max 0 $

    required / resource - allocated 3 4 cpu resource allocated required = 5 max 0 $ required / resource - allocated 6 7 network resource allocated required = 8 max 0 $ required / resource - allocated 9 10 timeliness remaining average variance = 11 (average + variance) / remaining
  118. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision]

    2 submit now capabiities tasks = 3 schedule now capabiities <$> 4 sortBy (compare `on` urgency now) tasks 5 6 schedule :: UTCTime -> [Capability] -> Task -> Decision 7 schedule now capabilities task = 8 decide $ 9 evaluate now task <$> capabilities
  119. 1 data Decision = 2 Decision Task Outcome 3 4

    data Outcome = 5 Assign Resource 6 | Delay Resource 7 | Scale Capability 8 | Preempt Capability 9 | Reject 10 11 decide :: [Scores] -> Decision 12 decide scores = 13 let Scores t ss = merge score in Decision t $ 14 case last . sortBy (compare `on` weight) $ ss of 15 Nothing -> 16 Reject 17 Just (Free resource _) -> 18 Assign resource 19 Just (Scale capability _) -> 20 Scale capability 21 Just (Delay capability _) -> 22 Preempt capability
  120. None
  121. None
  122. missing bits

  123. real world

  124. EPILOGUE Scheduling Fairness

  125. None
  126. None
  127. None
  128. None
  129. $ $

  130. $

  131. 2 x cpu task 4gb memory 12 hours 128 gb

    disk σ 1.1 hours, μ 4hours preemptible y/n?
  132. 2 x cpu task 4gb memory 12 hours 128 gb

    disk σ 1.1 hours, μ 4hours preemptible y/n?
  133. 2 x cpu task 4gb memory 12 hours 128 gb

    disk σ 1.1 hours, μ 4hours x second batch time
  134. impacts time based bin packing

  135. CONCLUSION IN

  136. control scaling of resources and scheduling of tasks for optimal

    cost within required sla
  137. control scaling of resources and scheduling of tasks for optimal

    cost within required sla
  138. control scaling of resources and scheduling of tasks for optimal

    cost within required sla
  139. control scaling of resources and scheduling of tasks for optimal

    cost within required sla
  140. prior knowledge wins

  141. layered bin packing

  142. layers prioritise conflicting goals

  143. G PRICE AWARE SCHEDULING