Price Aware Scheduling

A look at an algorithm in haskell that I am working on for doing fair
and less-fair-but-actually-kind-of efficient scheduling of tasks
across across multiple machines. The goal is to take existing work
scheduling approaches that tend to be ok at optimising for resource
utilisation across multiple dimensions (IO, Memory, CPU) for a fixed
set of resources, and introduce variably available / variable price
resources. Using budgets (min/max concurrent resources, cost of
execution, timelines of completion) I hope to arbitrate between
allocation of existing and new resources without sacrificing
utilisation.

Transcript

1. G PRICE
   AWARE
   SCHEDULING
   

   View Slide

2. View Slide

3. task
   

   View Slide

4. task
   12 hours
   

   View Slide

5. 2 x cpu
   task
   12 hours
   

   View Slide

6. 2 x cpu
   task
   12 hours
   128 gb disk
   

   View Slide

7. 2 x cpu
   task
   4gb memory
   12 hours
   128 gb disk
   

   View Slide

8. 2 x cpu
   task
   4gb memory
   12 hours
   128 gb disk
   σ 1.1 hours,
   μ 4hours
   

   View Slide

9. 2 x cpu
   task
   4gb memory
   12 hours
   128 gb disk
   σ 1.1 hours,
   μ 4hours
   preemptible
   y/n?
   

   View Slide

10. resource
    

    View Slide

11. resource
    4 x cpu
    

    View Slide

12. resource
    4 x cpu 8gb memory
    

    View Slide

13. resource
    4 x cpu
    240 gb disk
    8gb memory
    

    View Slide

14. resource
    $0.40 / hr
    4 x cpu
    240 gb disk
    8gb memory
    

    View Slide

15. resource
    $0.40 / hr
    4 x cpu
    240 gb disk
    1Gb network
    8gb memory
    

    View Slide

16. resource
    $0.40 / hr
    4 x cpu
    240 gb disk
    1Gb network 4m boot-time
    8gb memory
    

    View Slide

17. scheduler
    

    View Slide

18. scheduler
    

    View Slide

19. heterogeneous tasks
    

    View Slide

20. heterogeneous resources
    

    View Slide

21. access to large pool of resources
    

    View Slide

22. pay only when used
    

    View Slide

23. variable price
    

    View Slide

24. variable availability
    

    View Slide

25. coarse grained billing unit
    

    View Slide

26. control scaling of resources
    and scheduling of tasks
    for optimal cost within required sla
    

    View Slide

27. control scaling of resources
    and scheduling of tasks
    for optimal cost within required sla
    

    View Slide

28. control scaling of resources
    and scheduling of tasks
    for optimal cost within required sla
    

    View Slide

29. control scaling of resources
    and scheduling of tasks
    for optimal cost within required sla
    

    View Slide

30. WHY DO I
    CARE?
    

    View Slide

31. “We are in the business of
    turning technical debt into
    monetary debt”
    - Old Ambiata Proverb
    

    View Slide

32. View Slide

33. View Slide

34. View Slide

35. ONE
    Resource Utilisation
    

    View Slide

36. View Slide

37. knapsack problem
    

    View Slide

38. View Slide

39. $1
    $7
    $11
    

    View Slide

40. View Slide

41. View Slide

42. meet
    constraints
    

    View Slide

43. maximise
    $$$
    meet
    constraints
    

    View Slide

44. bin-packing problem
    

    View Slide

45. View Slide

46. 1cpu
    8cpu
    2cpu
    

    View Slide

47. View Slide

48. View Slide

49. View Slide

50. 1cpu
    8cpu
    2cpu
    

    View Slide

51. View Slide

52. meet
    constraints
    

    View Slide

53. meet
    constraints
    maximise
    utilisation
    

    View Slide

54. np-hard
    (with ptas)
    

    View Slide

55. vector-packing problem
    

    View Slide

56. View Slide

57. View Slide

58. 1cpu, 4gb
    8cpu, 16gb
    2cpu, 1gb
    

    View Slide

59. View Slide

60. View Slide

61. meet all
    constraints
    

    View Slide

62. meet all
    constraints
    maximise
    utilisation
    

    View Slide

63. meet all
    constraints
    balance all
    dimensions
    maximise
    utilisation
    

    View Slide

64. vector-scheduling problem
    

    View Slide

65. meet all
    constraints
    balance all
    dimensions
    minimise
    makespan
    

    View Slide

66. WHY NOT?
    SO
    

    View Slide

67. View Slide

68. View Slide

69. no fixed resources
    

    View Slide

70. variable capability means i can
    fit resources to tasks rather than
    tasks to resources
    

    View Slide

71. TWO
    Auto Scaling
    

    View Slide

72. how do we combine
    auto scaling with scheduling?
    

    View Slide

73. few public or genuine attempts
    

    View Slide

74. View Slide

75. View Slide

76. trade urgency off against fitness
    

    View Slide

77. trade urgency off against fitness
    yet another packing problem
    

    View Slide

78. View Slide

79. decouple scaling and scheduling questions
    

    View Slide

80. View Slide

81. trigger scaling
    based on queue
    

    View Slide

82. how not
    to do it
    

    View Slide

83. packing very important for scale down
    

    View Slide

84. scheduler
    can’t scale
    down
    

    View Slide

85. 2 x cpu
    task
    4gb memory
    12 hours
    128 gb disk
    σ 1.1 hours,
    μ 4hours
    preemptible
    y/n?
    

    View Slide

86. 2 x cpu
    task
    4gb memory
    12 hours
    128 gb disk
    σ 1.1 hours,
    μ 4hours
    preemptible
    y/n?
    

    View Slide

87. WHY NOT?
    SO
    

    View Slide

88. View Slide

89. View Slide

90. optimised for latency
    

    View Slide

91. not optimised for cost
    

    View Slide

92. doesn’t / can’t leverage
    knowledge specific to our workloads
    

    View Slide

93. dependence on co-ordination
    

    View Slide

94. urgency is a stretch
    

    View Slide

95. THREE
    Cost Optimisation
    

    View Slide

96. 2 x cpu
    task
    4gb memory
    12 hours
    128 gb disk
    σ 1.1 hours,
    μ 4hours
    preemptible
    y/n?
    

    View Slide

97. 2 x cpu
    task
    4gb memory
    12 hours
    128 gb disk
    σ 1.1 hours,
    μ 4hours
    preemptible
    y/n?
    

    View Slide

98. 2 x cpu
    task
    4gb memory
    12 hours
    128 gb disk
    σ 1.1 hours,
    μ 4hours
    preemptible
    y/n?
    

    View Slide

99. 2 x cpu
    task
    4gb memory
    12 hours
    128 gb disk
    σ 1.1 hours,
    μ 4hours
    preemptible
    y/n?
    

    View Slide

100. resource
     $0.40 / hr
     4 x cpu
     240 gb disk
     1Gb network 4m boot-time
     8gb memory
     

     View Slide

101. resource
     $0.40 / hr
     4 x cpu
     240 gb disk
     1Gb network 4m boot-time
     8gb memory
     

     View Slide

102. resource
     $0.04 / hr
     4 x cpu
     240 gb disk
     1Gb network 4m boot-time
     8gb memory
     

     View Slide

103. View Slide

104. View Slide

105. FOUR
     The Scheduler
     

     View Slide

106. 2 x cpu
     task
     4gb memory
     12 hours
     128 gb disk
     σ 1.1 hours,
     μ 4hours
     preemptible
     y/n?
     

     View Slide

107. 1 newtype TaskId =
     2 TaskId Text
     3
     4 data Timeliness =
     5 Interactive
     6 | Lazy
     7 | FinishBy UTCTime
     8
     9 data Requirement =
     10 Requirement {
     11 memory :: Memory
     12 , disk :: Disk
     13 , network :: Network
     14 , timeliness :: Timeliness
     15 } deriving (Eq, Show)
     16
     17 data Task =
     18 Task {
     19 taskId :: !TaskId
     20 , taskRequirement :: !Requirement
     21 } deriving (Eq, Show)
     

     View Slide

108. View Slide

109. 1 newtype ResourceId =
     2 ResourceId Text
     3
     4 data Resource =
     5 Resource {
     6 memory :: Memory
     7 , disk :: Disk
     8 , network :: Network
     9 , cost :: Price
     10 }
     11
     12 data Allocation =
     13 Allocated ResourceId Task
     14 | Unallocated ResourceId
     15
     16 data Capability =
     17 Capability {
     18 capabilityResource :: Resource
     19 , capabilityAllocated :: [Allocation]
     20 }
     

     View Slide

110. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision]
     2 submit now capabiities tasks =
     3 schedule now capabiities <$>
     4 sortBy (compare `on` urgency now) tasks
     

     View Slide

111. first-fit decreasing
     

     View Slide

112. 1 urgency :: UTCTime -> Task -> Double
     2 urgency now t =
     3 case timelinessOf t of
     4 Interactive ->
     5 1.0
     6 Lazy ->
     7 0.0
     8 FinishBy when ->
     9 distributionOver (statisticsOf t) when
     

     View Slide

113. 99th percentile 1.0
     0.o
     

     View Slide

114. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision]
     2 submit now capabiities tasks =
     3 schedule now capabiities <$>
     4 sortBy (compare `on` urgency now) tasks
     

     View Slide

115. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision]
     2 submit now capabiities tasks =
     3 schedule now capabiities <$>
     4 sortBy (compare `on` urgency now) tasks
     5
     6 schedule :: UTCTime -> [Capability] -> Task -> Decision
     7 schedule now capabilities task =
     8 decide $
     9 evaluate now task <$> capabilities
     

     View Slide

116. 1 data Score =
     2 Free Resource [Double]
     3 | Scale Capability [Double]
     4 | Delay Capability [Double]
     5
     6 data Scores =
     7 Scores Task [Score]
     8
     9 evaluate :: UTCTime -> Task -> Capability -> Scores
     10 evaluate now task capability =
     11 merge [
     12 evaluateFree now task capability
     13 , evaluateRunning now task capability
     14 , evaluateScaled now task capability
     15 ]
     16
     17 evaluateFree :: UTCTime -> Task -> Capability -> Scores
     18
     19 evaluateRunning :: UTCTime -> Task -> Capability -> Scores
     20
     21 evaluateScaled :: UTCTime -> Task -> Capability -> Scores
     

     View Slide

117. 1 memory resource allocated required =
     2 max 0 $ required / resource - allocated
     3
     4 cpu resource allocated required =
     5 max 0 $ required / resource - allocated
     6
     7 network resource allocated required =
     8 max 0 $ required / resource - allocated
     9
     10 timeliness remaining average variance =
     11 (average + variance) / remaining
     

     View Slide

118. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision]
     2 submit now capabiities tasks =
     3 schedule now capabiities <$>
     4 sortBy (compare `on` urgency now) tasks
     5
     6 schedule :: UTCTime -> [Capability] -> Task -> Decision
     7 schedule now capabilities task =
     8 decide $
     9 evaluate now task <$> capabilities
     

     View Slide

119. 1 data Decision =
     2 Decision Task Outcome
     3
     4 data Outcome =
     5 Assign Resource
     6 | Delay Resource
     7 | Scale Capability
     8 | Preempt Capability
     9 | Reject
     10
     11 decide :: [Scores] -> Decision
     12 decide scores =
     13 let Scores t ss = merge score in Decision t $
     14 case last . sortBy (compare `on` weight) $ ss of
     15 Nothing ->
     16 Reject
     17 Just (Free resource _) ->
     18 Assign resource
     19 Just (Scale capability _) ->
     20 Scale capability
     21 Just (Delay capability _) ->
     22 Preempt capability
     

     View Slide

120. View Slide

121. View Slide

122. missing bits
     

     View Slide

123. real world
     

     View Slide

124. EPILOGUE
     Scheduling Fairness
     

     View Slide

125. View Slide

126. View Slide

127. View Slide

128. View Slide

129. $ $
     

     View Slide

130. $
     

     View Slide

131. 2 x cpu
     task
     4gb memory
     12 hours
     128 gb disk
     σ 1.1 hours,
     μ 4hours
     preemptible
     y/n?
     

     View Slide

132. 2 x cpu
     task
     4gb memory
     12 hours
     128 gb disk
     σ 1.1 hours,
     μ 4hours
     preemptible
     y/n?
     

     View Slide

133. 2 x cpu
     task
     4gb memory
     12 hours
     128 gb disk
     σ 1.1 hours,
     μ 4hours
     x second
     batch time
     

     View Slide

134. impacts time based bin packing
     

     View Slide

135. CONCLUSION
     IN
     

     View Slide

136. control scaling of resources
     and scheduling of tasks
     for optimal cost within required sla
     

     View Slide

137. control scaling of resources
     and scheduling of tasks
     for optimal cost within required sla
     

     View Slide

138. control scaling of resources
     and scheduling of tasks
     for optimal cost within required sla
     

     View Slide

139. control scaling of resources
     and scheduling of tasks
     for optimal cost within required sla
     

     View Slide

140. prior knowledge wins
     

     View Slide

141. layered bin packing
     

     View Slide

142. layers prioritise conflicting goals
     

     View Slide

143. G PRICE
     AWARE
     SCHEDULING
     

     View Slide