Mark Hibberd
May 25, 2016
480

Price Aware Scheduling

A look at an algorithm in haskell that I am working on for doing fair
and less-fair-but-actually-kind-of efficient scheduling of tasks
across across multiple machines. The goal is to take existing work
scheduling approaches that tend to be ok at optimising for resource
utilisation across multiple dimensions (IO, Memory, CPU) for a fixed
set of resources, and introduce variably available / variable price
resources. Using budgets (min/max concurrent resources, cost of
execution, timelines of completion) I hope to arbitrate between
allocation of existing and new resources without sacrificing
utilisation.

May 25, 2016

Transcript

1. G PRICE
AWARE
SCHEDULING

12 hours

4. 2 x cpu
12 hours

5. 2 x cpu
12 hours
128 gb disk

6. 2 x cpu
4gb memory
12 hours
128 gb disk

7. 2 x cpu
4gb memory
12 hours
128 gb disk
σ 1.1 hours,
μ 4hours

8. 2 x cpu
4gb memory
12 hours
128 gb disk
σ 1.1 hours,
μ 4hours
preemptible
y/n?

9. resource

10. resource
4 x cpu

11. resource
4 x cpu 8gb memory

12. resource
4 x cpu
240 gb disk
8gb memory

13. resource
\$0.40 / hr
4 x cpu
240 gb disk
8gb memory

14. resource
\$0.40 / hr
4 x cpu
240 gb disk
1Gb network
8gb memory

15. resource
\$0.40 / hr
4 x cpu
240 gb disk
1Gb network 4m boot-time
8gb memory

16. scheduler

17. scheduler

19. heterogeneous resources

21. pay only when used

22. variable price

23. variable availability

24. coarse grained billing unit

25. control scaling of resources
for optimal cost within required sla

26. control scaling of resources
for optimal cost within required sla

27. control scaling of resources
for optimal cost within required sla

28. control scaling of resources
for optimal cost within required sla

29. WHY DO I
CARE?

30. “We are in the business of
turning technical debt into
monetary debt”
- Old Ambiata Proverb

31. ONE
Resource Utilisation

32. knapsack problem

33. \$1
\$7
\$11

34. meet
constraints

35. maximise
\$\$\$
meet
constraints

36. bin-packing problem

37. 1cpu
8cpu
2cpu

38. 1cpu
8cpu
2cpu

39. meet
constraints

40. meet
constraints
maximise
utilisation

41. np-hard
(with ptas)

42. vector-packing problem

43. 1cpu, 4gb
8cpu, 16gb
2cpu, 1gb

44. meet all
constraints

45. meet all
constraints
maximise
utilisation

46. meet all
constraints
balance all
dimensions
maximise
utilisation

47. vector-scheduling problem

48. meet all
constraints
balance all
dimensions
minimise
makespan

49. WHY NOT?
SO

50. no fixed resources

51. variable capability means i can
fit resources to tasks rather than

52. TWO
Auto Scaling

53. how do we combine
auto scaling with scheduling?

54. few public or genuine attempts

55. 1

56. trade urgency off against fitness

57. trade urgency off against fitness
yet another packing problem

58. decouple scaling and scheduling questions

59. trigger scaling
based on queue

60. how not
to do it

61. packing very important for scale down

62. scheduler
can’t scale
down

63. 2 x cpu
4gb memory
12 hours
128 gb disk
σ 1.1 hours,
μ 4hours
preemptible
y/n?

64. 2 x cpu
4gb memory
12 hours
128 gb disk
σ 1.1 hours,
μ 4hours
preemptible
y/n?

65. WHY NOT?
SO

66. optimised for latency

67. not optimised for cost

68. doesn’t / can’t leverage

69. dependence on co-ordination

70. urgency is a stretch

71. THREE
Cost Optimisation

72. 2 x cpu
4gb memory
12 hours
128 gb disk
σ 1.1 hours,
μ 4hours
preemptible
y/n?

73. 2 x cpu
4gb memory
12 hours
128 gb disk
σ 1.1 hours,
μ 4hours
preemptible
y/n?

74. 2 x cpu
4gb memory
12 hours
128 gb disk
σ 1.1 hours,
μ 4hours
preemptible
y/n?

75. 2 x cpu
4gb memory
12 hours
128 gb disk
σ 1.1 hours,
μ 4hours
preemptible
y/n?

76. resource
\$0.40 / hr
4 x cpu
240 gb disk
1Gb network 4m boot-time
8gb memory

77. resource
\$0.40 / hr
4 x cpu
240 gb disk
1Gb network 4m boot-time
8gb memory

78. resource
\$0.04 / hr
4 x cpu
240 gb disk
1Gb network 4m boot-time
8gb memory

79. FOUR
The Scheduler

80. 2 x cpu
4gb memory
12 hours
128 gb disk
σ 1.1 hours,
μ 4hours
preemptible
y/n?

3
4 data Timeliness =
5 Interactive
6 | Lazy
7 | FinishBy UTCTime
8
9 data Requirement =
10 Requirement {
11 memory :: Memory
12 , disk :: Disk
13 , network :: Network
14 , timeliness :: Timeliness
15 } deriving (Eq, Show)
16
21 } deriving (Eq, Show)

82. 1 newtype ResourceId =
2 ResourceId Text
3
4 data Resource =
5 Resource {
6 memory :: Memory
7 , disk :: Disk
8 , network :: Network
9 , cost :: Price
10 }
11
12 data Allocation =
14 | Unallocated ResourceId
15
16 data Capability =
17 Capability {
18 capabilityResource :: Resource
19 , capabilityAllocated :: [Allocation]
20 }

83. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision]
2 submit now capabiities tasks =
3 schedule now capabiities <\$>
4 sortBy (compare `on` urgency now) tasks

84. first-fit decreasing

85. 1 urgency :: UTCTime -> Task -> Double
2 urgency now t =
3 case timelinessOf t of
4 Interactive ->
5 1.0
6 Lazy ->
7 0.0
8 FinishBy when ->
9 distributionOver (statisticsOf t) when

86. 99th percentile 1.0
0.o

87. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision]
2 submit now capabiities tasks =
3 schedule now capabiities <\$>
4 sortBy (compare `on` urgency now) tasks

88. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision]
2 submit now capabiities tasks =
3 schedule now capabiities <\$>
4 sortBy (compare `on` urgency now) tasks
5
6 schedule :: UTCTime -> [Capability] -> Task -> Decision
7 schedule now capabilities task =
8 decide \$
9 evaluate now task <\$> capabilities

89. 1 data Score =
2 Free Resource [Double]
3 | Scale Capability [Double]
4 | Delay Capability [Double]
5
6 data Scores =
8
9 evaluate :: UTCTime -> Task -> Capability -> Scores
10 evaluate now task capability =
11 merge [
13 , evaluateRunning now task capability
14 , evaluateScaled now task capability
15 ]
16
17 evaluateFree :: UTCTime -> Task -> Capability -> Scores
18
19 evaluateRunning :: UTCTime -> Task -> Capability -> Scores
20
21 evaluateScaled :: UTCTime -> Task -> Capability -> Scores

90. 1 memory resource allocated required =
2 max 0 \$ required / resource - allocated
3
4 cpu resource allocated required =
5 max 0 \$ required / resource - allocated
6
7 network resource allocated required =
8 max 0 \$ required / resource - allocated
9
10 timeliness remaining average variance =
11 (average + variance) / remaining

91. 1 submit :: UTCTime -> [Capability] -> [Task] -> [Decision]
2 submit now capabiities tasks =
3 schedule now capabiities <\$>
4 sortBy (compare `on` urgency now) tasks
5
6 schedule :: UTCTime -> [Capability] -> Task -> Decision
7 schedule now capabilities task =
8 decide \$
9 evaluate now task <\$> capabilities

92. 1 data Decision =
3
4 data Outcome =
5 Assign Resource
6 | Delay Resource
7 | Scale Capability
8 | Preempt Capability
9 | Reject
10
11 decide :: [Scores] -> Decision
12 decide scores =
13 let Scores t ss = merge score in Decision t \$
14 case last . sortBy (compare `on` weight) \$ ss of
15 Nothing ->
16 Reject
17 Just (Free resource _) ->
18 Assign resource
19 Just (Scale capability _) ->
20 Scale capability
21 Just (Delay capability _) ->
22 Preempt capability

93. missing bits

94. real world

95. EPILOGUE
Scheduling Fairness

96. \$ \$

97. \$

98. 2 x cpu
4gb memory
12 hours
128 gb disk
σ 1.1 hours,
μ 4hours
preemptible
y/n?

99. 2 x cpu
4gb memory
12 hours
128 gb disk
σ 1.1 hours,
μ 4hours
preemptible
y/n?

100. 2 x cpu
4gb memory
12 hours
128 gb disk
σ 1.1 hours,
μ 4hours
x second
batch time

101. impacts time based bin packing

102. CONCLUSION
IN

103. control scaling of resources
for optimal cost within required sla

104. control scaling of resources
for optimal cost within required sla

105. control scaling of resources
for optimal cost within required sla

106. control scaling of resources