Task Planning on the Grid

A look at why you might want use grid computing, as well as the challenges that lie in creating schedules that optimize resource use. Potential architectures and algorithms are presented.

Transcript

1. TASK PLANNING ON THE GRID Chad Taylor
2. Beware: OPINIONS ahead!
3. This is interesting!
4. Interesting = Ninjas ⋀ Pirates ⋀ Zombies
5. Let’s pretend...
8. One day...
9. This image is licensed under the Creative Commons Attribution 3.0 Unported license. Attribution: Laura Poitras / Praxis Films
16. ...
17. ...
18. ... I like turtles!
20. Motivation? Check.
21. Do you even CLOUD, bro?
22. Why cloud?
23. Why cloud? Homogeneity
24. Why cloud? Homogeneity Reliability
25. Why not cloud?
26. Why not cloud? $
27. What percentage of time is your computer running 100% CPU usage?
28. What percentage of time is your organization’s computers running 100% CPU usage?
29. The Grid
30. Why grid?
31. Why grid? Use existing resources
32. Why grid? Use existing resources $
33. Challenges?
34. Challenges? Heterogeneous resources
35. Challenges? Heterogeneous resources Dynamic nature
36. Planner DAG GRID Services Executor (Yu, et al. 2007) (Blythe, et al. 2005)
37. Planner DAG GRID Services Executor (Yu, et al. 2007) (Blythe, et al. 2005)
38. Planner Performance History Repository Predictor Scheduler Scheduler (Yu, et al. 2007)
39. Performance History Repository
40. Stores: * Task type * Input task parameters * Resource characteristics * Measured fitness values (Burton, et al. 1997)
41. NoSQL?
42. Predictor
43. Learn task performance characteristics and make accurate predictions for future task executions Goal:
44. Artificial neural network with on-line learning Algorithm:
45. Implementation: * One neural network per task type * Train in real-time with data in repository * Skip expensive backtracking costs * Custom hardware? (Burton, et al. 1997)
46. * How many nodes are required for each ANN? * How to prevent unlearning? Challenges:
47. Scheduler
48. Create accurate, optimal schedules Goal:
49. What is a schedule?
50. What is a schedule? “Assignment of tasks to specific time intervals of resources” (Fibich, et al. 2005)
51. A schedule with 5 tasks and 3 resources
52. What is an optimal schedule?
53. What is an optimal schedule? A schedule that “minimizes a given optimality criterion” (Fibich, et al. 2005)
54. * makespan * maximum lateness * # tardy jobs Optimality criteria: (Fibich, et al. 2005)
55. Scheduler Performance History Repository Predictor Symbiosis!
56. Scheduling is NP-Complete (García-Galán, et al. 2012)
57. Scheduling is NP-Complete (García-Galán, et al. 2012) :’-(
58. Multi-objective Particle Swarm Optimization (MOPSO) Algorithm:
59. PSO that minimizes more than one optimization function
60. Demo
61. What next?
63. Compare MOPSO against other algorithms (e.g., multi-objective GAs)
64. What about pirates?
66. Questions? https://github.com/tessellator/mopso @tessellator
67. References Blythe, James, et al. "Task scheduling strategies for workflow-based applications in grids." Cluster Computing and the Grid, 2005. CCGrid 2005. IEEE International Symposium on. Vol. 2. IEEE, 2005. Burton, Bruce, et al. "Identification and control of induction motor stator currents using fast on-line random training of a neural network." Industry Applications, IEEE Transactions on 33.3 (1997): 697- 704. Coello Coello, Carlos A., and Maximino Salazar Lechuga. "MOPSO: A proposal for multiple objective particle swarm optimization." Evolutionary Computation, 2002. CEC'02. Proceedings of the 2002 Congress on. Vol. 2. IEEE, 2002. Fibich, Pavel, Ludek Matyska, and Hana Rudová. "Model of grid scheduling problem." Exploring Planning and Scheduling for Web Services, Grid and Autonomic Computing (2005): 17-24. García-Galán, S., R. P. Prado, and J. E. Muñoz Expósito. "Fuzzy scheduling with swarm intelligence- based knowledge acquisition for grid computing." Engineering Applications of Artificial Intelligence 25.2 (2012): 359-375. Reyes-Sierra, Margarita, and CA Coello Coello. "Multi-objective particle swarm optimizers: A survey of the state-of-the-art." International Journal of Computational Intelligence Research 2.3 (2006): 287- 308. Schopf, Jennifer M. "A general architecture for scheduling on the grid." Special issue of JPDC on Grid Computing 4 (2002). Yu, Zhifeng, and Weisong Shi. "An adaptive rescheduling strategy for grid workflow applications." Parallel and Distributed Processing Symposium, 2007. IPDPS 2007. IEEE International. IEEE, 2007.