Anatomy of a Cluster Manager

An overview of the fundamental components or building blocks of a general purpose cluster manager, the future of cloud computing, and a brief rant on the state of the industry.

Transcript

1. Anatomy of a Cluster Manager Jeff Lindsay @progrium Glider Labs @gliderlabs
2. Hosts Transitional Clusters
3. Hosts Transitional Clusters
4. Hosts Transitional Clusters 1995
5. Hosts Transitional Clusters 1995 2005
6. Hosts Transitional Clusters 1995 2005 2015
7. Hosts Transitional Clusters 1995 2005 2015
8. Hosts Transitional Clusters 1995 2005 2015
9. Hosts Transitional Clusters 1995 2005 2015
10. Hosts Transitional Clusters 1995 2005 2015
19. Disaggregation
21. "We’re big believers in the value of disaggregation – of breaking down traditional data center technologies into their core components so we can build new systems that are more flexible, more scalable, and more efficient.”
 —Somebody at Facebook
36. The Systems Software Mess
37. A general purpose cluster manager is hard It’s sort of a distributed system manager
38. Distributed systems are inherently complex
39. 2k lines 40k lines Flynn Dokku
40. 2k lines 40k lines Flynn Dokku { Distributed
41. Dokku • Docker powered mini-Heroku • First Docker “killer app” • Single host only • Originally 65 lines of Bash • Simple and extensible #!/bin/bash set -e export PLUGIN_PATH=${PLUGIN_PATH:="/var/lib/dokku/plugins"} case "$1" in receive) APP="$2"; IMAGE="app/$APP" echo "-----> Building $APP ..." cat | dokku build $IMAGE echo "-----> Build complete!" echo "-----> Deploying $APP ..." dokku deploy $APP $IMAGE echo "=====> Application deployed:" echo " $(dokku url $APP)" echo ;; build) IMAGE="$2" id=$(cat | docker run -i -a stdin progrium/buildstep /bin/bash -c "mkdir -p /app && tar -xC /app") test $(docker wait $id) -eq 0 docker commit $id $IMAGE > /dev/null id=$(docker run -d $IMAGE /build/builder) docker attach $id test $(docker wait $id) -eq 0 docker commit $id $IMAGE > /dev/null ;; deploy) APP="$2"; IMAGE="$3" pluginhook pre-deploy $APP $IMAGE if [[ ! -f "$HOME/$APP/PORT" ]]; then # First deploy id=$(docker run -d -p 5000 -e PORT=5000 $IMAGE /bin/bash -c "/start web") echo $id > "$HOME/$APP/CONTAINER" port=$(docker port $id 5000) echo $port > "$HOME/$APP/PORT" echo "$(< "$HOME/HOSTNAME"):$port" > "$HOME/$APP/ADDRESS" else # Regular deploy oldid=$(< "$HOME/$APP/CONTAINER") docker kill $oldid > /dev/null port=$(< "$HOME/$APP/PORT") id=$(docker run -d -p ":$port" -e "PORT=$port" $IMAGE /bin/bash -c "/start web") echo $id > "$HOME/$APP/CONTAINER" fi pluginhook post-deploy $APP $port ;; plugins) ls -1 -d $PLUGIN_PATH/*/ pluginhook install ;; help) cat<<EOF | pluginhook commands help | sort help Print the list of commands plugins Print active plugins EOF ;; *) pluginhook commands "$@" ;; esac
42. Disaggregation
47. Cluster Manager
48. Cluster Manager
49. Building Blocks of a Cluster Manager
50. Prerequisite Building Blocks
51. Prerequisite Building Blocks 1. Container Engine
52. Prerequisite Building Blocks 1. Container Engine 2. Distributed Coordination
55. Inventory Hosts
56. Inventory Hosts
57. Unit of Compute
58. Unit of Compute “Job”
59. Placement of Jobs
60. Placement of Jobs
61. Placement of Jobs
62. Placement of Jobs
63. “Scheduling”
66. Job Types
67. Job Types Service
 “LRP”
68. Job Types Service
 “LRP” Command “Batch”
69. Service Supervision
70. Service Supervision
71. Service Supervision
72. Service Supervision
73. Orthogonal Concerns
74. Orthogonal Concerns 1. Service discovery
75. Orthogonal Concerns 1. Service discovery 2. Health checks
76. Orthogonal Concerns 1. Service discovery 2. Health checks 3. Logging/metrics
77. Orthogonal Concerns 1. Service discovery 2. Health checks 3. Logging/metrics 4. Networking, SDN
78. Orthogonal Concerns 1. Service discovery 2. Health checks 3. Logging/metrics 4. Networking, SDN 5. Security policy
80. You should be able to throw together a cluster manager But I also believe you should be able to Throw Together Anything (tm)
85. Cmd.io Demo
86. slack.gliderlabs.com #cmd