Virtualization with Ganeti - Setting up your private cluster on commodity hardware

Pycon UK 2013, My talk on showing a simple outline on setting up a ganeti cluster. (without installtion details)

Virtualization with Ganeti - Setting up your private cluster on commodity hardware

Transcript

1. Virtualization with Ganeti Setting up a private cluster on commodity hardware Pranjal Mittal @pramttl Pycon UK 2013 Some Images / Content Courtesy - Lance Albertson, OSUOSL
2. About Me OSU Open Source Lab Ganeti Web Manager Google Summer of Code 2013 Indian Institute of Technology, Varanasi @pramttl on Github, Twitter, ..
3. Overview • What is Ganeti • Ganeti components • A visual outline to setup a ganeti cluster on your hardware • A quick glimpse at Ganeti CLI • Ganeti Web Manager • Why would you prefer Ganeti over other cloud software.
4. Open source cloud computing software GANETI
5. What is Ganeti? A cluster management software Developed at Google & open sourced in 2007 Written in Python and Haskell
6. Ganeti Components Python Haskell LVM / DRBD Hypervisor
7. To set up a Ganeti Cluster we take some...
8. Nodes • A networked set of Physical machines. NODE NODE NODE
9. Nodes • A networked set of Physical machines. • Can be added/removed at will from a cluster. NODE NODE NODE
10. Nodes • A networked set of Physical machines. • Can be added/removed at will from a cluster. • Every cluster has a master-node, which is used to control the cluster. NODE NODE NODE
11. Instances • Virtual machines • This is what users interact with without knowing about the nodes behind. INSTANCE INSTANCE INSTANCE INSTANCE NODE NODE NODE
13. • One instance runs on only one node at a time called a primary node 2 1 3 4 NODE 3 NODE 2 NODE 1
14. When using DRBD with ganeti.. • One instance runs on only one node at a time called a primary node • A backup copy or a mirror is created for each instance on a secondary node. (on demand) 2 1 3 4 NODE 3 NODE 2 NODE 1 2 1 3 Mirror copies
15. 2 1 3 4 We are making the cluster “fault tolerant” • One instance runs on only one node at a time called a primary node • A backup copy or a mirror is created for each instance on a secondary node. (on demand) • DRBD handles the live sync up of the mirror with the corresponding primary instance. NODE 3 NODE 2 NODE 1 Mirror copies 2 1 3
16. What do I mean by “fault tolerant”? • Lets say a physical node goes down. 2 1 NODE 1 3 4 NODE 3 NODE 2 Mirror copies 2 1 3
17. What do I mean by “fault tolerant”? • Lets say a physical node goes down. • All the instances attached to it also go down. 2 1 NODE 1 3 4 NODE 3 NODE 2 Mirror copies 2 1 3
18. What do I mean by “fault tolerant”? • Lets say a physical node goes down. • All the instances attached to it also go down. • Ganeti helps us avoid the trouble. The affected instances can be failovered to the running secondary node. 2 1 NODE 1 3 4 NODE 3 NODE 2 Mirror copies 2 1 3
19. Wait a minute - What is a “failover”? • Waking up the instance mirror copy already present on the secondary nodes when the original instance fails. NODE 1 NODE 3 NODE 2 2 1 3 Mirror copies 2 1 3 4
20. Back to our cluster with four instances. • The instances are up and running again in a jiffy. Failover done in a few seconds. 4 3 3 NODE 3 NODE 2 2 1
21. Back to our cluster with four instances. • The instances are up and running again in a jiffy. Failover done in a few seconds. • So the problems in the physical hardware are invisible to the end user, even if a node fails the cluster runs without losing any data. This is fault tolerance. 3 4 3 NODE 3 NODE 2 2 1
22. Ganeti Commands • gnt-cluster ... • gnt-node ... • gnt-instance … More command groups: gnt-* debug, os, job, backup, group 1. commands are grouped into various sets 2. most commands are used on the master-node
23. A few basic ganeti commands 1. Initializing a Ganeti Cluster gnt-cluster init --(some parameters) 2. Adding nodes to the cluster gnt-node add --(some parameters) 3. Create an instance on the cluster gnt-instance add --(some parameters) 4. Listing all nodes or instances in a cluster gnt-node list gnt-instance list
24. Don't like the command line?
25. Ganeti Web Manager (GWM) A web-GUI interface to Ganeti built using Django Does most of the stuff you could do with the Ganeti CLI Interacts with Ganeti via the Ganeti Rest API. OSUOSL, initiated project, 18+ Releases, 20+ Contributors
26. Courtesy: Lance Albertson, @ramereth
27. 5 main reasons you would prefer Ganeti..
28. • Easy to install and maintain • Reliable, fault-tolerant even on commodity hardware • Faster: Uses local block storage instead of NFS • Maintained by Google Open Source team. • Because of the smart guys who use it… (next)
29. Who uses Ganeti? • Google - Corporate Computing Infrastructure • grnet.gr - Greek Research & Technology Network • fsffrance.org - Free Software Foundation France • debian.org - Debian • osuosl.org - Oregon State University Open Source Lab - The most awesome Open Source Lab.
30. Who uses Ganeti? • Google - Corporate Computing Infrastructure • grnet.gr - Greek Research & Technology Network • fsffrance.org - Free Software Foundation France • debian.org - Debian • osuosl.org - Oregon State University Open Source Lab - The most awesome Open Source Lab.
31. Some OSL hosted projects running on a Ganeti cluster • Busybox • Inkscape • Python Software Foundation • OpenMRS • phpBB • OSGeo • Yum
32. Courtesy: Lance Albertson, @ramereth
33. Useful Links Ganeti Source: http://code.google.com/p/ganeti/wiki/SourceCode Installation docs: http://docs.ganeti.org/ganeti/current/html/install.html Ganeti common com’nds: http://wiki.osuosl.org/public/ganeti/common_commands Ganeti Rest API docs: http://docs.ganeti.org/ganeti/current/html/rapi.html Ganeti Web Manger docs: https://ganeti-webmgr.readthedocs.org/en/latest/ Ganeti Web Manager Source: https://github.com/osuosl/ganeti_webmgr Run a Virtual Ganeti Cluster: https://github.com/ramereth/vagrant-ganeti
34. CREDITS [1] Lance Albertson, Director, Lead Systems Administrator, OSU | Open Source Lab [2] Kenneth Lett - Senior Software Engineer, Open Source Lab, OSU | Open Source Lab [3] Chance Zibolski, OSUOSL - Student developer, GWM committer [4] Piotr Banaszkiewicz, GWM Developer, GSoC 2013, 2012, Google Code-In [5] Google - For funding me. Thank you for all the suggestions, content and help.
35. THANK YOU Questions? I love them, btw. https://github.com/pramttl mittal.pranjal@gmail.com