A One stop solution for Puppet and Openstack - PuppetConf 13'

Transcript

1. A one stop solution for Puppet and Openstack Daniel Lobato

   Garcia daniel.lobato.garcia@cern,ch @eLobatoss
2. None

3. What is CERN Between Geneva and the Jura mountains, straddling

   the Swiss- French border Mission: learn what is the universe made of and how does it work? 3

4. Fundamental questions in physics Why do particles have mass? What

   is 96% of the universe made of? Why isn’t there anti-matter in the universe? What was the state of matter after the Big Bang? 4

5. 8/12/2013 Document reference 5

6. 8/12/2013 Document reference 6

7. 8/12/2013 Document reference 7

8. 8/12/2013 Document reference 8

9. Current status • 270 Openstack hypervisors • 2900 virtual machines

   • 300 users • 14 Puppet masters • 6 Foreman backend nodes • Some production services migrating to our cloud – early birds 9

10. Goals • Ramp up to 15K hypervisors – 150-200K vms

    in 2015 • Multi-site (Hungary) 10

11. 8/12/2013 Document reference 11

12. 8/12/2013 Document reference 12

13. Why? • Unnecessary homebrew stack of tools • Shift to

    cloud standards with minimal customizations • High turnover – can’t teach new tools 13

14. Why? • Symbiotic relationship with the community 14

15. Openstack? • Modular IaaS free open source project • APIs

    ~compatible with those of Amazon 15

16. Openstack Nova (compute) Cloud fabric controller 16

17. Openstack Keystone (Identity) RBAC Integrated with LDAP Multiple auth* methods

    17

18. Openstack Glance (Images) Discovery, registration, delivery of images 18

19. Openstack Horizon (Dashboard) 19

20. Modules • Puppet definitions for every use case you can

    imagine. • Dynamic environments • Hadoop node • Openstack hypervisor • … you name it 20

21. 21 Workflow..?

22. Modules and Git • Manifests and hieradata are version controlled

    22

23. 23 Git workflow Puppet masters

24. 24 Easy cherry pick

25. 25 Git workflow

26. 26 Git workflow Jens ‘Puppetfiles’ Separate repositories Makes environments from

    YAML config files Puppet master rsync

27. 27 upstream.yaml --- default: production notifications: ai@cern.ch overrides: modules: foreman:

    upstream puppet: upstream ‘Puppetfiles’

28. Foreman • Lifecycle management tool for VMs and physical servers

    • External Node Classifier – tells the puppet master what a node should look like 28

29. 29

30. 30

31. Power operations & Foreman 8/12/2013 Document reference 31 Foreman Proxy

    Physical box IPMI Physical box IPMI Physical box IPMI VM VM VM Openstack Nova API

32. Openstack VM creation 8/12/2013 Document reference 32

33. Openstack VM creation 8/12/2013 Document reference 33

34. Openstack VM creation 8/12/2013 Document reference 34

35. Scalability experiences • Split up services • Puppet – critical

    vs non critical 35 12 backend nodes Batch 4 backend nodes Interactive

36. Scalability experiences • Foreman – split into different services 36

    ENC Reports processing UI/API Load balancer 9443 – UI/API 9444 – Reports 9445 – ENC …

37. Scalability experiences • Autoscale via alarms (Heat) • Define situations

    (i.e: load threshold..) • Spin up VMs as needed 37

38. Scalability guidelines 38

39. github.com / cernops 39

40. 40

41. None