A One stop solution for Puppet and Openstack - PuppetConf 13'

A One stop solution for Puppet and Openstack - PuppetConf 13'

Fe4dc5ea02ac73b9981bcc549a7a288c?s=128

Daniel Lobato García

August 23, 2013
Tweet

Transcript

  1. A one stop solution for Puppet and Openstack Daniel Lobato

    Garcia daniel.lobato.garcia@cern,ch @eLobatoss
  2. None
  3. What is CERN Between Geneva and the Jura mountains, straddling

    the Swiss- French border Mission: learn what is the universe made of and how does it work? 3
  4. Fundamental questions in physics Why do particles have mass? What

    is 96% of the universe made of? Why isn’t there anti-matter in the universe? What was the state of matter after the Big Bang? 4
  5. 8/12/2013 Document reference 5

  6. 8/12/2013 Document reference 6

  7. 8/12/2013 Document reference 7

  8. 8/12/2013 Document reference 8

  9. Current status • 270 Openstack hypervisors • 2900 virtual machines

    • 300 users • 14 Puppet masters • 6 Foreman backend nodes • Some production services migrating to our cloud – early birds 9
  10. Goals • Ramp up to 15K hypervisors – 150-200K vms

    in 2015 • Multi-site (Hungary) 10
  11. 8/12/2013 Document reference 11

  12. 8/12/2013 Document reference 12

  13. Why? • Unnecessary homebrew stack of tools • Shift to

    cloud standards with minimal customizations • High turnover – can’t teach new tools 13
  14. Why? • Symbiotic relationship with the community 14

  15. Openstack? • Modular IaaS free open source project • APIs

    ~compatible with those of Amazon 15
  16. Openstack Nova (compute) Cloud fabric controller 16

  17. Openstack Keystone (Identity) RBAC Integrated with LDAP Multiple auth* methods

    17
  18. Openstack Glance (Images) Discovery, registration, delivery of images 18

  19. Openstack Horizon (Dashboard) 19

  20. Modules • Puppet definitions for every use case you can

    imagine. • Dynamic environments • Hadoop node • Openstack hypervisor • … you name it 20
  21. 21 Workflow..?

  22. Modules and Git • Manifests and hieradata are version controlled

    22
  23. 23 Git workflow Puppet masters

  24. 24 Easy cherry pick

  25. 25 Git workflow

  26. 26 Git workflow Jens ‘Puppetfiles’ Separate repositories Makes environments from

    YAML config files Puppet master rsync
  27. 27 upstream.yaml --- default: production notifications: ai@cern.ch overrides: modules: foreman:

    upstream puppet: upstream ‘Puppetfiles’
  28. Foreman • Lifecycle management tool for VMs and physical servers

    • External Node Classifier – tells the puppet master what a node should look like 28
  29. 29

  30. 30

  31. Power operations & Foreman 8/12/2013 Document reference 31 Foreman Proxy

    Physical box IPMI Physical box IPMI Physical box IPMI VM VM VM Openstack Nova API
  32. Openstack VM creation 8/12/2013 Document reference 32

  33. Openstack VM creation 8/12/2013 Document reference 33

  34. Openstack VM creation 8/12/2013 Document reference 34

  35. Scalability experiences • Split up services • Puppet – critical

    vs non critical 35 12 backend nodes Batch 4 backend nodes Interactive
  36. Scalability experiences • Foreman – split into different services 36

    ENC Reports processing UI/API Load balancer 9443 – UI/API 9444 – Reports 9445 – ENC …
  37. Scalability experiences • Autoscale via alarms (Heat) • Define situations

    (i.e: load threshold..) • Spin up VMs as needed 37
  38. Scalability guidelines 38

  39. github.com / cernops 39

  40. 40

  41. None