Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Infrastructure In Code

Infrastructure In Code

2fcc875f98607b3007909fe4be99160d?s=128

Pierre-Yves Ritschard

November 21, 2013
Tweet

Transcript

  1. INFRASTRUCTURE IN CODE STRIVING FOR BETTER ABSTRACTIONS AND AUTOMATION @pyr

  2. SHORT BIO Pierre-Yves Ritschard CTO @ exoscale - The leading

    swiss public cloud provider Open Source Developer - riemann, collectd, pallet, openbsd Architect of several cloud platforms - paper.li Recovering Operations Engineer
  3. Cloud History 101 Shedding the cognitive load Let cloudstack help

    you
  4. CLOUD HISTORY 101 Getting rid of the physical world

  5. Three decades of infrastructure Where we are today What went

    wrong
  6. THREE DECADES OF INFRASTRUCTURE

  7. In the 90's we had switches, routers and servers and

    they all had funny names
  8. In the 2000's we had switches, routers ,hypervisors and virtual

    machines Tangentially, configuration management became a thing
  9. Now we have commoditized infrastructure In common speak we have

    moved from machines to instances
  10. WHERE WE ARE TODAY

  11. Programmable provisioning

  12. Programmable decomissioning

  13. Ubiquitous IAAS

  14. Much much simpler capacity planning

  15. Negligible provision times

  16. So, that's it ?

  17. « Welp, looks like we done fixed infrastructure once and

    for all » - no one, ever
  18. WHAT WENT WRONG

  19. Credit: Jochen Smolka, Lund University

  20. We still apply a big ball of mud approach to

    infrastructure instances are still black boxes full of mutable state
  21. As a corollary, we keep a mapping of services to

    instances Although admittedly, configuration management helps
  22. We insist on carrying over concepts from the physical world

    c r e a t e I P F o r w a r d i n g R u l e ... are we crazy ?
  23. snowflakes !

  24. SHEDDING THE COGNITIVE LOAD Embracing simplicity

  25. Better abstractions Better automation

  26. BETTER ABSTRACTIONS

  27. Complexity is the enemy of scalability

  28. Complexity is the enemy of security

  29. We need to change our trust model No more NAT

    No more Address association No more volumes
  30. Simpler abstractions Security groups Snapshots

  31. BETTER AUTOMATION

  32. Stop treating instances as the base unit of reasoning mitigates

    the risk of ending with a big ball of mud
  33. Nodes are part of homogeneous groups (clusters)

  34. Nodes should avoid configuration drift Strive for immutable infrastructure

  35. Integrate IAAS features in automation by storing hints for configuration

    management when provisioning instances
  36. Use your IAAS as a low level config registry let

    clusters discover themselves
  37. LET CLOUDSTACK HELP YOU A short guide to better infrastructure

  38. Abstractions Tooling Library support

  39. ABSTRACTIONS

  40. Keypairs you probably use them already

  41. User Data c l o u d - i n

    i t is a great tool
  42. Basic networking suffices for almost all use cases More scalable

    routing Great firewalling abilities through security groups
  43. Tags Arbitrary metadata on instances

  44. Projects, Instance Groups More heavy weight abstractions

  45. TOOLING

  46. Puppet Resource manipulation support cloud-init integration

  47. Chef knife support for bootstrapping instances cloud-init integration

  48. Cloudmonkey Great interaction tool, actively maintained

  49. LIBRARY SUPPORT

  50. Name your language Java: jclouds Ruby: fog, cloudstack_ruby_client Python: libcloud

    Clojure: pallet
  51. Pallet: a small introduction Configuration management, provisionning, command and control

    Same tool space than chef and knife, but a library Built on top of apache Jclouds Clojure
  52. Pallet: simple abstractions Node specifications Server Specifications Groups

  53. Node specifications ( d e f u b u n

    t u - n o d e ( n o d e - s p e c : n e t w o r k { : i n b o u n d - p o r t s [ 2 2 , 8 0 , 4 4 3 ] } , : i m a g e { : o s - f a m i l y : u b u n t u , : o s - v e r s i o n - m a t c h e s " 1 2 . 0 4 " } , : h a r d w a r e { : m i n - c o r e s 1 , : m i n - d i s k 1 0 , : m i n - r a m 5 1 2 } ) )
  54. Server specifications ( d e f w e b -

    s e r v e r ( s e r v e r - s p e c : p h a s e s { : c o n f i g u r e ( p l a n - f n ( p a c k a g e " n g i n x " ) ( p a c k a g e " m y - w e b - a p p " ) ) } ) ) ( d e f l b - s e r v e r ( s e r v e r - s p e c : p h a s e s { : c o n f i g u r e ( p l a n - f n ( p a c k a g e " h a p r o x y " ) ( h a p r o x y / a d d - b a c k e n d " w e b " : s e r v e r s ( n o d e s - w i t h - r o l e : w e b ) ) ) } ) )
  55. Groups ( d e f w e b ( g

    r o u p - s p e c " w e b " : r o l e s [ : w e b ] : e x t e n d s [ b a s e - s e r v e r w e b - s e r v e r ] : n o d e - s p e c u b u n t u - n o d e ) ) ( d e f l b ( g r o u p - s p e c " l b " : r o l e s [ : l b ] : e x t e n d s [ b a s e - s e r v e r l b - s e r v e r ] : n o d e - s p e c u b u n t u - n o d e ) )
  56. Provision from the CLI l e i n p a

    l l e t c o n v e r g e l b 1 w e b 4 Or your Code ( c o n v e r g e { l b 1 , w e b 4 } )
  57. Pallet embraces the cloudstack API ( { : k e

    y " p a l l e t - g r o u p " , : r e s o u r c e i d " 3 b d 6 d 1 c d - e 8 f 8 - 4 4 5 f - 8 1 8 0 - 6 4 4 9 4 6 f 1 2 e d d " , : r e s o u r c e t y p e " U s e r V M " , : v a l u e " w e b " } , { : k e y " p a l l e t - s t a t e " , : r e s o u r c e i d " 3 b d 6 d 1 c d - e 8 f 8 - 4 4 5 f - 8 1 8 0 - 6 4 4 9 4 6 f 1 2 e d d " , : r e s o u r c e t y p e { : b o o t s t r a p p e d t r u e } , : v a l u e " w e b " } , { : k e y " p a l l e t - g r o u p " , : r e s o u r c e i d " 3 b d 6 d 1 c d - e 8 f 8 - 4 4 5 f - 8 1 8 0 - 6 4 4 9 4 6 f 1 2 e d d " , : r e s o u r c e t y p e " U s e r V M " , : v a l u e { : o s - v e r s i o n " 1 2 . 0 2 " , : o s - f a m i l y : u b u n t u , : i m a g e - i d " a 1 7 b 4 0 d 6 - 8 3 e 4 - 4 f 2 a - 9 e f 0 - d c e 6 a f 5 7 5 f f f a " } } )
  58. PARTING WORDS Where to go from here

  59. From infrastructure as code to infrastructure as data next step:

    reactive infrastructure
  60. More power to the controller From IAAS to COAAS

  61. Resources https://palletops.com https://github.com/exoscale/pallet-exoscale-demo

  62. THANK YOU ! QUESTIONS ? @pyr