Infrastructure In Code

Transcript

1. INFRASTRUCTURE IN CODE STRIVING FOR BETTER ABSTRACTIONS AND AUTOMATION @pyr
2. SHORT BIO Pierre-Yves Ritschard CTO @ exoscale - The leading swiss public cloud provider Open Source Developer - riemann, collectd, pallet, openbsd Architect of several cloud platforms - paper.li Recovering Operations Engineer
3. Cloud History 101 Shedding the cognitive load Let cloudstack help you
4. CLOUD HISTORY 101 Getting rid of the physical world
5. Three decades of infrastructure Where we are today What went wrong
6. THREE DECADES OF INFRASTRUCTURE
7. In the 90's we had switches, routers and servers and they all had funny names
8. In the 2000's we had switches, routers ,hypervisors and virtual machines Tangentially, configuration management became a thing
9. Now we have commoditized infrastructure In common speak we have moved from machines to instances
10. WHERE WE ARE TODAY
11. Programmable provisioning
12. Programmable decomissioning
13. Ubiquitous IAAS
14. Much much simpler capacity planning
15. Negligible provision times
16. So, that's it ?
17. « Welp, looks like we done fixed infrastructure once and for all » - no one, ever
18. WHAT WENT WRONG
19. Credit: Jochen Smolka, Lund University
20. We still apply a big ball of mud approach to infrastructure instances are still black boxes full of mutable state
21. As a corollary, we keep a mapping of services to instances Although admittedly, configuration management helps
22. We insist on carrying over concepts from the physical world c r e a t e I P F o r w a r d i n g R u l e ... are we crazy ?
23. snowflakes !
24. SHEDDING THE COGNITIVE LOAD Embracing simplicity
25. Better abstractions Better automation
26. BETTER ABSTRACTIONS
27. Complexity is the enemy of scalability
28. Complexity is the enemy of security
29. We need to change our trust model No more NAT No more Address association No more volumes
30. Simpler abstractions Security groups Snapshots
31. BETTER AUTOMATION
32. Stop treating instances as the base unit of reasoning mitigates the risk of ending with a big ball of mud
33. Nodes are part of homogeneous groups (clusters)
34. Nodes should avoid configuration drift Strive for immutable infrastructure
35. Integrate IAAS features in automation by storing hints for configuration management when provisioning instances
36. Use your IAAS as a low level config registry let clusters discover themselves
37. LET CLOUDSTACK HELP YOU A short guide to better infrastructure
38. Abstractions Tooling Library support
39. ABSTRACTIONS
40. Keypairs you probably use them already
41. User Data c l o u d - i n i t is a great tool
42. Basic networking suffices for almost all use cases More scalable routing Great firewalling abilities through security groups
43. Tags Arbitrary metadata on instances
44. Projects, Instance Groups More heavy weight abstractions
45. TOOLING
46. Puppet Resource manipulation support cloud-init integration
47. Chef knife support for bootstrapping instances cloud-init integration
48. Cloudmonkey Great interaction tool, actively maintained
49. LIBRARY SUPPORT
50. Name your language Java: jclouds Ruby: fog, cloudstack_ruby_client Python: libcloud Clojure: pallet
51. Pallet: a small introduction Configuration management, provisionning, command and control Same tool space than chef and knife, but a library Built on top of apache Jclouds Clojure
52. Pallet: simple abstractions Node specifications Server Specifications Groups
53. Node specifications ( d e f u b u n t u - n o d e ( n o d e - s p e c : n e t w o r k { : i n b o u n d - p o r t s [ 2 2 , 8 0 , 4 4 3 ] } , : i m a g e { : o s - f a m i l y : u b u n t u , : o s - v e r s i o n - m a t c h e s " 1 2 . 0 4 " } , : h a r d w a r e { : m i n - c o r e s 1 , : m i n - d i s k 1 0 , : m i n - r a m 5 1 2 } ) )
54. Server specifications ( d e f w e b - s e r v e r ( s e r v e r - s p e c : p h a s e s { : c o n f i g u r e ( p l a n - f n ( p a c k a g e " n g i n x " ) ( p a c k a g e " m y - w e b - a p p " ) ) } ) ) ( d e f l b - s e r v e r ( s e r v e r - s p e c : p h a s e s { : c o n f i g u r e ( p l a n - f n ( p a c k a g e " h a p r o x y " ) ( h a p r o x y / a d d - b a c k e n d " w e b " : s e r v e r s ( n o d e s - w i t h - r o l e : w e b ) ) ) } ) )
55. Groups ( d e f w e b ( g r o u p - s p e c " w e b " : r o l e s [ : w e b ] : e x t e n d s [ b a s e - s e r v e r w e b - s e r v e r ] : n o d e - s p e c u b u n t u - n o d e ) ) ( d e f l b ( g r o u p - s p e c " l b " : r o l e s [ : l b ] : e x t e n d s [ b a s e - s e r v e r l b - s e r v e r ] : n o d e - s p e c u b u n t u - n o d e ) )
56. Provision from the CLI l e i n p a l l e t c o n v e r g e l b 1 w e b 4 Or your Code ( c o n v e r g e { l b 1 , w e b 4 } )
57. Pallet embraces the cloudstack API ( { : k e y " p a l l e t - g r o u p " , : r e s o u r c e i d " 3 b d 6 d 1 c d - e 8 f 8 - 4 4 5 f - 8 1 8 0 - 6 4 4 9 4 6 f 1 2 e d d " , : r e s o u r c e t y p e " U s e r V M " , : v a l u e " w e b " } , { : k e y " p a l l e t - s t a t e " , : r e s o u r c e i d " 3 b d 6 d 1 c d - e 8 f 8 - 4 4 5 f - 8 1 8 0 - 6 4 4 9 4 6 f 1 2 e d d " , : r e s o u r c e t y p e { : b o o t s t r a p p e d t r u e } , : v a l u e " w e b " } , { : k e y " p a l l e t - g r o u p " , : r e s o u r c e i d " 3 b d 6 d 1 c d - e 8 f 8 - 4 4 5 f - 8 1 8 0 - 6 4 4 9 4 6 f 1 2 e d d " , : r e s o u r c e t y p e " U s e r V M " , : v a l u e { : o s - v e r s i o n " 1 2 . 0 2 " , : o s - f a m i l y : u b u n t u , : i m a g e - i d " a 1 7 b 4 0 d 6 - 8 3 e 4 - 4 f 2 a - 9 e f 0 - d c e 6 a f 5 7 5 f f f a " } } )
58. PARTING WORDS Where to go from here
59. From infrastructure as code to infrastructure as data next step: reactive infrastructure
60. More power to the controller From IAAS to COAAS
61. Resources https://palletops.com https://github.com/exoscale/pallet-exoscale-demo
62. THANK YOU ! QUESTIONS ? @pyr