Briding DevOps Islands with Pantry

A talk I gave at Mountain West DevOps 2014 announcing Pantry, the DevOps Pipeline Framework!

http://pantryops.org/
http://collectiveidea.com/

Transcript

1. collectiveidea.com Bridging DevOps Islands with Pantry Jason Roelofs jasonroelofs.com @jasonroelofs
2. collectiveidea.com What is Pantry? • Plugin-based • Client / Server • Asynchronous • Remote Execution Framework
3. collectiveidea.com http://mlpimgmacros.tumblr.com/post/27072884189/pinkie-pie-fascinating-image-macro
4. collectiveidea.com Let me try that again
5. collectiveidea.com What is Pantry? • DevOps Pipeline/Automation Framework • Use your tools of choice • Convention over Configuration
6. collectiveidea.com Pantry is not itself a configuration tool
7. collectiveidea.com Pantry Core Communication and Execution Framework
8. collectiveidea.com Pantry Chef Minimal Chef Server / Chef Client Replacement
9. collectiveidea.com Our Problem App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info All Chef All Different Keep adding more! App Cookbooks Application Info
10. collectiveidea.com Solutions? • Chef Server • Berkshelf • Librarian
11. collectiveidea.com Chef Server App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info Organizations don’t share.
12. collectiveidea.com Berkshelf • ??? • berkshelf and berkshelf-api • Unreliable
13. collectiveidea.com Librarian • Only cursory glance • Still just a Chef-only solution • Other tools coming into play
14. collectiveidea.com Drop Chef? • Throwing away thousands of hours of work • Years of knowledge • No guarantee other tools do this better
15. collectiveidea.com 10,000 Foot View of DevOps
16. collectiveidea.com State of DevOps • Lots of great, proven tools • Open and helpful communities • Lots of custom development still required
17. collectiveidea.com State of DevOps The Far Side by Gary Larson
18. collectiveidea.com New Stack Questions • Where do I store my data? • How do I provision new servers? • How do I configure the servers? • How do I share configurations? • How do I keep all of this secure?
19. collectiveidea.com Enter Pantry
20. collectiveidea.com Data Storage • SCM of Choice • Pantry Server • File System, no databases
21. collectiveidea.com Provision / Configure • Pantry Plugins • Configure with Chef with Pantry Chef
22. collectiveidea.com Sharing Configs • Get this for free! • No artificial access limitations • Doesn’t mean Pantry isn’t secure
23. collectiveidea.com Keeping it Secure • Yes
24. collectiveidea.com Using Pantry • gem install pantry • gem install [pantry plugin] • pantry-server • pantry-client • pantry
25. collectiveidea.com Dependencies • Ruby 2.0+ • ZeroMQ 3.x • ZeroMQ 4.x + libsodium • Plans for omnibus installer
26. collectiveidea.com Using Pantry :: Server • One Server Multiple Clients • All communication through Server
27. collectiveidea.com Using Pantry :: Client • Identity • Application • Environment • Roles
28. collectiveidea.com Using Pantry :: CLI • Run Commands in a Pantry Network • All, Some, or One Client ! • pantry -a my_app -e production status • pantry -a my_app -e staging chef:run
29. collectiveidea.com Using Pantry :: Plugins • gem install [plugin] • Server Commands • Client Commands
30. collectiveidea.com Our Problem App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info App Cookbooks Application Info All Chef All Different Keep adding more! App Cookbooks Application Info
31. collectiveidea.com Pantry Chef Cookbooks App1 Application Info App 2 Application Info App 3 Application Info App 4 Application Info App 5 Application Info Server Cookbooks App 5 Application Info Client Cookbooks App 3 Application Info Client Cookbooks App 2 Application Info Client
32. collectiveidea.com Pantry Chef • No account/users/permissions management • No cookbook versioning • Lazy cookbook dependencies • Use Chef configuration files (roles, envs)
33. collectiveidea.com Pantry Chef • pantry chef:cookbook:upload • pantry -a app chef:environment:upload • pantry -a app chef:role:upload • pantry -a app chef:data_bag:upload • pantry -a app chef:run
34. collectiveidea.com Security
35. collectiveidea.com Security • Don’t build your own • Getting it wrong has dire consequences • What is and isn’t secure?
36. collectiveidea.com Security in Pantry • The default is … None! • Plain text communication • Tool like socat and stunnel recommended
37. collectiveidea.com OpenSSL? • The defacto standard, but… • So many protocols and ciphers available • Best Practices are regularly changing • No built-in support in ZeroMQ
38. collectiveidea.com CurveZMQ • ZeroMQ 4.x • Implementation of CurveCP • Configure Pantry with “security: curve”
39. collectiveidea.com CurveCP • http://curvecp.org/ • Daniel J Bernstein (djb) • TCP replacement • Using Curve 25519 encryption
40. collectiveidea.com Curve 25519 • Elliptical Curve Cryptography (ECC) • Uses the prime number 2^255 - 19 • Uninfluenced by NIST, NSA, etc • ArsTechnica on ECC • http://cr.yp.to/ecdh.html
41. collectiveidea.com CurveZMQ • Improved implementation of CurveCP • NaCl and libsodium • Prevents many common attacks • Still young, not yet vetted • http://curvezmq.org
42. collectiveidea.com • Diffie-Hellman • Requires knowing all of the keys Attacks: Man in the Middle
43. collectiveidea.com Attacks: Packet Replay • Numbers Used Once (nonce) • Old nonce packet is immediately discarded
44. collectiveidea.com Attacks: Alter / Forge Packet • Any change invalidates the whole packet • Valid packet creation requires secret key
45. collectiveidea.com Attacks: Amplification • Recent DNS and NTP DDoS attacks • Initial handshake packet bigger than reply
46. collectiveidea.com Attacks: Key Theft • Stream encrypted with transient keys • Transient keys discarded at end of connection • Can only brute-force a saved stream
47. collectiveidea.com Attacks: Client Identification • Initial handshake with transient keys • Client identification never sent plain text
48. collectiveidea.com Attacks: Denial of Service • Memory starvation attacks • Only allocate memory on valid connection
49. collectiveidea.com CurveZMQ in Pantry • Clients and CLI need three keys • Public, Private, Server Public Key • Server has list of valid Client public keys • Server generates new Client key pairs
50. collectiveidea.com Is Pantry Secure? • Yes, but opt-in • Use at own risk • Improvements coming regularly
51. collectiveidea.com Recap • Pantry is a DevOps Pipeline Framework • Remove the tedium from DevOps • Pantry and Pantry Chef out now • More on the way!
52. collectiveidea.com ZeroMQ • IS AWESOME! • Because socket programming sucks • Built-in communication topologies
53. collectiveidea.com Celluloid • IS AWESOME! • Because concurrency sucks • Actors, Futures, Conditionals, etc • Celluloid-ZMQ
54. collectiveidea.com pantryops.org ! github.com/pantry