From Chef 0.6 to 11: Engine Yard steps into a time machine

Over the past eight years Engine Yard has built a highly successful PaaS with chef as a core tool. However, our growth rate lead to insufficient automated testing, brittle integration patterns, and unintended customer behaviors. With a slow, breakage prone development environment, our tools and libraries(including Chef) were left to stagnate, becoming a burden to engineers and customers alike.

What started as a small experiment in bringing contemporary chef ideas and practices snowballed into an integral piece of a complete product overhaul, with plenty of bumps and detours along the way. Come hear what we’ve learned during this process. We’ll share what we’ve learned from the past and the tricks we’ve picked up along the way. We will also discuss what our stack looks like these days and the challenges and improvements we see in our future.

Transcript

1. From Chef 0.6 to 11 Engine Yard steps into a time machine
2. Ryan Souza
3. Engine Yard
5. PaaS Infrastructure Chef Orchestration
6. Infrastructure Chef Orchestration
7. Journey Past Present Future
8. The Past
9. 2009 Bleeding edge Young community
10. Grew too fast Didn’t keep up
11. Anti-Patterns
12. Highly coupled cookbooks
13. Stagnated
14. Ruby 1.8.6 Chef 0.6
15. “Engine Yard Chef”
16. No community guides No community cookbooks
17. Testing
18. Wanted to be E2E Faked data
19. Slow Flakey Low coverage
20. Manual Testing
23. Anti-Patterns
24. Out of date environment Coupled cookbooks Manual testing
25. New
26. New Multi-cloud provisioner
27. New Javascript UI
28. New Full-service API
29. Cookbooks
30. New Cookbooks
31. Anti-Anti-Patterns
32. Learn from the community Focused cookbooks Good testing
33. The Now
34. Community Cookbooks Solid foundation Wrapper cookbooks Integrate the community
35. Patterns Learn from the community Focused cookbooks Good testing
36. Tools
37. Vagrant Berkshelf Test Kitchen
39. Fast easy development Not just Chef
40. Can be too fast Uncaught race conditions
41. Berkshelf
42. One cookbook per repo Individual components Community cookbooks
43. Patterns Learn from the community Focused cookbooks Good testing
44. 2.0 dependency issues Moving to 3.0
45. Test Kitchen
47. Quick & Easy Component-sized Cookbook’s API
48. kitchen-ec2 Encrypted creds Busy-waiting worker :\
49. Careful
50. Fake data Not really integration testing… Definitely not E2E
51. We needed more
53. Real API E2E acceptance tests
55. Real-life scenarios Confidence Assurance
56. Patterns Learn from the community Focused cookbooks Good testing
57. Workflow
58. One Github repo per cookbook Pull requests w/ Travis status
62. Anti-Anti-Patterns Learn from the community Focused cookbooks Good testing
63. Coming up Giving back
64. Thanks!
65. Q & A