Operations Innovation from Startup to Growth

Operations is critical to delivering your product to the market. In a growing product company getting consistent and reliable operations right is crucial to success. But when you're innovating all the time on the product, is it right to innovate in operations too? The temptation is strong, but what's the right balance?

I'll talk about how we've scaled operations successfully over the 8 year growth of our company, and how we made choices along the way.

Transcript

1. Growth and Innovation “Good" Life Choices
2. Growth Is Hard • Focus on your mission • What numbers matter? • You can only herd so many cats • "Weird tech" costs more when it goes wrong • Open Source isn't free - you still have to operate it • Boring is awesome • Don't rewrite the world if you can avoid it
3. Me • Jess Szmajda • CTO, Optoro • Started in March 2010 (!) • Built everything from the ground up • My background is Ops
4. What Hasn’t Changed • Ruby, Rails • Rich front-end technology • Agile practice - continuous improvement • Focus on automation
5. Early Days (2010 - 2013) Just me! AWS Puppet Pros Cons Well-understood pattern w/ AWS Good community around Puppet Could buy help Automation drove focus Worked for 3 years! Brittle, barely maintained puppet master Our config wasn’t built for flexibility / scale I was the whole show
6. We’re growing?! We’re growing! Wait, I can’t do everything anymore!
7. I Guess We Need Ops (2013 - 2015) Ops Team (1 - 4) AWS Chef ELK, Sensu Pros Cons Reusability Testing, CI Community Clear demark Rewrite the (small) world Delegating & trusting is hard
8. AWS is expensive… And we have this great ops team…
9. R O I
10. Move to Hybrid Cloud (2015 - 2016) Ops Team (4-7) METAL Chef Triton Local link to AWS Pros Cons $avings - 65% over 2 years Still public cloud tech CNs are dumb (easy) 6 months of conversion I guess we all have to learn Solaris again (Zones, LX, KVM, ) Still lots of bespoke work
11. Autonomous Teams Vision Infrastructure Applications Tools Infrastructure Team Product Teams Dev Tools Team
12. Docker is everywhere
13. Release the Services! (2016 - 2017) Ops Team (4-7) METAL Chef, Triton, Docker The MetaCluster Hammer Pros Cons Enabled SOA Leveraged SDC Decentralized Infra Higher leverage ops DevOps! Innovation time Figuring out opinions Opportunity Cost
14. Geez we’re still spending a lot of time on the apps…
15. Autopilot Pattern (2017 - now) Ops Team (5-7) METAL Chef, Triton, Docker, MC Hammer Vault, Consul, Prometheus, Traefik Pros Cons No more Data Bags Security++ Fully automatic luxury space operations Many core services Hard to understand dependencies Complexity
16. The Future?!
17. Future • More Autopilot - CockroachDB, Certbot • ESB / Kafka • Dependency graphing • Multi datacenter (active/active) • Kubernetes? (“it’s eating the world” -Brock Wilcox) • SRE Model
18. Thank You • Jess Szmajda • CTO, Optoro • @jszmajda • meetup.com/Leaders-In-Tech