Spinup: Yet another fruitfly tool

The slides I did for a talk at a Reykjavík AWS Meetup. Telling the story of a tool I wrote as an early EKS adopter.

Transcript

1. Spinup Easiest way to get up and running with EKS? Steinn Eldjárn Sigurðarson Co-Founder @ GRID June 2019
2. Obligatory self-aggrandizing slide • Backend dev for most of my life • Love stages 1, 2, 3, and 7 of DevOps toolchain • Love and hate both Python and AWS • Used to run a big™ AWS stack
3. EKS?
4. EKS? Amazon Elastic Container Service for Kubernetes
5. kubernetes?
6. kubernetes? Jeez, do I have to explain everything?
7. Kubernetes • container orchestration system • more efficient resource management • application oriented infrastructure • …it's pretty dreamy!
8. My kubernetes story? • love immutable infrastructure • docker containers for everything since 2014 • (same time as k8s was announced)
9. My kubernetes story? • Ran my first clusters using kops • upgrading was scary • everything was immutable except k8s (yes I'm an idiot)
10. EKS to the rescue!
11. EKS to the rescue? We travel back in history, all the way to September 2018 • EKS cluster creation in console • Nodes added via CloudFormation
12. EKS to the rescue? • Ok, so not quite what I expected • But: • This was 2018: time for me to (finally) study CloudFormation
13. You promised us a tool!
14. spinup A story about a boy and his CloudFormation • My company shuts down Reykjavík office! • I needed to handover infrastructure • Why not...
15. CloudFormation* all the things! *Is CloudFormation even a verb?
16. spinup Reasons / Motivation: • I hate YAML and JSON • Found a nice python library: troposphere • Wanted CF with: ◦ VPC ◦ EKS ◦ RDS ◦ ElasticSearch ◦ Redis
17. spinup Acceptance Criteria: • Ability to spin up a new *everything in as few commands as possible • Generate custom CF • Use stack outputs to update local kubectl • Install k8s resources (yaml template munging)
18. spinup: the plan $ spinup mybrandnewstack [--options..] formation.json Wrote CF JSON to: 'formation.json' .. Now use the CloudFormation web console to create a stack from this json https://eu-west-1.console.aws.amazon.com/cloudformation/home?re gion=eu-west-1#/stacks?filter=active $ spinup kubeconfig mybrandnewstack ... $ spinup finish_setup mybrandnewstack ...
19. spinup Great success! • Left Takumi in October 2018 • Everything migrated to a brand new spinup-derived stack • Exposed all the skeletons in my closet • Made documenting infrastructure much easier
20. spinup More success! • Used it again to create new infrastructure for GRID • Convenient for new devs to get up and running • Still need to edit CM's with IAM -> k8s user mapping • Adding new features possible but … • Infrastructure is full of messy details!
21. spinup: reality $ history | grep spinup 10431 spinup formation --help 10432 spinup formation --eks --postgres --redis grid2019 10434 vi /Users/ses/w/spinup/spinup/formation.py 10435 vi spinup/formation.py 10438 vi spinup/__init__.py 10441 spinup formation --eks --postgres --redis grid2019 10464 spinup 10466 vi spinup/__init__.py 10469 git add spinup/tpl/* 10496 spinup kubeconfig grid 10500 cat spinup/tpl/traefik/flahaha
22. spinup So where does that leave us? • YADT (yet another deprecated tool) • There are better, more mature tools out there (eksctl, I'm looking at you!) • Don't force everything into a single CF stack. Just don't. • Feel free to check out the repo and troposphere is cool, you might learn something: github.com/steinnes/spinup
23. Thanks! Feel free to educate me about all things cloud, and if you'd like to learn more about GRID, grab me after the talks :-)