How to build your own cloud while staying in business

We're deeply sorry if you decided to deploy a new product in 2016. Not only are systems getting more and more complex, but you're spoilt by the choice between oodles of infrastructure solutions. But do not fear as Docker is here. Containarization set out to redeem you from all your ops hassle. But since there are still minor inconveniences like Networking, Service Discovery, Persistence of even Security to be taken care of, your job isn't done with containerizing everything. All of the above are fundamental architectural decisions you shouldn't approach reckless.

While Hashicorp tools are no silver bullet, they allow for pragmatic solutions while you still remain at the wheel. We want to show you how to setup an infrastructure suited for modern software needs.

Disclaimer: We're not affiliated by any means with Hashicorp. We just happen to like their tools a lot ;-)

Note: This is the talk @bascht and I gave at microxchg 2016.

Transcript

3. HOW TO BUILD YOUR OWN CLOUD WHILE STAYING IN BUSINESs?
4. Hey, microxchg 2016!
5. Why are we doing this talk?
6. Infrastructure Because
7. Docker Because
8. We have to GTD™ Because
9. We're loving it. Because
10. & Challenges Opportunities
11. Challenges Increasing complexity Multiple environments in different stages Multi/Hybrid cloud and evolving infrastructure New technologies Communication & Documentation
12. Opportunities Infrastructure (not just servers) on demand Flexible APIs for all infrastructure components Affordable but powerful hardware and services Virtualization/Containerization
13. Why are we doing this talk?
14. Who are WE anyway? WE
15. WE Sebastian Schulze Dirk Breuer Freelance Freelance eloper eration Dev Op s
16. DevOps
19. Software Engineering
20. Infrastructure Engineering
21. TOoLS
22. Docker Ansible Puppet Scripting AWS Mesos Virtualization ELK Vagrant curl | sudo bash
23. THE ONE ToOL™
24. “ is designed to handle many of the mundane tasks of developers and allow them to get to the important, unique part of their application. […] they host the developer’s application, back it up, monitor it and scale it – so the developer doesn’t have to. They handle logging, groups, users, session management […]” Solution at the end of the talk.
25. ? THE ONE ToOL™
26. Monty Python and the Holy Grail © Monty Python
27. The ADEQUATE toOl for your problem
28. THE FOLLOWING SLIDES HAVE BEEN APPROVED FOR ALL AUDIENCES BY THE BOARD OF BUZZWORDERY, INC. THE VIEWS ADVERTISED HAVE BEEN RATED ® SUBJECTIVE S www.yakshed.org CEOS REQUIRE ACCOMPANYING CERTIFIED DEVOP WHAT YOU ARE ABOUT TO SEE MAY CONTAIN SUBJECTIVE VIEWS OF THE SPEAKERS
29. Good Ol' Days
30. $ vagrant provision
31. $ puppet apply
32. FIN
34. Development $ cd million-dollar-app $ rake build Created package "million-dollar-app-0.8.20.deb"
35. Code COMPILE ArtiFact
36. Operations package { 'million-dollar-app': ensure => latest } ~> service { 'blazing-fast-webserver': ensure => running }
37. Code COMPILE ArtiFact
38. SERVER ARTIFact
39. SERVER SERVER SERVER
40. SERVER SERVER SERVER Network Database Monitoring Database Storage Network LOGgING
41. SERVER SERVER SERVER Network Database Monitoring Database Storage Network LOGgING ARTIFactS
42. Behold The Monolith © http://jordangrimmer.deviantart.com/
43. Code COMPILE ARTIFact Code COMPILE Code ARTIFact Code COMPILE ARTIFact COMPILE Code Code COMPILE ARTIFact
45. Caveats ! Increasing complexity ! Changing running systems is still hard ! Bootstrapping is still a mess ! Every infrastructure component covered?
46. Infrastructure AS COdE
47. What's the Problem Most of the time you want to run a business, not a platform Adapt your infrastructure to changing business needs Be able to evaluate new technologies
48. PaaS Checklist Dev-Production parity Repeatable builds Track artifacts Immutable infrastructure Cover the entire stack
49. Repeatable builds
50. –packer.io “[…] creating machine and container images for multiple platforms from a single source configuration.” Packer
51. { "builders": [{ "type": "docker", "image": "ubuntu", "commit": true }], "provisioners": [{…}], "post-processors": [[ { "type": "docker-import", "repository": "hotstartup/million-dollar-frontend", "tag": "{{user `tag`}}" }, { "type": "docker-push" } ]] }
52. $ packer build
53. Infrastructure-Code
54. –terraform.io “Build, Combine, and Launch Infrastructure” Terraform
55. resource "aws_security_group" "db" { name = "db" ingress {} egress {} } resource "aws_db_instance" "db" { identifier = "million-dollar-app-db" engine = "postgres" name = "${var.db_name}" username = "${var.db_username}" password = "${var.db_password}" vpc_security_group_ids = ["${aws_security_group.db.id}"] }
56. $ terraform plan $ terraform apply
57. And now KISS…
58. –nomadproject.io “A Distributed, Highly Available, Datacenter-Aware Scheduler” Nomad
59. job "million-dollar-app" { region = "dach" datacenters = ["eu-fra-01", "eu-ber-01", "eu-ber-02"] type = "system" group "frontends" { count = 20 task "frontend" { driver = "docker" config { image = "hotstartup/million-dollar-frontend" } } } }
60. SERVER SERVER SERVER Network Database Monitoring Database Storage Network LOGgING Dev-Production parity Repeatable builds Track artifacts Immutable infrastructure Service Discovery
61. “Zimki is designed to handle many of the mundane tasks of developers and allow them to get to the important, unique part of their application. […] they host the developer’s application, back it up, monitor it and scale it – so the developer doesn’t have to. They handle logging, groups, users, session management […]” Quiz Solution Zimki
62. Summary Find the adequate toolkit for you Look out for open and pluggable solutions Appreciate the parts of your infrastructure that work
63. THANK you