Microservices infrastructure with Terraform, Consul and Vault

Slide 1

Slide 1 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion µservices With Terraform, Consul and Vault Carlos León September 2016, Napa HashiConf

Slide 2

Slide 2 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion About me 2 Carlos León Software Engineer @ Container Solutions Hang gliding pilot @mongrelion linkedin.com/in/mongrelion [email protected]

Slide 3

Slide 3 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion Agenda 3 Software development: Monolith to microservices Rise of the containers Challenges with microservices: Putting everything together (MANTL) Bringing the solution to the customers Q&A

Slide 4

Slide 4 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion Software development : pre-DevOps 4

Slide 5

Slide 5 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion Software development : DevOps 5

Slide 6

Slide 6 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 6

Slide 7

Slide 7 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 7

Slide 8

Slide 8 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion Software development : DevOps 8

Slide 9

Slide 9 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion Software development : Post-DevOps 9

Slide 10

Slide 10 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 10 How to move to microservices?

Slide 11

Slide 11 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 11

Slide 12

Slide 12 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 12

Slide 13

Slide 13 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 13

Slide 14

Slide 14 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 14 Cluster Management?

Slide 15

Slide 15 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 15 Orchestration? Cluster Management?

Slide 16

Slide 16 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 16 Orchestration? Networking? Cluster Management?

Slide 17

Slide 17 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 17 Security? Orchestration? Networking? Cluster Management?

Slide 18

Slide 18 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 18 Security? Service discovery? Orchestration? Networking? Cluster Management?

Slide 19

Slide 19 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 19 Security? Service discovery? Orchestration? Networking? Persistence? Cluster Management?

Slide 20

Slide 20 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 20 Security? Service discovery? Orchestration? Networking? Persistence? Monitoring? Cluster Management?

Slide 21

Slide 21 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 21 We start with Docker

Slide 22

Slide 22 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 22 Maybe add some scheduling?

Slide 23

Slide 23 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 23 And Consul for service discovery?

Slide 24

Slide 24 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 24 And Project Calico (canal) for the networking?

Slide 25

Slide 25 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 25 And ELK for the logs collection?

Slide 26

Slide 26 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 26 … 6 months later … ● Terraform & Ansible deployment to cloud providers ● etcd distributed key-value store for Calico ● Calico a new kind of virtual network ● Mesos cluster manager ● Marathon long running services ● Consul for service discovery ● Vault for managing secrets ● Docker container runtime ● collectd for metrics collection ● Logstash for log forwarding ● GlusterFS for container volume storage ● Traefik for proxying external traffic

Slide 27

Slide 27 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 27 Why are you building infrastructure and not your application?

Slide 28

Slide 28 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 28

Slide 29

Slide 29 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion What is MANTL ? 29 ● Mantl is a modern platform for rapidly deploying globally distributed microservices ● Open source ● Cloud agnostic ● Available now: github.com/CiscoCloud/mantl

Slide 30

Slide 30 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion What problems does MANTL solve? 30 ● Networking ● Security ● Service Discovery ● Persistence ● Monitoring ● Orchestration ● Cluster Management ● And others...

Slide 31

Slide 31 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion What problems does MANTL solve? 31 ● Networking ● Security ● Service Discovery ● Persistence ● Monitoring ● Orchestration ● Cluster Management ● And others... ● Supports easier migration to microservices ● Solves integration conflicts ● Lowers configuration time ● More time for development ● Runs anywhere ● Avoid cloud locking ● End-to-End platform ● Customizable (open-source)

Slide 32

Slide 32 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 32 MANTL Architecture 3 types of components: ● Control ● Edge ● Resource

Slide 33

Slide 33 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 33 MANTL Architecture

Slide 34

Slide 34 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 34 MANTL Architecture

Slide 35

Slide 35 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 35 MANTL Architecture

Slide 36

Slide 36 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 36 Adding MANTL to your infrastructure

Slide 37

Slide 37 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 37 Adding MANTL to your infrastructure

Slide 38

Slide 38 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 38

Slide 39

Slide 39 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 39

Slide 40

Slide 40 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 40

Slide 41

Slide 41 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 41 Security Service discovery Orchestration Networking Persistence Monitoring Cluster Management

Slide 42

Slide 42 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 42 How to build them? Policies? How to run them? How to ship them? Cloud or on-premises? Infrastructure monitoring? Auto-scaling?

Slide 43

Slide 43 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 43

Slide 44

Slide 44 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 44 What is Shipped ? ● Shipped is a continuous delivery platform for microservices (cloud native apps) ● Commercial Cisco product ● Runs on top of MANTL ● Complements MANTL ● In the cloud or on premises ● In open Beta now: ciscoshipped.io

Slide 45

Slide 45 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 45 What problems does Shipped solve ?

Slide 46

Slide 46 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion What problems does Shipped solve? 46 ● CI/CD platform (build/deploy/run) ● Application intent engine ● Infrastructure monitoring ● Contiv (Infra Policy Enforcement)

Slide 47

Slide 47 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 47 Shipped overview

Slide 48

Slide 48 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 48 Shipped overview Application Intent

Slide 49

Slide 49 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion Shipped - Implementing App Intent - Scaling

Slide 50

Slide 50 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion Shipped - Implementing App Intent - Scaling

Slide 51

Slide 51 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion Shipped - Implementing App Intent - Scaling

Slide 52

Slide 52 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 52

Slide 53

Slide 53 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion 53 Security Service discovery Orchestration Networking Persistence Monitoring Cluster Management How to build them? How to run them? Infrastructure monitoring? Policies? How to ship them? Cloud or on-premises? Auto-scaling?

Slide 54

Slide 54 text

container-solutions.com | @containersoluti µservices with Terraform, Consul & Vault | @mongrelion Thanks, links & more 54 MANTL: mantl.io (contribute on ) Shipped: ciscoshipped.io (open beta) Microservices: Applications & Microservices with Docker & Containers - thenewstack.io martinfowler.com/articles/microservices.html Docker : Using Docker - Adrian Mouat Blogs: blogs.cisco.com/cloud container-solutions.com/blog