How we built a heroku for ourselves

Senthil V S siliconsenthil jeechudeka Jeechu Deka

Before we start…

Done microservices?

Used AWS?

Know to cook?

Know to order food?

Cook or Buy

Agenda The problem cloudlift, our solution Learnings Q & A

Simpl Virtual CC for users make money simple, so that people can live well and do amazing things. PG for merchants We enable payment without banks

Platform Simpl Merchant servers to API Merchant apps SDKs Merchant apps Sim pl user facing Apps

Simpl architecture Services Streaming Platform Data Pipeline Dashboards Analytics ML models Merchant Apps Simpl Website Different tech stacks Scalability requirements Security requirements services

So… Four tech stacks (RoR, GoLang, Node, Python) Five developers. No (Dev)Ops Scaling needs are different Security needs are different Centralized logging Centralized config management

We wanted a solution, that…. Multiple tech stacks No (Dev)Ops. Easy to use Standard stack across platform Customizable standard

Possible solutions • Hand-crafted (?!) servers • Config management tools (chef, puppet etc.) • PaaS (Heroku, now.sh, AWS ElasticBeanstalk etc.)

Config management tools • Using Chef/puppet/Terraform etc. • Heavy-weight. Highly customizable leading to complex codebase. • Platform architecture being reflected. • Every change in service communications could require change in DevOps platform

PaaS • Easier to get started • No way to customize • Microservices: Internal vs external • Costly

PaaS is force-fitting IaaS is hard What did we do…

Or something in-between Cook or Buy

Build a PaaS on IaaS

Cloudformation • Infra as code • AWS specific • Resources defined in template and applied • Idempotent

Cloudformation template

ECS • ECS: Elastic Container Service • Orchestrator engine to run containers • Runs on cluster of EC2 instances • Exposes API to run/stop/scale containers • Container repository to push images

Service Task definition ✓ Network mode ✓ Constraints Container definition ✓ ECR image version ✓ Env vars ✓ Port mapping ✓ Command to run ✓ Logs ✓ Cluster i.e. set of EC2 instances ✓ Number of tasks ✓ High availability deployment ✓ Task placement ✓ Load balancing (ELB/ALB) ✓ Autoscaling ECS terms ✓ECR: EC2 Container registry ✓ELB: Elastic load balancer ✓ALB: Application load balancer ✓Port mapping ✓ Container port <-> Container instance port ✓ Can be dynamic ✓Task placement ✓ AZ Balanced spread ✓ One Task per Host ✓ BinPack ✓Constraints ✓ allow you to filter the instances ✓ using built-in or custom attributes ✓Network mode ✓ Bridge/Host etc. ✓Tasks run from Task definition ✓Containers run from Container definition

Demo Creating a service Deploying the service Scaling it Monitoring & logs

Cloudlift architecture click library (cli -> Python methods) Boto Python code to AWS api Cloudformation template

Cloudlift • Built by two developers at Simpl & in production use • Used in a heroku style. (Yeah. You can call it heroic style ;)) • Fully customizable • Open sourced

What happens on deploy_service ? Push local docker 
 image to ECR Ensure S3 has all required config env.sample Docker registry ECR Update the task definition with image version & env vars ECS runs new containers with updated version Stops the old version once new are up ★ The service name is guessed from current dir ★ image name is same as repo name in 
 spinal case without simpl- prefix ★ latest version is picked to push 1 2 3 4 5

A solution Multiple tech stacks No (Dev)Ops. Easy to use Standard stack across platform Customizable standard Docker }Cloudformation CLI

Show me code

Learnings

Microservices & DevOps - Keep it lightweight - Treat service creation/removal a frequent thing to do - Empower developers

Microservices & DevOps - Build time dependency => library - Runtime dependency => another microservice - Lifecycle dependency => DevOps

ECS is not batteries-included. Simple and does the job. Well.

Never hand-code Cloudformation template

Bake-in monitoring and alerts

Treat secrets and other configurations same.

12 factor apps Env var config Port binding Dev/prod parity Logs as event streams

Prefer CLI over GUI Building CLI is easier Easier to integrate in CI tools Security

CLI: Use defaults for almost everything

CLI: Avoid positional params except, readable command options

CLI: Help is must Because it’s for humans

CLI: Dry-run is must Because humans err

CLI: Dry-run is must Because humans err Confirm before running Ability to abort

CLI: Log the progress & details Because humans are impatient Log levels As colorful as possible

Follow standard unix exit codes Because machines have to understand too

Version it Because things improve

Q & A

siliconsenthil jeechudeka