Tweet
Tweet

Slide 1

Slide 1 text

Healthy cloud for a HealthTech company Sergei Egorov @bsideup

Slide 2

Slide 2 text

About me • Head of Backend at Uvita GmbH • OSS enthusiast (TestContainers, Apache Foundation, …) • Before: N26, Zalando, ZeroTurnaround, TransferWise, … • First time tried AWS in 2010-ish, Docker in 2014 @bsideup

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Digital health companion

Slide 5

Slide 5 text

WHAT HOW WHY What drives us forward

Slide 6

Slide 6 text

WHAT HOW WHY What drives us forward We believe a healthier life is a happier life.

Slide 7

Slide 7 text

WHAT HOW WHY What drives us forward We believe a healthier life is a happier life. Utilizing technology and data, we therefore provide personal health insights and thus empower to understand and improve your well-being.

Slide 8

Slide 8 text

WHAT HOW WHY What drives us forward We believe a healthier life is a happier life. Utilizing technology and data, we therefore provide personal health insights and thus empower to understand and improve your well-being. This guidance we deliver through a personal digital health companion that connects you to a ecosystem of health services.

Slide 9

Slide 9 text

WHAT HOW WHY What drives us forward We believe a healthier life is a happier life. Utilizing technology and data, we therefore provide personal health insights and thus empower to understand and improve your well-being. This guidance we deliver through a personal digital health companion that connects you to a ecosystem of health services.

Slide 10

Slide 10 text

WHY

Slide 11

Slide 11 text

@bsideup Number of micro-services 0 2 4 6 8 10 12 14 16 18 20 August '17 September '17 … January '18 20 3 1

Slide 12

Slide 12 text

@bsideup Expected number of micro-services 0 250 500 750 1000 August '17 September '17 … January '18 January '19 1000 20 3 1

Slide 13

Slide 13 text

@bsideup Sanely expected number of micro-services 0 4 8 12 16 20 24 28 32 36 40 August '17 September '17 … January '18 January '19 40 20 3 1

Slide 14

Slide 14 text

HOW

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

@bsideup

Slide 17

Slide 17 text

@bsideup

Slide 18

Slide 18 text

@bsideup

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

Elastic Container Service

Slide 21

Slide 21 text

No managed Kubernetes on AWS (yet) Well-integrated in the rest of AWS services Shared pool of EC2 instances (cattle) Blue/Green, Rolling and other deployment strategies @bsideup

Slide 22

Slide 22 text

Challenges

Slide 23

Slide 23 text

Service discovery @bsideup Internal AWS ALB AWS VPC Private VPC DNS zone, i.e.
 *.vivy.services

Slide 24

Slide 24 text

Service discovery @bsideup Internal AWS ALB AWS VPC Private VPC DNS zone, i.e.
 *.vivy.services there is cheap .services TLD

Slide 25

Slide 25 text

Service discovery Staging VPC https:/ /auth.vivy.services Staging internal ALB Routes to i.e. 10.20.30.40:36836 Prod VPC https:/ /auth.vivy.services Prod internal ALB Routes to i.e. 10.50.60.70:48162 @bsideup

Slide 26

Slide 26 text

Service discovery benefits @bsideup ✅ Devs can “hardcode” 
 https:/ /something.mycorp.services ✅ Load Balanced (on the server side) ✅ Yet great performance ✅ You don’t need Consul until you need it

Slide 27

Slide 27 text

Service discovery downsides @bsideup ⚠ Every Target in ALB must have a unique priority, even if “host” filter is already unique ⚠ ALB does not work with gRPC
 (Current solution: NLB next to ALB) ⚠ Bottleneck, but only if you’re Netflix

Slide 28

Slide 28 text

Secrets management & config @bsideup Parameters Store AWS KMS
 encrypted params + + https://github.com/remind101/ssm-env

Slide 29

Slide 29 text

@bsideup FROM ****/ssm-env:0.0.2 AS ssm-env FROM openjdk:8u151-jre COPY --from ssm-env /ssm-env /bin/ssm-env COPY build/libs/*.jar /app.jar CMD ["ssm-env", "-with-decryption", "sh", "-c", "java -jar /app.jar”] …and then with env vars: TWILIO_AUTH_TOKEN=ssm:///twilio/accountSid

Slide 30

Slide 30 text

@bsideup FROM ****/ssm-env:0.0.2 AS ssm-env FROM openjdk:8u151-jre COPY --from ssm-env /ssm-env /bin/ssm-env COPY build/libs/*.jar /app.jar CMD ["ssm-env", "-with-decryption", "sh", "-c", "java -jar /app.jar”] …and then with env vars: TWILIO_AUTH_TOKEN=ssm:///twilio/accountSid

Slide 31

Slide 31 text

SSM+KMS benefits @bsideup ✅ Encrypted & managed by AWS ✅ Key rotation ✅ key/value storage of params, both encrypted and not ✅ IAM-controlled ✅ You don’t need Vault until you need it

Slide 32

Slide 32 text

SSM+KMS downsides @bsideup ⚠ No manual key rotation in KMS ⚠ SSM Parameters Store’s UI ⚠ Not integrated with ECS out of the box

Slide 33

Slide 33 text

Cattle-like instance upgrades @bsideup Auto-scaling group v1 EC2 EC2

Slide 34

Slide 34 text

Cattle-like instance upgrades @bsideup Auto-scaling group v1 EC2 EC2 Auto-scaling group v2

Slide 35

Slide 35 text

Cattle-like instance upgrades @bsideup Auto-scaling group v1 EC2 EC2 Auto-scaling group v2 EC2 EC2

Slide 36

Slide 36 text

Cattle-like instance upgrades @bsideup Auto-scaling group v1 EC2 EC2 Auto-scaling group v2 EC2 EC2 Drain instances in ECS

Slide 37

Slide 37 text

Cattle-like instance upgrades @bsideup Auto-scaling group v1 EC2 EC2 Auto-scaling group v2 EC2 EC2

Slide 38

Slide 38 text

Cattle-like instance upgrades @bsideup Auto-scaling group v1 EC2 EC2 Auto-scaling group v2 EC2 EC2

Slide 39

Slide 39 text

Cattle-like instance upgrades @bsideup Auto-scaling group v1 EC2 EC2 Auto-scaling group v2 EC2 EC2

Slide 40

Slide 40 text

Cattle-like instance upgrades @bsideup Auto-scaling group v1 EC2 EC2 Auto-scaling group v2 EC2 EC2

Slide 41

Slide 41 text

Cattle-like instance upgrades @bsideup Auto-scaling group v1 EC2 EC2 Auto-scaling group v2 EC2 EC2

Slide 42

Slide 42 text

Cattle-like instance upgrades @bsideup Auto-scaling group v2 EC2 EC2

Slide 43

Slide 43 text

How @bsideup 1. AWS ASG “EC2_INSTANCE_TERMINATING” LifecycleHook 2. AWS Lambda will mark EC2 instance in ECS as “DRAINING” 3. Complete the lifecycle once no running containers left on EC2 instance

Slide 44

Slide 44 text

Benefits @bsideup ✅ Fargate-like ECS experience ✅ …but in eu-central-1 ✅ Up/Down scaling with no downtime

Slide 45

Slide 45 text

DoS protection @bsideup AWS ALB AWS WAF
 (Web Application Firewall) +

Slide 46

Slide 46 text

AWS WAF benefits @bsideup ✅ Attach it to your ALB ✅ Analyses the access log ✅ Flexible rules (regex by path, headers, …) ✅ Minutes to trigger

Slide 47

Slide 47 text

AWS WAF benefits @bsideup ✅ Attach it to your ALB ✅ Analyses the access log ✅ Flexible rules (regex by path, headers, …) ✅ Minutes to trigger no ALB config changes needed

Slide 48

Slide 48 text

AWS WAF downsides @bsideup ⚠ No “block by user” ⚠ No DDoS protection

Slide 49

Slide 49 text

How to manage?

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

CloudFormation

Slide 52

Slide 52 text

Terraform CloudFormation No vendor lock-in Native service by AWS OSS Managed state Modular New APIs support is fast More than just AWS resources Custom resources with Lambda @bsideup

Slide 53

Slide 53 text

CloudFormation because • The state is managed by AWS, not by us (as it is with TF) • Better documentation and examples • Some APIs/Resources are still not supported in TF • Custom Resources with AWS Lambda @bsideup

Slide 54

Slide 54 text

How to cook templates?

Slide 55

Slide 55 text

@bsideup

Slide 56

Slide 56 text

@bsideup

Slide 57

Slide 57 text

@bsideup

Slide 58

Slide 58 text

@bsideup

Slide 59

Slide 59 text

No content

Slide 60

Slide 60 text

@bsideup from uvita import ECSMicroService, SSMParameter, DynamoDBTable from troposphere import Template, Parameter t = Template() t.add_parameter(Parameter('Env', Type="String")) t.add_parameter(Parameter('DockerImage', Type="String")) questionnaire_table = DynamoDBTable('QuestionnaireDynamoDBTable', ('subjectId', 'S')) questionnaire_table.inject_to(t) subject_info_table = DynamoDBTable('SubjectInfoDynamoDBTable', ('subjectId', 'S'), ('id', 'S')) subject_info_table.inject_to(t) goal_table = DynamoDBTable('GoalDynamoDBTable', ('subjectId', 'S')) goal_table.inject_to(t) service = ECSMicroService( "sensemaking", priority=2100, public=True, envs = { "kafka_bootstrapServers": "kafka.uvita.services:9092", "kafka_eventBusTopic": "user-event-log", "aws_dynamodb_questionnairesTableName": questionnaire_table.table_ref(), "aws_dynamodb_subjectInfoTableName": subject_info_table.table_ref(), "aws_dynamodb_goalTableName": goal_table.table_ref(), "security_oauth2_resource_jwt_keyValue": SSMParameter('/${Env}/jwt/password') } ) service.inject_to(t)

Slide 61

Slide 61 text

@bsideup from uvita import ECSMicroService, SSMParameter, DynamoDBTable from troposphere import Template, Parameter t = Template() t.add_parameter(Parameter('Env', Type="String")) t.add_parameter(Parameter('DockerImage', Type="String")) questionnaire_table = DynamoDBTable('QuestionnaireDynamoDBTable', ('subjectId', 'S')) questionnaire_table.inject_to(t) subject_info_table = DynamoDBTable('SubjectInfoDynamoDBTable', ('subjectId', 'S'), ('id', 'S')) subject_info_table.inject_to(t) goal_table = DynamoDBTable('GoalDynamoDBTable', ('subjectId', 'S')) goal_table.inject_to(t) service = ECSMicroService( "sensemaking", priority=2100, public=True, envs = { "kafka_bootstrapServers": "kafka.uvita.services:9092", "kafka_eventBusTopic": "user-event-log", "aws_dynamodb_questionnairesTableName": questionnaire_table.table_ref(), "aws_dynamodb_subjectInfoTableName": subject_info_table.table_ref(), "aws_dynamodb_goalTableName": goal_table.table_ref(), "security_oauth2_resource_jwt_keyValue": SSMParameter('/${Env}/jwt/password') } ) service.inject_to(t) Standard Troposphere APIs

Slide 62

Slide 62 text

@bsideup from uvita import ECSMicroService, SSMParameter, DynamoDBTable from troposphere import Template, Parameter t = Template() t.add_parameter(Parameter('Env', Type="String")) t.add_parameter(Parameter('DockerImage', Type="String")) questionnaire_table = DynamoDBTable('QuestionnaireDynamoDBTable', ('subjectId', 'S')) questionnaire_table.inject_to(t) subject_info_table = DynamoDBTable('SubjectInfoDynamoDBTable', ('subjectId', 'S'), ('id', 'S')) subject_info_table.inject_to(t) goal_table = DynamoDBTable('GoalDynamoDBTable', ('subjectId', 'S')) goal_table.inject_to(t) service = ECSMicroService( "sensemaking", priority=2100, public=True, envs = { "kafka_bootstrapServers": "kafka.uvita.services:9092", "kafka_eventBusTopic": "user-event-log", "aws_dynamodb_questionnairesTableName": questionnaire_table.table_ref(), "aws_dynamodb_subjectInfoTableName": subject_info_table.table_ref(), "aws_dynamodb_goalTableName": goal_table.table_ref(), "security_oauth2_resource_jwt_keyValue": SSMParameter('/${Env}/jwt/password') } ) service.inject_to(t) … but with our reusable definitions

Slide 63

Slide 63 text

@bsideup from uvita import ECSMicroService, SSMParameter, DynamoDBTable from troposphere import Template, Parameter t = Template() t.add_parameter(Parameter('Env', Type="String")) t.add_parameter(Parameter('DockerImage', Type="String")) questionnaire_table = DynamoDBTable('QuestionnaireDynamoDBTable', ('subjectId', 'S')) questionnaire_table.inject_to(t) subject_info_table = DynamoDBTable('SubjectInfoDynamoDBTable', ('subjectId', 'S'), ('id', 'S')) subject_info_table.inject_to(t) goal_table = DynamoDBTable('GoalDynamoDBTable', ('subjectId', 'S')) goal_table.inject_to(t) service = ECSMicroService( "sensemaking", priority=2100, public=True, envs = { "kafka_bootstrapServers": "kafka.uvita.services:9092", "kafka_eventBusTopic": "user-event-log", "aws_dynamodb_questionnairesTableName": questionnaire_table.table_ref(), "aws_dynamodb_subjectInfoTableName": subject_info_table.table_ref(), "aws_dynamodb_goalTableName": goal_table.table_ref(), "security_oauth2_resource_jwt_keyValue": SSMParameter('/${Env}/jwt/password') } ) service.inject_to(t) … and higher level abstractions

Slide 64

Slide 64 text

@bsideup from uvita import ECSMicroService, SSMParameter, DynamoDBTable from troposphere import Template, Parameter t = Template() t.add_parameter(Parameter('Env', Type="String")) t.add_parameter(Parameter('DockerImage', Type="String")) questionnaire_table = DynamoDBTable('QuestionnaireDynamoDBTable', ('subjectId', 'S')) questionnaire_table.inject_to(t) subject_info_table = DynamoDBTable('SubjectInfoDynamoDBTable', ('subjectId', 'S'), ('id', 'S')) subject_info_table.inject_to(t) goal_table = DynamoDBTable('GoalDynamoDBTable', ('subjectId', 'S')) goal_table.inject_to(t) service = ECSMicroService( "sensemaking", priority=2100, public=True, envs = { "kafka_bootstrapServers": "kafka.uvita.services:9092", "kafka_eventBusTopic": "user-event-log", "aws_dynamodb_questionnairesTableName": questionnaire_table.table_ref(), "aws_dynamodb_subjectInfoTableName": subject_info_table.table_ref(), "aws_dynamodb_goalTableName": goal_table.table_ref(), "security_oauth2_resource_jwt_keyValue": SSMParameter('/${Env}/jwt/password') } ) service.inject_to(t) … and conventions

Slide 65

Slide 65 text

How to operate?

Slide 66

Slide 66 text

Bastion instances @bsideup https:/ /github.com/widdix/aws-ec2-ssh Users upload their SSH public keys
 in IAM Docker-based commands over SSH

Slide 67

Slide 67 text

@bsideup EC2 SSH connect

Slide 68

Slide 68 text

@bsideup EC2 SSH connect Download public
 SSH key from IAM

Slide 69

Slide 69 text

@bsideup EC2 SSH connect Download public
 SSH key from IAM Sync IAM users
 to local system

Slide 70

Slide 70 text

DEMO

Slide 71

Slide 71 text

What @bsideup 1. Deploy VPC, ECS Cluster, Public + Internal ALBs 2. Deploy an internal micro-service 3. Deploy one public micro-service 4. Debug them with a Bastion instance

Slide 72

Slide 72 text

@bsideup bsideup