Tweet
Tweet

Slide 1

Slide 1 text

The Hidden Costs of Managed Open Source Ben Bridts aka: tables and numbers

Slide 2

Slide 2 text

? ~$ whoami

Slide 3

Slide 3 text

Some ways to run software in AWS DIY Managed Native DIY Managed Native Pay per Infrastructure Pay per Capacity Unit Pay per Request Everything in your account Most things in your account Some things in your account DIY Authentication DIY Authorization Sometimes IAM authentication Custom or IAM authorization IAM authentication IAM authorization

Slide 4

Slide 4 text

Authentication Authorization application integration

Slide 5

Slide 5 text

Connecting to DynamoDB Effect: "Allow" Action: "dynamodb:GetItem" Resource: "arn:aws:dynamodb:eu-west-1:123456789012:table/ $table" aws dynamodb get-item \ --table-name $table --key '{"Id":{"S": "$item"}}'

Slide 6

Slide 6 text

Connecting to RDS Effect: "Allow" Action: "rds-db:connect" Resource: "arn:aws:rds-db:eu-west-1:123456789012:db-user: $db-instance/$db-user" admin@$db-instance: CREATE USER '$db-user' IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS'; aws rds generate-db-auth-token \ --hostname $hostname --port 3306 \ --username $db-user mysql --host $hostname --port 3306 --user $db-user --password $db-auth-token

Slide 7

Slide 7 text

Some ways to run software in AWS DIY Managed Native DIY Managed Native Pay per Infrastructure Pay per Capacity Unit Pay per Request Everything in your account Most things in your account Some things in your account DIY Authentication DIY Authorization Sometimes IAM authentication Custom or IAM authorization IAM authentication IAM authorization

Slide 8

Slide 8 text

Managed Native Amazon Kinesis Amazon Managed Streaming for Apache Kafka (Amazon MSK) Amazon Simple Queue Service (Amazon SQS) Amazon MQ AWS Step Functions Amazon Managed Workflows for Apache Airflow (Amazon MWAA) Amazon Elastic Kubernetes Service (Amazon EKS) Amazon Elastic Container Service (Amazon ECS) Amazon DocumentDB (with MongoDB compatibility) Amazon DynamoDB Amazon CloudWatch Amazon Managed Service for Prometheus Amazon FSx Amazon Elastic File System (Amazon EFS)

Slide 9

Slide 9 text

You are all smart people and we all run open source software*

Slide 10

Slide 10 text

Why Managed Open Source? lift and shift insourcing features cost modeling

Slide 11

Slide 11 text

The bill please Some examples

Slide 12

Slide 12 text

Amazon EKS

Slide 13

Slide 13 text

Amazon EKS 1x Cluster /h 0.10 USD 73.20 3x VPC Endpoint /h 0.011 USD 24.16 ------------- Subtotal: 97.36 1x Ext. Support Cluster /h 0.60 USD 439.20 ?x Inter-AZ Datatransfer /GB 0.02 USD ??.?? ------------- Total: > 536.56

Slide 14

Slide 14 text

Amazon EKS

Slide 15

Slide 15 text

Amazon EKS Shared Clusters ~ increase blast radius ~ operational overhead Less cross-AZ data transfer ~ Topology Aware Routing (beta) ~ sub. kube-proxy w. ALB Less NAT transfer ~ S3 Gateway Endpoint for ECR ~ ECR Pull-Through Cache for e.g. dockerhub

Slide 16

Slide 16 text

Amazon Managed MQ RabbitMQ

Slide 17

Slide 17 text

Amazon Managed MQ RabbitMQ 1x mq.t3.micro /h 0.030 USD 21.70 1x Amazon EBS 5GB /month 0.55 USD 0.55 ------------- Subtotal: 22.35

Slide 18

Slide 18 text

Amazon Managed MQ RabbitMQ 1x mq.t3.micro /h 0.030 USD 21.70 1x Amazon EBS 5GB /month 0.55 USD 0.55 ------------- Subtotal: 22.35 3x mq.t3.micro /h 0.030 USD 65.09 3x Amazon EBS 5GB /month 0.55 USD 1.65 ------------- Subtotal: 66.09

Slide 19

Slide 19 text

1x mq.t3.micro /h 0.030 USD 21.70 1x Amazon EBS 5GB /month 0.55 USD 0.55 ------------- Subtotal: 22.35 3x mq.t3.micro /h 0.030 USD 65.09 3x Amazon EBS 5GB /month 0.55 USD 1.65 ------------- Subtotal: 66.09

Slide 20

Slide 20 text

1x mq.t3.micro /h 0.030 USD 21.70 1x Amazon EBS 5GB /month 0.55 USD 0.55 ------------- Subtotal: 22.35 3x mq.t3.micro /h 0.030 USD 65.09 3x Amazon EBS 5GB /month 0.55 USD 1.65 ------------- Subtotal: 66.09 3x mq.m5.large /h 0.321 USD 704.91 3x Amazon EBS 5GB /month 0.55 USD 1.65 ------------- Subtotal: 706.56

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

Amazon MSK 1x kafka.t3.small /h 0.0499 USD 35.53 1x kafka.m7g.large /h 0.2275 USD 166.53 3x kafka.t3.small /h 0.0499 USD 106.59 3x kafka.m7g.large /h 0.2275 USD 499.59 1x serverless cluster /h 0.8625 USD 631.35 ?x partition /h 0.0017 USD 1.26 5x storage /GB /month 0.1150 USD 0.58 ?x Data in /GB 0.1150 USD ?.?? ?x Data out /GB 0.0575 USD ?.??

Slide 23

Slide 23 text

Talking About MSK More hidden costs

Slide 24

Slide 24 text

Amazon MSK + AWS Lambda

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

Amazon MSK + AWS Lambda 3x kafka.t3.small /h 0.0499 USD 106.59 3x kafka.m7g.large /h 0.2275 USD 499.59 1x serverless cluster /h 0.8625 USD 631.35 3x NAT-GW /h 0.0480 USD 184.46 ?x NAT-GW data /GB 0.0480 USD ???.??

Slide 27

Slide 27 text

Amazon MSK + AWS Lambda 3x kafka.t3.small /h 0.0499 USD 106.59 3x kafka.m7g.large /h 0.2275 USD 499.59 1x serverless cluster /h 0.8625 USD 631.35 3x NAT-GW /h 0.0480 USD 184.46 ?x NAT-GW data /GB 0.0480 USD ???.?? 3x VPC-Endpoint /h. 0.0110 USD 24.16 ?x VPC-Endpoint data /GB. 0.0100 USD ?.??

Slide 28

Slide 28 text

NAT Gateway VPC-Endpoint Data Transfer Cost

Slide 29

Slide 29 text

What does that tell us?

Slide 30

Slide 30 text

Rules of Cost Modeling include all environments data transfer is not free units and gaps matter

Slide 31

Slide 31 text

It’s not about the license it's about the control plane stable cost vs. pay for value spectrum of software cost of operations

Slide 32

Slide 32 text

@benbridts@hachyderm.io ben@cloudar.be benbridts @benbridts Thank you!