AWS PrivateLink - Deep Dive

A deep dive covering AWS PrivateLink.

Transcript

1. AWS PrivateLink Deep Dive
2. aws sts get-caller-identity • Enri Peters • Zutphen • 30 • 3 girls • 1 dog (a boy 🎉) • Study • Horror • Gaming (lately Zelda botw) • Working for SBP since 2019 • Jumbo -> PostNL team
3. What is AWS PrivateLink? • Tech stack (8 nov. 2017) • Kinesis/EC2/SSM + • AWS PrivateLink makes it easy to connect services across different AWS accounts • W/O exposing data to the public internet
4. Prior to PrivateLink, services in an Amazon VPC were Connected through public IP addresses using an internet gateway or by private IP addresses using VPC peering
5. With AWS PrivateLink Service connectivity can be established from the service provider’s VPC to the service consumer’s VPCs
6. AWS PrivateLink does this • VPC peering connections • Transit VPC
7. What is AWS PrivateLink? • Customers can securely access services on AWS while staying on Amazon’s private network • Exist of mainly 2 things • Endpoint services • Your own application/service in your VPC • VPC endpoints • Interface endpoints • Gateway endpoints • GWLB endpoints Service provider Service consumer
8. Powered by • AWS Hyperplane (internal AWS service) • Amazon EFS • AWS Managed NAT • AWS Network Load Balancer • AWS PrivateLink • Mapping service for ENI’s • State tracking • Routing • Runs on EC2 (in-memory) • Keeps state for months/years (EFS)
9. PrivateLink main benefits Private • IP addresses • Security groups • Does not traverse the internet Simplify • Network management • Removes need for • IP whitelisting • IGW/NAT • Firewalls Facilitate • Your Cloud Migration • On-premises -> Direct Connect -> AWS services
10. PrivateLink use cases Securely • Access SAAS applications • You are the connection initiator Maintain • Regulatory compliance • Restrict/No internet access Migrate • To hybrid cloud • Direct Connect Shared • Services • W/O Peering
11. What are VPC Endpoints? • Virtual devices • Service provider • AWS • Marketplace • Your own service associated with NLB • Service consumer • Interface endpoints • Gateway endpoints • GWLB endpoints
12. Endpoint services • Existing AWS endpoints • Custom endpoints • Your own application • Marketplace • Can be connected to through an interface endpoint • (Auto) Allow/Deny
13. VPC Interface endpoints • Enable connectivity to services over AWS PrivateLink • Supports • IPv4 / TCP only • Direct Connect • Site-to-Site VPN • VPC Peering • Include • AWS managed services • Marketplace services • Endpoint services (Your own App) • (Hyperplane) ENI’s in subnet (Not HA by default)
14. VPC Interface endpoints • Security group • inbound 443 (for AWS) • outbound empty (Hyperplane magic) • Private DNS (optionally) • The owner of a service is a service provider • The principal creating the interface endpoint and using that service is a service consumer
15. VPC Interface endpoints • Endpoint policy (default allow) • Running cost = $8,- p/m • Data transfer cost (GB/month) • First 1PB = $0.01 • Next 4PB = $0.006 • Anything over 5 PB = $0.004 • S3 support • Can use in shared subnet (RAM) • But..
16. W/O Interface endpoints
17. With Interface endpoints & PrivateDNS
18. Interface endpoint policies
19. Availabilty Zone IDs AWS maps the physical Availability Zones randomly to the available zone names for each AWS account.
20. Availabilty Zone IDs AWS maps the physical Availability Zones randomly to the available zone names for each AWS account.
21. VPC Gateway endpoints • Adds specific IP routes (prefix-list) in a route table • Traffic flows via GW endpoint • S3 / DynamoDB • Free • HA in region • Regional • Can’t access other regions buckets
22. VPC Gateway endpoints • Prevent leaky buckets by using endpoint policies • AWS managed prefix list • Route tables • Security groups • No need for public IP addressing (IGW) • Gateway endpoints do not enable AWS PrivateLink
23. W/O Gateway endpoints
24. With Gateway endpoints
25. Gateway endpoint policies
26. VPC Gateway Load Balancer endpoints • Helps run and scale 3rd party appliances • GWLB Endpoints • Like a interface endpoint but can be added to a (ingress) route table as next hop • GWLB • Balances across backend appliances • Geneve (tunnelling protocol) • Unaltered packets
27. VPC Gateway Load Balancer endpoints • For things like… • Firewall • Intrusion detection • Prevention systems • Horizontal scaling • Security groups are not supported. • Endpoint policies are not supported.
28. Gateway endpoints vs. Interface endpoints • Gateway endpoints • S3 • DynamoDB • Interface endpoints • Most common services • Around 160 services • https://docs.aws.amazon.com/vpc /latest/privatelink/integrated- services-vpce-list.html
29. Gateway endpoint vs Interface endpoint • Prefix list (logical representation) added to route table • Does not sit inside a subnet • Magic happens at VPC router level • No security groups, because no ENI’s
30. Gateway endpoint vs Interface endpoint • Sits inside subnet (put 1 in each AZ for HA) • Attached to a security group • Endpoint specific DNS name • Regional • Zonal • Resolves to private IP address of the endpoint ENI • PrivateDNS = associate a private R53 hosted zone with your VPC • Overwrites the default DNS for the service • Can be used outside of VPC (Direct Connect etc.) vpce-0fe5b17a0707d6abc-29p5708s.ec2.us-east-1.vpce.amazonaws.com vpce-0fe5b17a0707d6abc-29p5708s-us-east-1a.ec2.us-east-1.vpce.amazonaws.com
31. Cost overview
32. VPC Interface endpoint costs example • 1 VPC endpoints x 3 ENIs per VPC endpoint x 730 hours in a month x 0.011 USD = 24.09 USD (Hourly cost for endpoint ENI) • Tiered price for: 10000 GB • 10000 GB x 0.0100000000 USD = 100.00 USD • Total tier cost = 100.0000 USD (PrivateLink data processing cost) • 24.09 USD + 100 USD = 124.09 USD (Total PrivateLink Cost) • Total PrivateLink endpoints and data processing cost (monthly): 124.09 USD
33. NAT Gateway costs example • 730 hours in a month x 0.048 USD = 35.04 USD (Gateway usage hourly cost) • 10,000 GB per month x 0.048 USD = 480.00 USD (NAT Gateway data processing cost) • 35.04 USD + 480.00 USD = 515.04 USD (NAT Gateway processing and month hours) • 3 NAT Gateways x 515.04 USD = 1,545.12 USD (Total NAT Gateway usage and data processing cost) • Total NAT Gateway usage and data processing cost (monthly): 1,545.12 USD
34. Limitations • You cannot create an endpoint between a VPC and a service in a different Region • API Gateway interface endpoint with PrivateDNS enabled • Breakes public API gateways access • ECR pull through cache • First time pull • AZ mapping • Supports only IPV4 TCP traffic • Check service specific PrivateLink docs
35. Limitations • Downtimes while creating them • +- 5 seconds for Gateway endpoint (also creation) • For CloudWatch Logs the average time was approximately 54 seconds with a minimum of 15 seconds and a maximum of 169 seconds (2m 49s). • For SNS the average was around 44 seconds with a minimum of 14 seconds and a maximum of 172 seconds (2m 51s). • For SQS the average was around 30 seconds with a minimum of 13 seconds and a maximum of 56 seconds. • Trick DNS to prevent this downtime
36. End Thank you!