Pinterest on AWS

Transcript

1. Pinterest Engineering Pinterest on AWS Yash Nelapati Ascii Artist
2. Pinterest Engineering How AWS?
3. Pinterest Engineering Pinterest on AWS web m1.small DB m1.small Humble Beginnings
4. Pinterest Engineering Growth 1.7G m1.small 7.5G m1.large 15G m1.xlarge 34G m2.2xlarge 68G m2.4xlarge Pinterest on AWS STOP
5. Pinterest Engineering Pinterest on AWS Workload Write heavy workload Need more IOPS Slave lag Logical Partitioning 68G m2.4xlarge pins 15G m1.xlarge 15G m1.xlarge followers
6. Pinterest Engineering Pinterest on AWS EBS and lessons Storage on demand Snapshotting Durability Variability Failures One slow disk kills entire RAID
7. Pinterest Engineering Pinterest on AWS Network
8. Pinterest Engineering Pinterest on AWS Noisy neighbors memcache RabbitMQ MySQL App
9. Pinterest Engineering Pinterest on AWS An Era of Sharding
10. Pinterest Engineering Scaling Pinterest How? Type Shard Local id As traffic grew split shards Excessive splitting on m1.xlarge More machines = more failures Failover's Hurts availability 64 bit IDS
11. Pinterest Engineering Pinterest on AWS Then came the light
12. Pinterest Engineering
13. Pinterest Engineering Disaster Recovery Forced to think early on Prioritized much earlier Multi AZ / Region Pinterest on AWS
14. Pinterest Engineering Amazon Chaos Monkey Effect Pinterest on AWS
15. Pinterest Engineering Limitations S3 bucket size issues API stops responding after some magic number Need to shard objects in directories Packets/sec limitations High memory boxes dont work for all workloads Shard early and often Hard to get boot up failures Kernel panic System log of Amazon is 4K and its a buffer Pinterest on AWS
16. Pinterest Engineering Limitations (contd..) Only 100 rules in a security group Firewall Issues. New instances cant talk to others S3 upload multi-part MIME upload results in no e- tag support. S3 read after write Data move between regions is painful. Protocols like tsunami on top of UDP Everything is EPHEMERAL Pinterest on AWS
17. Pinterest Engineering Puppet v/s AMI Pinterest on AWS
18. Pinterest Engineering Tips Underperforming CPU architectures EBS is convenient for root But AMI’s are more reliable Regions may not be completely isolated Shared network peering Don’t rely on AWS services too much Keep datacenter move as part of your architecture VPC - Get in early Pinterest on AWS
19. Pinterest Engineering Scaling Pinterest Gist AWS is raw power Write your tools to make it work Build your platform
20. Pinterest Engineering Scaling Pinterest •Looking Forward • Continually improve Pinner experience • Better uptime and lower latency • Careful evaluation of datacenter move • Continually collaborate and build bigger, better, faster products • 140 Pinployees and beyond • MySQL 5.6 What’s next?
21. yashh@pinterest.com pinterest.com/yashh Thank for support Jeremy Carroll Danilo Stefanovic