Share
Embed
Netflix Cloud Architecture - continued
Cloud Genius
December 27, 2013
Transcript
25 slides
-
Netflix Business Strategy 2 Remove Late Fee Provide Choice from
Long Tail Ubiquitous Anytime access -
Incumbents in 1997 ¨ Blockbuster, Hollywood Videos, Movie Gallery ¤
Rented DVDs out from prime locations ¤ Kept inventory of about 2000 titles with 3 to 4 copies of each ¤ Mostly expensive DVDs with new movies and some long time hits ¨ New movies typically release to big screen and then release via DVD ¨ DVDs rent while its new ¨ DVD availability window is relatively small ¨ Charged late fee to maximize turns 3 Must impose steep rental fees and steeper late fees -
The Red Envelope (1999-2008) ¨ Removed late fee as they
saw opportunity in long tail of DVDs ¨ Stocked 120,000 movies mailed out from 50 hubs ¨ Matched renter interest with a low cost movie from the long tail ¨ Announced $1M prize: Help find a better recommendation algorithm ¨ 4 years and a team of 10+ PhDs solved that ¨ In 8 years, Blockbuster, Hollywood video et. Al. are almost out of business 4 Recommend movies in the long tail, personalized to the customer, lowering costs -
Streaming Movies – 2008 and beyond ¨ Netflix lets users
manage their DVD queue for what they would rent next ¨ Netflix had their own problems ¤ DVD theft ¤ DVD piracy ¤ FBI involved in the US ¤ Could not grow internationally with DVD renting model ¨ Streaming model looked very appealing ¨ Cost to stream out DVD via CDN was a few pennies but postage is 10-100 times more ¨ Streaming lets them grow internationally ¨ 20M US subscribers ¨ Started streaming to US subscribers from their datacenter ¨ Datacenter faced cooling issues and downtime ¨ They issued customer refunds which directly impacted their bottom line ¨ Netflix at that time was making about 1.6 billion in revenue and had 400 employees 5 -
Netflix Decisions 6 ¨ Who will manage the streaming backend?
¨ What devices will users watch streaming content on? ¨ On the client end ¤ What should be the client experience? ¤ Should Netflix create their own device? ¤ Should Netflix partner with device makers? ¤ If so, which device makers should Netflix pursue? ¤ What about DRM to prevent piracy and honor IP? ¨ On the cloud end ¤ Who will manage the streaming backend? ¤ Who will maintain and upkeep the data center? ¤ How to deal with data center growth? -
Decided against creating own device 9 ¨ HDMI ports on
household TVs were already occupied by a variety of devices ¨ Introducing one more Netflix device wasn’t the best -
Partnerships with Device makers 10 ¨ Grew subscribers 2008-2010 by
partnering with device makers ¤ Game Consoles n Xbox 360, Wii, PS3 ¤ Mobile Devices n Apple iPad, iPhone, iPod Touch, Windows Phone 7 n Android was not in this list until late 2011 (DRM concerns) -
Netflix Cloud Story 12 ¨ Circa 2008, Netflix had a
single data center ¤ Single point of failure (SPOF) ¤ Approaching the limits on n Cooling n Power n Physical space n Network traffic capacity -
Netflix Cloud Story 13 ¨ Strategic choices included: ¤ Build
more data centers n Manage more data centers ¤ Outsource capacity planning and scale out n Focus on core competency -
Netflix Cloud Story 14 ¨ Winning Choice: ¤ Outsource capacity planning
and scale out n Leverage a leading IAAS provider n Amazon Web Services ¨ In two years, from start of outsourcing, Netflix served over 90% traffic out of AWS ¨ This excluded video streaming traffic which came from CDNs -
How Netflix Uses AWS 15 ¨ AWS provided various IAAS
offering at that time but Netflix apps needed more • Data SimpleDB,RDS,S3, EBS • Load Handling EC2, EMR, ASG, ELB • Monitoring CloudWatch AWS ? Netflix apps -
Netflix developers filled the gap 16 ¨ Netflix developers ported
Netflix platform to the AWS AWS Netflix Platform Netflix App Netflix Platform • Platform.JAR • Middle-tier Load Balancing • Discovery • Encryption Services • Caching • Configuration -
How Netflix Used AWS 17 ELB EC2 EC2 EC2 EC2
EC2 EC2 instances for Q EC2 instances for merch Simple DB S3 SQS Edge services (such as www) Netflix mid-tier services (such as Q or merchandizing) AWS IAAS ? -
How Netflix Used AWS 18 ELB EC2 EC2 EC2 EC2
EC2 Middle Tier Load Balancer and Discovery EC2 instances for Q EC2 instances for merch Simple DB S3 SQS Edge services (such as www) Netflix mid-tier services (such as Q or merchandizing) AWS IAAS Netflix platform -
Netflix stack on EC2 19 Netflix Application Platform and other
infrastructure jar files Guest OS/Virtual Machine/Host OS/Machine Discovery client Mid tier load balancer Crypto Client S3 client Simple DB client SQS client Configuration client -
Persistent Data: Simple DB 20 ¨ Moved large transactional dataset
to Simple DB ¤ Rental history info: Everything that any one has watched on Netflix (including streaming and DVD rentals) ¨ Netflix uses Simple DB to manage ¤ Thousands of domains ¤ Over 1Tera byte of online transaction data ¤ Billions of rows of data -
Persistent Storage: S3 21 ¨ Simple key-value store organized as
objects in buckets ¤ Each AWS account can create multiple buckets ¤ Each bucket can hold unlimited number of objects ¨ Netflix uses S3 to store data that does not fit into Simple DB ¤ Logs from streaming devices ¤ Files used in movie encoding ¤ Truncated tail of rental history -
Data Replication: Item Replicator 22 ¨ IR: Item Replicator: Netflix
homegrown solution ¨ Keeps data in sync between Data Center and Cloud ¤ Replication is mostly unidirectional n Data Center to the Cloud -
Netflix Migration to Cloud 23 ¨ Netflix case is just
one example. ¨ There are several cases of businesses in a variety of industry that are migrating to the cloud