from
Dr. Matt Wood
[email protected]
General Manager, Data Science
a presentation by
Matt Wood
N
E
S
W
@mza
FIELD NOTES
EXPEDITIONS
IN THE CLOUD
Slide 2
Slide 2 text
RETAIL
280
Slide 3
Slide 3 text
RETAIL MERCH
280
Slide 4
Slide 4 text
RETAIL MERCH AWS
280
Slide 5
Slide 5 text
14th March 2006 S3
25th AUGUST 2006 EC2
22nd OCTOBER 2009 RDS
15th FEBRUARY 2013 Redshift
18th JANUARY 2012 DYNAMODB
RETAIL MERCH AWS
280
Slide 6
Slide 6 text
14th March 2006 S3
25th AUGUST 2006 EC2
22nd OCTOBER 2009 RDS
15th FEBRUARY 2013 Redshift
18th JANUARY 2012 DYNAMODB
RETAIL MERCH AWS 2008
2009
2010
2011
2012
2013
159
82
61
48
24
280
Slide 7
Slide 7 text
14th March 2006 S3
25th AUGUST 2006 EC2
22nd OCTOBER 2009 RDS
15th FEBRUARY 2013 Redshift 170 launches in 2014
18th JANUARY 2012 DYNAMODB
RETAIL MERCH AWS 2008
2009
2010
2011
2012
2013
159
82
61
48
24
280
Slide 8
Slide 8 text
LOTS OF DATA
Slide 9
Slide 9 text
LOTS OF DATA
LOTS OF USES
Slide 10
Slide 10 text
LOTS OF DATA
LOTS OF USES
LOTS OF USERS
Slide 11
Slide 11 text
LOTS OF DATA
LOTS OF USES
LOTS OF USERS
LOTS OF LOCATIONS
Slide 12
Slide 12 text
LOTS OF DATA
LOTS OF USES
LOTS OF USERS
LOTS OF LOCATIONS
COMPLEXITY
Slide 13
Slide 13 text
SOLAR PANELS
Notes on
CAN we make A
better solar panel?”
“
Slide 14
Slide 14 text
SOLAR PANELS
Notes on
CAN we make A
better solar panel?”
“
Synthesize
PURIFY
ANALYZE
PHOTO CELL?
Slide 15
Slide 15 text
SOLAR PANELS
Notes on
CAN we make A
better solar panel?”
“
Synthesize
PURIFY
ANALYZE
PHOTO CELL?
GRAD
STUDENT
EQUIPMENT
+
Slide 16
Slide 16 text
SOLAR PANELS
Notes on
CAN we make A
better solar panel?”
“
205,000
organic molecules
Synthesize
PURIFY
ANALYZE
PHOTO CELL?
GRAD
STUDENT
EQUIPMENT
+
Slide 17
Slide 17 text
No content
Slide 18
Slide 18 text
1.21 PFLOPS
16,788 instances
Slide 19
Slide 19 text
1.21 PFLOPS
16,788 instances
264 years of compute time
<18 hours
Slide 20
Slide 20 text
1.21 PFLOPS
16,788 instances
264 years of compute time
<18 hours
$33k
Slide 21
Slide 21 text
1.21 PFLOPS
16,788 instances
264 years of compute time
<18 hours
$33k
COST MODELS
ENABLE INNOVATION
UTILIZATION
Slide 22
Slide 22 text
1.21 PFLOPS
16,788 instances
264 years of compute time
<18 hours
$33k
COST MODELS
ENABLE INNOVATION
UTILIZATION
RESERVED CAPACITY
Slide 23
Slide 23 text
1.21 PFLOPS
16,788 instances
264 years of compute time
<18 hours
$33k
COST MODELS
ENABLE INNOVATION
UTILIZATION
RESERVED CAPACITY
ON DEMAND
Slide 24
Slide 24 text
1.21 PFLOPS
16,788 instances
264 years of compute time
<18 hours
$33k
COST MODELS
ENABLE INNOVATION
UTILIZATION
RESERVED CAPACITY
ON DEMAND
Slide 25
Slide 25 text
VIDEO GAMES
Notes on
Slide 26
Slide 26 text
VIDEO GAMES
Notes on
“WHO BUYS VIDEO GAMES?”
Slide 27
Slide 27 text
VIDEO GAMES
Notes on
“WHO BUYS VIDEO GAMES?”
3.5 billion records
13 TB of click stream logs
71 million unique cookies
C
luster
Slide 28
Slide 28 text
VIDEO GAMES
Notes on
“WHO BUYS VIDEO GAMES?”
3.5 billion records
13 TB of click stream logs
71 million unique cookies
C
luster
HUGE TVs
Slide 29
Slide 29 text
500% return on
AD SPEND
17000% decrease in
procurement time
Slide 30
Slide 30 text
RETARGETING
Notes on
Slide 31
Slide 31 text
RETARGETING
Notes on HOW DO WE SERVE THE
BEST POSSIBLE AD?”
”
Slide 32
Slide 32 text
RETARGETING
Notes on HOW DO WE SERVE THE
BEST POSSIBLE AD?”
”
AD SERVERS
Slide 33
Slide 33 text
RETARGETING
Notes on HOW DO WE SERVE THE
BEST POSSIBLE AD?”
”
AD SERVERS
BIDDING ENGINE
Slide 34
Slide 34 text
RETARGETING
Notes on HOW DO WE SERVE THE
BEST POSSIBLE AD?”
”
AD SERVERS
BIDDING ENGINE
DYNAMODB
Slide 35
Slide 35 text
RETARGETING
Notes on HOW DO WE SERVE THE
BEST POSSIBLE AD?”
”
AD SERVERS
BIDDING ENGINE
10ms
20MS
20MS 20MS
HARD LIMIT: 100ms
10k advertisers
100 countries
1.5PB in S3
DYNAMODB
Slide 36
Slide 36 text
100 countries
Slide 37
Slide 37 text
100 countries
Slide 38
Slide 38 text
100 countries
HUNDREDS OF MILLIONS
OF RECORDS
10ks WRITES PER SECOND
in sync at
WITH APACHE STORM
Slide 39
Slide 39 text
DYNAMODB THROUGHPUT
AVERAGE LATENCY
11/1/2013 11/7/2013
Slide 40
Slide 40 text
DYNAMODB THROUGHPUT
AVERAGE LATENCY
# REQUESTS
11/1/2013 11/7/2013
Slide 41
Slide 41 text
DYNAMODB THROUGHPUT
AVERAGE LATENCY
# REQUESTS
“THE LAW OF LARGE
NUMBERS IS YOUR FRIEND
UNTIL YOU REACH
LARGE NUMBERS”
11/1/2013 11/7/2013
Slide 42
Slide 42 text
DYNAMODB THROUGHPUT
“THE LAW OF LARGE
NUMBERS IS YOUR FRIEND
UNTIL YOU REACH
LARGE NUMBERS”
PRECISE DESCRIPTION
FORMAL METHODS
TEST, TEST, TEST
AVERAGE LATENCY
# REQUESTS
11/1/2013 11/7/2013
Slide 43
Slide 43 text
SOCIAL GAMINg
Notes on
Slide 44
Slide 44 text
SOCIAL GAMINg
Notes on
CAN PHYSICAL AND VIRTUAL
WORLDS INTERACT?”
”
SOCIAL EXPERIENCE
BUILT RIGHT INTO THE GAMES
Slide 45
Slide 45 text
SOCIAL GAMINg
Notes on
CAN PHYSICAL AND VIRTUAL
WORLDS INTERACT?”
”
SOCIAL EXPERIENCE
BUILT RIGHT INTO THE GAMES
LOW LATENCY
HIGH AVAILABILITY
ANALYTICS
Slide 46
Slide 46 text
JAPAN
US EUROPE
Slide 47
Slide 47 text
JAPAN
US EUROPE
S3 CLOUDFRONT
VPC
Slide 48
Slide 48 text
JAPAN
US EUROPE
S3 CLOUDFRONT
VPC
EC2 EC2 EC2
AZ1 AZ2 AZ3
Slide 49
Slide 49 text
JAPAN
US EUROPE
REDSHIFT
S3 CLOUDFRONT
VPC
EC2 EC2 EC2
AZ1 AZ2 AZ3
Slide 50
Slide 50 text
GENOMICS
Notes on
Slide 51
Slide 51 text
Chromosome 11 GENOMICS
Notes on
Slide 52
Slide 52 text
Chromosome 11
Chromosome X
GENOMICS
Notes on
Slide 53
Slide 53 text
Chromosome 11
Chromosome X
GENOMICS
Notes on
Chromosome 19
Slide 54
Slide 54 text
Chromosome 11
Chromosome X
Chromosome 15
GENOMICS
Notes on
Chromosome 19
Slide 55
Slide 55 text
Chromosome 11
Chromosome X
Chromosome 15
GENOMICS
Notes on
Chromosome 19
Slide 56
Slide 56 text
Chromosome 11
Chromosome X
Chromosome 15
GENOMICS
Notes on
Chromosome 19
Slide 57
Slide 57 text
DNA SEQUENCING COSTS
< $1000
PER GENOME
Slide 58
Slide 58 text
DNA SEQUENCING COSTS
< $1000
PER GENOME
Slide 59
Slide 59 text
DNA SEQUENCING COSTS
< $1000
PER GENOME
S3
COLLABORATION and TOOLING
Slide 60
Slide 60 text
No content
Slide 61
Slide 61 text
CLINICAL TRIALs
Notes on
Slide 62
Slide 62 text
CLINICAL TRIALs
Notes on
BUSINESS DRIVERS
Speeding server provisioning for R&D apps
Extending capacity for internal grid
environments
Slowing internally hosted compute
infrastructure growth
On-boarding security, validation services and
compliance
Reducing cost while extending
capabilities
Slide 63
Slide 63 text
CLINICAL TRIAL SIMULATIONS
Individual Clinical Trial Simulation Run Time (Min)
Total number of trials
Number of servers
Number of CPUS
TOTAL ANALYSIS RUN TIME (hours)
COST
INTERNAL AWS
56 56
2000 2000
2
32
60
??? $336
1.2
2048
256
Slide 64
Slide 64 text
REDUCED BURDEN ON
pEDIATRIC SUBJECTS
NUmber of subjects
NUmber of blood samples per subject
length of stay per subject
length of study
total study cost
traditional
DESIGN
DESIGN OPTIMIZED
USING CLINICAL
TRIAL SIMULATION
60 40
12 5
72 hours
2.5 years
$700k $250k
1.7 years
26 hours
Slide 65
Slide 65 text
CONNECTED
DEVICES
GENOMICS
CLINICAL
TRIALS
GAMING
MOBILE
DEVELOPMENT
ADVERTISING
HOSPITALITY
RETAIL
E-COMMERCE MANUFACTURING
TRAVEL
FREE STEAK
SOCIAL APPS
FINANCIAL
ANALYTICS
MEDIA STREAMING
DISASTER
RECOVERY
ARCHIVAL
INTERACTIVE
TV APPS
WEB APPS
MARS
EXPLORATION
Slide 66
Slide 66 text
WHERE TO
FOCUS?
EARLY STAGE
START-UP
BILLION DOLLAR
MULTINATIONAL
Slide 67
Slide 67 text
AGILITY
REASON
#1:
Slide 68
Slide 68 text
AGILITY
EARLY STAGE
START-UP
REASON
#1:
Slide 69
Slide 69 text
AGILITY
EARLY STAGE
START-UP
BILLION DOLLAR
MULTINATIONAL
REASON
#1:
Slide 70
Slide 70 text
FLY WHEELS
FOR AGILITY
SECURITY
SCALE
PERFORMANCE
RELIABILITY
COST
Slide 71
Slide 71 text
ADOPTION
PATTERNS OF How are enterprises
using the cloud in
2014?
Slide 72
Slide 72 text
DEVELOPMENT &
TESTING
SHAREPOINT & SAP
ORACLE
SAP
Slide 73
Slide 73 text
NEW WORKLOADS
GLOBAL DEALS ENGINE
PRODUCT PROTOTYPING
MOBILE MUSIC DISCOVERY
HOTEL BOOKING
VIDEO STREAMING
Slide 74
Slide 74 text
SUPPLEMENT
EXISTING
WORKFLOWS
operational applications
equipment leasing
analytics
with the cloud
Slide 75
Slide 75 text
SUPPLEMENT
EXISTING
WORKFLOWS
with on-premises infrastructure
AWS serves up application
content & data
Integration back to Samsung
Data Centers
for financial transactions
Slide 76
Slide 76 text
MIGRATING
EXISTING
APPLICATIONS
500 web properties in 5 months
CLINICAL TRIALS
1,700 web properties
Slide 77
Slide 77 text
ALL IN
“WE HAVE TO BE GREAT AT A NUMBER OF THINGS...
OPERATING DATA CENTERS IS NOT ONE OF THOSE THINGS”
“WE ARE A HOSPITALITY MANAGEMENT COMPANY,
NOT AN IT SERVICES COMPANY”
“FRIENDS DON’T LET FRIENDS
BUILD DATA CENTERS”
INFOR CLOUDSUITE
micro-vertical BUSINESS APPS
Slide 78
Slide 78 text
IN SUMMARY...
DATA DRIVES INNOVATION WHEN
COMBINED WITH UTILITY COMPUTING
Broad and deep services
enable new business
AGILITY REMAINS THE #1 REASON
FOR ADOPTING THE AWS CLOUD
ENTERPRISES LEAD WITh
DEV, TEST & NEW WORKLOADS
MIGRATION AND SUPPLEMENTS
QUICKLY FOLLOW
Increasingly ‘ALL IN’
More sessions from AWS today
11.00am: Transforming Applications into SaaS
11.40am: Big Data Is Best Served Fresh: Real-time Analytics in the Cloud
1.55pm: What is the Internet of Things – and how can it help my business?
4.00pm: Moving Enterprises to the Cloud the Right Way and for the Right Reasons
4.40pm: Creating a Cloud Based DR Solution for your Enterprise