From Analytics to Intelligence: Amazon Redshift

Slide 1

Slide 1 text

to from ANALYTICS INTELLIGENCE a presentation at MICROSTRATEGY WORLD 2013 by DR MATT WOOD

Slide 2

Slide 2 text

Hello.

Slide 3

Slide 3 text

Thank you.

Slide 4

Slide 4 text

I Data, data everywhere

Slide 5

Slide 5 text

I II Collection & storage Data, data everywhere

Slide 6

Slide 6 text

I II III Data security Data, data everywhere Collection & storage

Slide 7

Slide 7 text

I II III IV Data movement Data, data everywhere Data security Collection & storage

Slide 8

Slide 8 text

I II III IV Data, data everywhere Data movement Data security Collection & storage 0. Amazon web Services

Slide 9

Slide 9 text

Building blocks.

Slide 10

Slide 10 text

Compute, storage & databases.

Slide 11

Slide 11 text

Retail Merchant services Web services

Slide 12

Slide 12 text

Blinding ﬂash of the obvious.

Slide 13

Slide 13 text

Available.

Slide 14

Slide 14 text

Low cost.

Slide 15

Slide 15 text

Flexible.

Slide 16

Slide 16 text

Every day, AWS adds enough server capacity to power amazon.com in 2003, when it was a $5B enterprise

Slide 17

Slide 17 text

Data, data everywhere I

Slide 18

Slide 18 text

Data for competitive advantage.

Slide 19

Slide 19 text

Customer segmentation, ﬁnancial modeling, system analysis, line of sight, business intelligence...

Slide 20

Slide 20 text

Generation Collection & storage Analytics & computation Collaboration & sharing

Slide 21

Slide 21 text

Cost of data generation is falling.

Slide 22

Slide 22 text

Kindle Fire HD, Kindle Fire, Kindle Paperwhite and Kindle hold the top four spots on the Amazon world wide best seller chart since launch. devices

Slide 23

Slide 23 text

Amazon Appstore selection tripled in 2012. apps and games

Slide 24

Slide 24 text

Amazon customers purchased more than one toy per second on mobile devices. commerce

Slide 25

Slide 25 text

most gifted kindle book

Slide 26

Slide 26 text

Generation Collection & storage Analytics & computation Collaboration & sharing lower cost, increased throughput

Slide 27

Slide 27 text

Generation Collection & storage Analytics & computation Collaboration & sharing highly constrained

Slide 28

Slide 28 text

Gap.

Slide 29

Slide 29 text

1990 2000 2010 2020 The Data Analysis Gap Enterprise Data Data in Warehouse Gartner: User Survey Analysis: Key Trends Shaping the Future of Data Center Infrastructure Through 2011 IDC: Worldwide Business Analytics Software 2012–2016 Forecast and 2011 Vendor Shares Generated data Available for analysis Data volume

Slide 30

Slide 30 text

Enter AWS.

Slide 31

Slide 31 text

Utility.

Slide 32

Slide 32 text

Remove constraints.

Slide 33

Slide 33 text

Generation Collection & storage Analytics & computation Collaboration & sharing highly constrained

Slide 34

Slide 34 text

Generation Collection & storage Analytics & computation Collaboration & sharing

Slide 35

Slide 35 text

Full value.

Slide 36

Slide 36 text

Close the gap.

Slide 37

Slide 37 text

Reduced time to market.

Slide 38

Slide 38 text

Identify and meet new business opportunities.

Slide 39

Slide 39 text

Lower costs.

Slide 40

Slide 40 text

Collection & Storage II

Slide 41

Slide 41 text

One schema to rule them all.

Slide 42

Slide 42 text

One schema to rule them all.

Slide 43

Slide 43 text

Lots of data. Lots of users. Lots of uses. Lots of locations.

Slide 44

Slide 44 text

Cost.

Slide 45

Slide 45 text

Multipliers.

Slide 46

Slide 46 text

Object storage.

Slide 47

Slide 47 text

99.999999999% durability

Slide 48

Slide 48 text

Relational databases.

Slide 49

Slide 49 text

NoSQL data stores.

Slide 50

Slide 50 text

HDFS based stores.

Slide 51

Slide 51 text

Undi erentiated heavy lifting.

Slide 52

Slide 52 text

Lower costs. Ease of use.

Slide 53

Slide 53 text

Lower costs. Ease of use. Lower costs. no capital investment pay as you go no subscriptions only pay for what you use

Slide 54

Slide 54 text

Lower costs. Ease of use. Ease of use. programmable zero admin easy to configure integrate with existing tools

Slide 55

Slide 55 text

Data warehousing.

Slide 56

Slide 56 text

Expensive. Complicated.

Slide 57

Slide 57 text

Enterprises average between 3 and 4 DBAs per data warehouse. Source: Gartner. Critical factors in calculating the data warehouse TCO, July 2009

Slide 58

Slide 58 text

Source: Oracle technology global price list 11/1/2012

Slide 59

Slide 59 text

Expensive. Complicated.

Slide 60

Slide 60 text

Unobtainable.

Slide 61

Slide 61 text

Amazon Redshift.

Slide 62

Slide 62 text

Fast. Powerful. Petabyte scale.

Slide 63

Slide 63 text

Managed service.

Slide 64

Slide 64 text

Automated deployment & conﬁguration.

Slide 65

Slide 65 text

SQL access and BI tool integration.

Slide 66

Slide 66 text

Parallel execution.

Slide 67

Slide 67 text

Leader Node

Slide 68

Slide 68 text

Compute Node Compute Node Compute Node Leader Node

Slide 69

Slide 69 text

Compute Node Compute Node Compute Node Leader Node

Slide 70

Slide 70 text

10gigE full bisection network.

Slide 71

Slide 71 text

Compute Node Compute Node Compute Node Leader Node

Slide 72

Slide 72 text

Compute Node Compute Node Compute Node Leader Node Common BI Tools JDBC/ODBC

Slide 73

Slide 73 text

Certiﬁed for use with Microstrategy.

Slide 74

Slide 74 text

Data compression.

Slide 75

Slide 75 text

Automated backup to S3.

Slide 76

Slide 76 text

Data encrypted in transit & at rest.

Slide 77

Slide 77 text

Automated failover.

Slide 78

Slide 78 text

Compute Node Compute Node Compute Node Leader Node Common BI Tools JDBC/ODBC

Slide 79

Slide 79 text

Compute Node Compute Node Compute Node Leader Node Common BI Tools JDBC/ODBC

Slide 80

Slide 80 text

Compute Node Compute Node Compute Node Leader Node Common BI Tools JDBC/ODBC

Slide 81

Slide 81 text

Elastic.

Slide 82

Slide 82 text

Compute Node Compute Node Compute Node Leader Node Common BI Tools JDBC/ODBC

Slide 83

Slide 83 text

Compute Node Compute Node Compute Node Leader Node Common BI Tools JDBC/ODBC Compute Node Compute Node

Slide 84

Slide 84 text

Compute Node Compute Node Compute Node Leader Node Common BI Tools JDBC/ODBC

Slide 85

Slide 85 text

Data warehouse node types.

Slide 86

Slide 86 text

15GB RAM 2TB local attached storage 3 drives 2 virtual cores High Storage Extra Large (XL)

Slide 87

Slide 87 text

High Storage Extra Large (XL) 15GB RAM 2TB local attached storage 3 drives 2 virtual cores 8 High Storage Extra Large (8XL) 120GB RAM 16TB local attached storage 24 drives 16 virtual cores

Slide 88

Slide 88 text

Pay as you go.

Slide 89

Slide 89 text

2 TB nodes 16 TB nodes On-demand $0.850 $6.80 1 Year Reservation $0.50 $4.00 3 Year Reservation $0.228 $1.824 Hourly Prices

Slide 90

Slide 90 text

2 TB nodes 16 TB nodes On-demand $0.850 $6.80 1 Year Reservation $0.50 $4.00 3 Year Reservation $0.228 $1.824 Hourly Prices

Slide 91

Slide 91 text

$999 per TB

Slide 92

Slide 92 text

Don’t pay for the leader node.

Slide 93

Slide 93 text

No additional storage charge for backups of active clusters.

Slide 94

Slide 94 text

VPC ready.

Slide 95

Slide 95 text

Low cost. Easy to use.

Slide 96

Slide 96 text

Focus on analysis.

Slide 97

Slide 97 text

Private beta today.

Slide 98

Slide 98 text

Available early this year.

Slide 99

Slide 99 text

aws.amazon.com/redshift

Slide 100

Slide 100 text

2 billion row dataset. 6 representative queries.

Slide 101

Slide 101 text

Compared to 32 nodes. 128 CPUs. 4.2 TB RAM. 1.6 PB storage. 2 billion row data set. Amazon Redshift: 2 instance cluster 12x to 150x faster

Slide 102

Slide 102 text

29 minutes 58 seconds down to 12 seconds

Slide 103

Slide 103 text

Data security. III

Slide 104

Slide 104 text

Security is our number one priority.

Slide 105

Slide 105 text

Shared responsibility.

Slide 106

Slide 106 text

No content

Slide 107

Slide 107 text

Choose your region.

Slide 108

Slide 108 text

Availability zones.

Slide 109

Slide 109 text

ITAR FIPS 140-2 MPAA ISO 27001 SOC 2 ISAE 3402 PCI DSS HIPAA FISMA Moderate

Slide 110

Slide 110 text

No content

Slide 111

Slide 111 text

“You basically turn yourself into a polymorphic surface to which the attack guy has a much tougher time getting at. That, ultimately, is the real key advantage to drive security and make things much better for us across the board.” Gus Hunt, CTO Central Intelligence Agency

Slide 112

Slide 112 text

Virtual Private Cloud.

Slide 113

Slide 113 text

Network isolated environment.

Slide 114

Slide 114 text

Public and private subnets.

Slide 115

Slide 115 text

Redshift, relational databases, Hadoop can run inside the VPC.

Slide 116

Slide 116 text

Extend your VPN.

Slide 117

Slide 117 text

Identity and access federation.

Slide 118

Slide 118 text

Identity and access management.

Slide 119

Slide 119 text

Data movement. IV

Slide 120

Slide 120 text

“How do I get my data into the cloud?”

Slide 121

Slide 121 text

Generated and stored in the AWS cloud.

Slide 122

Slide 122 text

Inbound transfer if free.

Slide 123

Slide 123 text

Multipart upload.

Slide 124

Slide 124 text

Aspera, IRODS.

Slide 125

Slide 125 text

Physical media.

Slide 126

Slide 126 text

AWS Direct Connect.

Slide 127

Slide 127 text

1Gbps or 10Gbps

Slide 128

Slide 128 text

Built in AZ replication.

Slide 129

Slide 129 text

Regional replication.

Slide 130

Slide 130 text

“How do I integrate my data?”

Slide 131

Slide 131 text

Amazon DynamoDB HDFS (Amazon EMR) Amazon S3 Amazon Redshift On Premise Amazon RDS

Slide 132

Slide 132 text

AWS Data Pipeline

Slide 133

Slide 133 text

Data-intensive orchestration & automation.

Slide 134

Slide 134 text

Reliable, scheduled data movement and analytics.

Slide 135

Slide 135 text

aws.amazon.com/datapipeline

Slide 136

Slide 136 text

aws.amazon.com

Slide 137

Slide 137 text

I Data, data everywhere

Slide 138

Slide 138 text

I II Collection & storage Data, data everywhere

Slide 139

Slide 139 text

I II III Data security Data, data everywhere Collection & storage

Slide 140

Slide 140 text

I II III IV Data movement Data, data everywhere Data security Collection & storage

Slide 141

Slide 141 text

Thank you.

Slide 142

Slide 142 text

to from ANALYTICS INTELLIGENCE get in touch [email protected] or @MZA AWS.AMAZON.COM