Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Steve Illingworth's presentation on Hadoop and Big Data

Michael Isvy
September 13, 2013

Steve Illingworth's presentation on Hadoop and Big Data

Used in Singapore on August 30th (see here for details: http://springhadoop.eventbrite.com/ )

Michael Isvy

September 13, 2013
Tweet

More Decks by Michael Isvy

Other Decks in Programming

Transcript

  1. 2 © Copyright 2013 EMC Corporation. All rights reserved. Big

    Data, Fast Data and the path to new business Value Steve Illingworth, Chief Technology Officer Pivotal APJ September 2013
  2. 3 © Copyright 2013 EMC Corporation. All rights reserved. Cloud

    Era: Changing I.T. MAINFRAME CLIENT-SERVER & WEB Automation of financial accounts Automation of most paper processes: ERP, CRM, Email, … Mainframes ISAM Relational Databases Mini’s & PC’s Software-Defined Datacenter New Data-fabrics CLOUD ERA New Experiences New Biz Models pioneered by new Consumer Internet giants
  3. 4 © Copyright 2013 EMC Corporation. All rights reserved. CLOUD

    ERA Innovation: From Consumer Internet Giants MAINFRAME Software-Defined Datacenter New Data-fabrics New Experiences New Biz Models pioneered by new Consumer Internet giants CLIENT-SERVER & WEB Store and reason over very large amounts of data Rapid App Development The Software Defined Data Center Operate at scale in highly automated way
  4. 5 © Copyright 2013 EMC Corporation. All rights reserved. Software-Defined

    Datacenter New Data-fabrics The Software Defined Data Center New Driver of Change… Rapid App Development Ingest huge number of events and react in real-time MAINFRAME CLIENT-SERVER & WEB CLOUD ERA Store and reason over very large amounts of data Operate at scale in highly automated way New Experiences New Biz Models pioneered by new Consumer Internet giants Internet of things Pervasive Telemetry
  5. 6 © Copyright 2013 EMC Corporation. All rights reserved. Software-Defined

    Datacenter New Data-fabrics The Software Defined Data Center Enterprise Needs MAINFRAME CLIENT-SERVER & WEB Store and reason over very large amounts of data Rapid App Development Ingest huge number of events and react in real-time CLOUD ERA Interact with Legacy Choice of Cloud AWS VMW …ETC Operate at scale in highly automated way New Experiences New Biz Models pioneered by new Consumer Internet giants Internet of things Pervasive Telemetry
  6. 7 © Copyright 2013 EMC Corporation. All rights reserved. Software-Defined

    Datacenter New Data-fabrics The Software Defined Data Center New Experiences New Biz Models pioneered by new Consumer Internet giants Internet-of-things Pervasive telemetry MAINFRAME CLIENT-SERVER & WEB CLOUD ERA Operate at Scale in a highly automated way AWS VMW ...ETC Pivotal: A New Platform Pivotal Platform Store and reason over very large amounts of data Rapid App Development Ingest huge number of events and react in real-time Interact with Legacy Choice of Cloud
  7. 8 © Copyright 2013 EMC Corporation. All rights reserved. Software-Defined

    Datacenter New Data-fabrics The Software Defined Data Center New Experiences New Biz Models pioneered by new Consumer Internet giants Internet-of-things Pervasive telemetry MAINFRAME CLIENT-SERVER & WEB CLOUD ERA Operate at Scale in a highly automated way AWS VMW ...ETC Pivotal: Integrating EMC & VMW Assets Pivotal Platform From VMware From EMC Big Data Fast Data Rapid Application Dev & Integration Cloud Abstraction & App Automation Store and reason over very large amounts of data Rapid App Development Ingest huge number of events and react in real-time Choice of Cloud Interact with Legacy
  8. 9 © Copyright 2013 EMC Corporation. All rights reserved. Software-Defined

    Datacenter New Data-fabrics The Software Defined Data Center New Experiences New Biz Models pioneered by new Consumer Internet giants Internet-of-things Pervasive telemetry MAINFRAME CLIENT-SERVER & WEB CLOUD ERA Operate at Scale in a highly automated way AWS VMW ...ETC Pivotal: Integrating EMC & VMW Assets Pivotal Platform Big Data Fast Data Rapid Application Dev & Integration Gemfire vFabric From VMware From EMC Cloud Abstraction & App Automation
  9. 10 © Copyright 2013 EMC Corporation. All rights reserved. PIVOTAL

    TECHNOLOGY MISSION A complete software-centric application+data platform . . . •  Driven by a strong open-source foundation •  Hadoop-centric data infrastructure & services (spanning storage, real-time operational, & analytic) •  Integrated PaaS supporting applications developed in multiple languages & frameworks •  Open model that enables the integration of self-service utilities to be plugged in and deployed within existing legacy environments •  Analytics & visualization instrumented by default •  Cloud enabled, whether deployed in the datacenter or the public cloud
  10. 12 © Copyright 2013 EMC Corporation. All rights reserved. ENTERPRISE

    DATA ARCHITECTURE Analytic Data Marts MPP Database Operational Intelligence In-Memory DB Run-Time Applications In-Memory Object Enterprise Data Warehouse RDBMS Data Staging Platform Traditional BI/Reporting Data Visualization Data Ingestion System Stream/CEP
  11. 13 © Copyright 2013 EMC Corporation. All rights reserved. Analytic

    Data Marts Operational Intelligence Run-Time Applications Enterprise Data Warehouse Data Staging Platform Traditional BI/Reporting Data Visualization Data Ingestion System PIVOTAL PRODUCT PORTFOLIO TODAY
  12. 14 © Copyright 2013 EMC Corporation. All rights reserved. PIVOTAL

    DATA FABRIC EVOLUTION Analytic Data Marts SQL Services Operational Intelligence In-Memory Services Run-Time Applications In-Memory Services Enterprise Data Warehouse RDBMS Continues to serve as system of record HDFS Data Staging Platform Data Mgmt. Services Data Visualization Compliance and financial reporting Traditional BI/Reporting Hadoop++ Platform Data Visualization Stream Ingestion Streaming Services
  13. 16 © Copyright 2013 EMC Corporation. All rights reserved. CLOUDFOUNDRY

    OVERVIEW Ÿ  Re-focusing efforts towards addressing enterprise needs and establishing strong OSS ecosystem Ÿ  Incrementally adopted at every step of the way, with an open utility approach to services – “white box PaaS” Ÿ  Radical focus on developer appeal and business agility including new apps and new data Ÿ  “Pivotal CF” as a commercial offering anchored by Spring, messaging and analytics as core differentiation
  14. 17 © Copyright 2013 EMC Corporation. All rights reserved. DOUBLING

    DOWN ON SPRING IN 2013 Big Java EE stacks Oracle WLS IBM WAS Lean Java stacks Tomcat, tc TomEE, Virgo Jetty Glassfish JBoss Mobile & APIs JVM & V8: Groovy & Grails Netty, Vert.x Akka, Play, Scala Meteor, Parse, .. App Service Infra Use cases: Netflix Twitter/Yammer LinkedIn Google Data & Streams Hadoop Pivotal HD Hortonworks Cloudera/MapR Groovy DSLs Spring will be the Java productivity platform for new runtimes across data and cloud infrastructure use cases World leading simplicity & Integrated Cloud Foundry Enhanced OSS and commercial support Ÿ  Enhanced Open Source Software (OSS) platform with Spring 4 launch Ÿ  New commercial app platform for cloud, API and big data cases Ÿ  Extend Spring leadership to where the puck is heading
  15. 18 © Copyright 2013 EMC Corporation. All rights reserved. EVOLUTION

    OF VFABRIC Ÿ  Move the dial from “vFabric license is cheaper than IBM” to real value around better agility, data capability, lower opex Ÿ  Deliver multi-tenant elastic “as a service” to enterprise on premise and across any cloud infra (VMW, OpenStack, EC2) VMware vFabric Pivotal CF Spring Apps -aaS Messaging -aaS APIs
  16. 19 © Copyright 2013 EMC Corporation. All rights reserved. SPRING

    APPS FOR PIVOTAL CF Ÿ  Pivotal CF is the baseline app platform Ÿ  Now add: an excellent experience for Spring apps Ÿ  Customer case: consolidate existing Java sprawl into a farm, and enable new app development with CF self-service agility Ÿ  Make Java App Runtimes and Spring first class on CF –  Complete: commercial support for full stack up to Java app –  Robust: integrated Java enterprise capability: HA, monitoring, .. –  Consumer Grade Spring – apps, upgrades, rollbacks “in Google time”
  17. 20 © Copyright 2013 EMC Corporation. All rights reserved. PIVOTAL

    SUMMARY Ÿ  We provide the platform modern enterprises want, for the new apps you must build, and at the velocity you need –  New data uniting applications and analytics centered on Hadoop –  New apps delivered to the customer’s preferred infrastructure, including VMWare, Open Stack, and EC2 Ÿ  We are anchored by Spring for your Java enterprise Ÿ  We are aligned with the Facebook/Google “consumer grade” era. Yet we are able to connect into your enterprise reality and deliver value from there incrementally.
  18. 21 © Copyright 2013 EMC Corporation. All rights reserved. Big

    Data, Fast Data and the path to new business Value Steve Illingworth, Chief Technology Officer Pivotal APJ September 2013