Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Big data on the rise: Views of emerging trends ...

John Mertic
December 07, 2017

Big data on the rise: Views of emerging trends and predictions from real-life end users

There are some key trends that are emerging in 2017 within the Hadoop and big data ecosystem, which center around the increase of the use of the cloud. These trends are the underpinning for a larger shift toward purpose-driven products positioned as the core of an organization’s data strategy. While these trends are clearest among early adopters with mature deployments, what about those more traditional end user organizations in the midst of a digital transformation? What are their views on these trends? How are their organizations reconciling the needs and desires to pave a path forward?

John Mertic and Cupid Chan present findings on big data trends, based on data from the ODPi End User Advisory Board (TAB). John and Cupid share real end-user perspectives from companies like GE on how they are using big data tools, the challenges they face, and where they are looking to focus investments—all from a vendor-neutral viewpoint.

John Mertic

December 07, 2017
Tweet

More Decks by John Mertic

Other Decks in Technology

Transcript

  1. John Mertic, Director, ODPi @jmertic | @ODPIorg Cupid Chan, Managing

    Partner, 4C Decision @cupidckchan The open ecosystem of big data Big data on the rise: Views of emerging trends and predictions from real-life end users
  2. @ODPiOrg TOP 3 KEY CONSIDERATIONS FOR DATA PROJECTS What problem

    am I trying to solve? What data do I have and where is it coming from? What questions need answered?
  3. Create Collect/Ingest/ Process/ETL Analysis Apply /Use Share/Publish/ Access Store/Archive Preserve

    Destroy/ Delete/Purge Quality Security Governance Management Architecture and Schema Compliance Metadata Strategy Platform
  4. @ODPiOrg Disclaimer 9 This Strata session was prepared or accomplished

    by Cupid Chan in his personal capacity. The opinions expressed in this talk are the author's own and do not reflect the view of any United States Government Agency or Department.
  5. Give a Man a Fish, and You Feed Him for

    a Day… Teach a Man To Fish, and You Feed Him for a Lifetime – Chinese Proverb
  6. Give a Computer a Code, and You Help It Solving

    for one Situation… Teach a Computer To Code, and You Help It Solving for a Collection of Situations – Cupid Proverb
  7. @ODPiOrg Cupid’s Prediction on Emerging Trends for Big Data in

    the next 5 years 23 AI + BI Artificial Intelligence + Business Intelligence
  8. Create Collect/Ingest/ Process/ETL Analysis Apply /Use Share/Publish/ Access Store/Archive Preserve

    Destroy/ Delete/Purge Quality Security Governance Management Architecture and Schema Compliance Metadata Strategy Platform
  9. @ODPiOrg Cupid’s Prediction on Emerging Trends for Big Data in

    the next 5 years 30 AI + BI Artificial Intelligence + Business Intelligence = CI (Cognitive Intelligence) Intelligence by combining the Speed of how a Machine Learns and Direction Intuited from Human Insight
  10. @ODPiOrg WHY ADD A FEEDBACK LOOP Modern data tools and

    platforms are non-trivial ( by design ) Enterprise deployment pattern considerations don’t make it upstream Interoperability standards are inconsistent, even though platform vendors often derive from the same codebase
  11. @ODPiOrg SUSTAINABLE DATA INVESTMENT Successful Projects depend on members, developers,

    infrastructure to develop products that the market will adopt. Downstream constituents (ISVs/IHVs, Solution Providers, End Users) as a part of this helps them be an active participant in their software supply chain.
  12. @ODPiOrg ODPi PROGRAMS: INVESTMENT IN UPSTREAM APACHE AND BIGTOP ODPi

    is a proud Gold sponsor of the ASF. In addition, ODPi is investing $50,000 to fund developer work with the world’s top Apache and big data developers and architects to expand Bigtop’s functionality and usability. Projects funded include: § Adding containerization support for big data projects built from BigTop § Strengthening the testing framework to improve interoperability These investments amongst others put ODPi investing nearly $100,000 annually back into upstream communities.
  13. @ODPiOrg ODPi PROGRAMS: ADVANCING DATA GOVERNANCE ODPi is investing in

    enabling cross industry collaboration to solve the key data governance challenges facing enterprises today. Focuses include: • Shared R&D on metadata models to support current and upcoming regulations such as GDPR • Driving standardized APIs across metadata repositories ( both Hadoop and outside Hadoop )
  14. @ODPiOrg HOW COMPANIES AND DEVELOPERS CAN GET INVOLVED WITH ODPI

    Have your organization support ODPi https://www.odpi.org/about/join Check out ODPi Data Governance on GitHub https://github.com/odpi/data-governance Join the ODPi User Advisory Board https://www.odpi.org/odpi-compliance-directory/odpi-end-users See how Apache Atlas is implementing this https://cwiki.apache.org/confluence/display/ATLAS/Open+Metadata+and+Governance