Big Data, analytics and 4th generation data warehousing by Martyn Jones at Big Data Spain 2015

Big Data, analytics and 4th generation data warehousing by Martyn Jones at Big Data Spain 2015

The ultimate business success of Big Data in business will depend on our ability to successfully bring about the realignment and placement of Big Data into a more generalized architectural framework, one that coalesces strategic, technical and management elements of data warehousing (DW 3.0), business intelligence, textual analysis and statistical analysis into a coherent, synergistic and usable whole.

Session presented at Big Data Spain 2015 Conference
16th Oct 2015
Kinépolis Madrid
http://www.bigdataspain.org
Event promoted by: http://www.paradigmatecnologico.com
Abstract: http://www.bigdataspain.org/program/fri/slot-26.html

Cb6e6da05b5b943d2691ceefa3381cad?s=128

Big Data Spain

October 21, 2015
Tweet

Transcript

  1. None
  2. Big Data, Analytics and 4th Generation Data Warehousing Martyn Jones

    Big Data Spain 2015
  3. agenda ∙ Imperatives. ∙ Data value chains. ∙ Resources. ∙

    4th Generation Data Warehousing. ∙ Analytics Data Store / Big Data. ∙ Information Supply Framework. Friday 16th from 12:30 pm to 13:15 pm Room 25 - Technical 0 5 10 15 20 25 30 35 40 45
  4. business background

  5. the ages of data B . C . L i

    f e o f B r i a n A . D .
  6. C h a n g e I n s i

    g h t P o t e n t i a l l y u s e f u l Simplicity A b u n d a n t V o l u m e V e l o c i t y V a r i e t y
  7. framework O b t a i n I n t

    e g r a t e A n a l y s e P r e s e n t D A T A D A T A D A T A
  8. the road to Big Data success… S t r a

    t e g i c T a c t i c a l O p e r a t i o n a l A n a l y t i c s A r c h i t e c t e d M a n a g e d I n t e g r a t i o n D a t a
  9. scope BIZ DATA DW BIG DATA STATS PRES

  10. Business Imperatives A good place to start

  11. what’s important to business? BE NOTICED CASH FLOW BE NOTICED

    CASH FLOW BE NOTICED CASH FLOW
  12. what else is important to business? Market share Differentiation Ability

    to execute Liquidity Profitability Time and place utility React to competitive threats Enhance service scope Improving customer service Respond to price pressure Segmentation of n Addressing short-term attention spans Ability to respond to irrationality Be noticed Cash flow Risk Legislation No press Bad press Customer centricity Front office empowerment Excellence Channel excellence Operational excellence Product excellence Cultures IT business value Base protection Expansion Diversification Consolidation
  13. Augmented Competitive Forces Competition from within the industry Suppliers Buyers

    Replacements Potential entrants Threat of replacement product or service Threat of new entrants Bargaining power Bargaining power Sources: Michael Porter;Martyn R Jones and others Rivalry with existing competitors Pressure groups Media Government Power to change the game Exposure
  14. McKinsey 7S Framework Culture

  15. differentiated capabilities

  16. operating models Customer segments Channels Products Services Organsational design Processes

    Data & information Physical assets Development Deployment Organsational design Performance management Information technology Business model Operating model People model Customers Systems People Processes Organisation
  17. objectives 1. Information awareness corresponding to areas of operation and

    spheres of control 2. Comprehensive data and information supply framework 3. Continually seek to maintain and then improve data’s contribution to business
  18. Business data everywhere Where, when, what, who, why... how?

  19. Data I n t e r n a l P

    a s t E x t e r n a l P r e s e n t S h a r e d F u t u r e
  20. Data O p e r a t i o n

    a l O n l i n e B i g D a t a A r c h i v e d D a r k D a t a U n m a n a g e d
  21. Data A r c h i v e s S

    o c i a l M e d i a D o c u m e n t s M a c h i n e L o g M e d i a S e n s o r B u s i n e s s A p p l i c a t i o n s D a t a S t o r a g e P u b l i c W e b
  22. Activities, Abstractions and Relations

  23. Velocity Volume Variety Adequacy Ambiguity Small Availability Accuracy Relevance Persistence

    Reliability Value Obtuseness Listo Complexity Utility Descriptiveness Big Velocidad Volumen Variedad Adecuación Ambigüedad Precisión Disponibilidad Exactitud Relevancia Persistencia Confiabilidad Valor Obtuso Smart Complejidad Utilidad Descriptivo Grande D a t a Facets of Big Data Facets of Data
  24. None
  25. B I G D A T A I n t

    e r n e t o f T h i n g s C L O U D S t a t i s t i c s D a t a W a r e h o u s i n g P r e s e n t a t i o n D a t a S u p p l y F r a m e w o r k
  26. Building Bill’s Data Warehouse 25 years of... sometimes getting it

    right
  27. Enterprise Data Warehousing – AS IS S u b j

    e c t o r i e n t e d S t r a t e g i c d e c i s i o n m a k i n g I n t e g r a t e d T i m e v a r I a n t N o n – v o l a t i l e
  28. Operational Systems Data Warehouse Purchasing HR Credit Order Processing Marketing

    Sales Logistics Billing Arrangements Products Party Time Geography Transactions Subject oriented
  29. Operational Systems Data Warehouse Euro Account Customer: Customer: Village Bank

    GmbH Country code: D Mutual Fund Customer: Customer: Village Bankers Region: Westphalia NTIP Customer: Customer: Village Bank International Country: Germany Account: Number Customer Type 230956 441353 Euro 010555 441353 MF 291284 441353 NTIP Party: Number: 100441353 Name: Village Bank GmbH Country: Germany Integrated
  30. Operational Systems Data Warehouse 0 10 20 30 40 50

    60 70 80 90 100 Trading Activity Snapshots: Date Security Amount 2006.09.01 MartyBank 79.000.000 2006.09.02 MartyBank 92.000.000 2006.09.03 MartyBank 44.000.000 2006.09.04 MartyBank 39.000.000 2006.09.05 MartyBank 80.000.000 Trading Activity: MartyBank Time variant
  31. Operational Systems Data Warehouse Order Processing Create Replace Update Delete

    Orders Read Read Read Read Write Read Non-volatile
  32. Data Warehousing 2.0 Data Sources Structured Data ETL Extract Transform

    Load Internal ODS ODS EDW ETL Extract Transform Load Data Marts Structured Data Unstructured Data Mart Data Mart Report Repository Reports & Extracts Stats Data selection and representation Data analytics Report set and extract creation Service Push / Pull Technology Visualisation Annotation Users Internal Clients Other stakeholders Metadata, Workflow/Process Control and CIW Management Metadata Process ÊDW Management Staging Staged Data EDW Unstructured EDW Data Mart Structured Data Unstructured
  33. Enterprise Data Warehousing – AS A BODGE G e t

    d a t a W o n d e r w h y i t ‘ s n o t m e e t i n g e x p e c t a t I o n s D u m p d a t a Q u e r y d a t a V i s u a l i s e d a t a
  34. Enterprise Data Warehousing – AS A BODGE DW BODGER TEAM

    HADOOP TEAM We built a data dog house using Oracle and IBM technology and we called it a data warehouse We can do data warehousing too and it will be cheaper, faster and smarter
  35. Data Supply Framework A data architecture for data sourcing, transformation,

    integration, storage, search, analysis and presentation
  36. Data Supply Framework Operational Data Store Data Warehouse Business Intelligence

    Data logistics Operational applications Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu Cambriano Energy 2015 - http://www.cambriano.es All information and data consumers All information consumers All digital data All data processing, enrichment and information creation
  37. Internal digital data Data Supply Framework External digital data Data

    logistics Operational Data Store Data Warehouse Analytics Data Store Data Marts Statistical Analysis Business Intelligence Scenarios Data logistics Primary data flow Secondary data flow Operational applications Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu Cambriano Energy 2015 - http://www.cambriano.es
  38. EDW ADS DM DM DM Statistical analysis ETL T/ETL ET(A)L

    Staging & Reduction Signal Appliance Message Adapter Message Queue Infrastructure Data Write back Message Adapter Message Queue OLTP Staging ODS ETL T/ETL Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 TL Data Supply Framework Data Sources 4th Generation Data Warehousing Data Sources Core Statistics Cambriano Energy 2015
  39. Core Data Sourcing Comprehensive data acquisition and transformation

  40. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DW 3.0 Information Supply Framework Cambriano Energy 2015 Core Data Warehousing Core Statistics Data Sources Message Adapter
  41. 4th Generation Data Warehousing Providing a solid foundation for strategic,

    tactical and operational decision making
  42. Enterprise Data Warehousing – 4 GEN S u b j

    e c t o r i e n t e d S t r a t e g i c , t a c t i c a l & o p e r a t i o n a l s u p p o r t I n t e g r a t e d T i m e v a r i a n c e & t i m e p e r s p e c t i v e s C o n s t r a i n e d v o l a t i l i t y C l a s s i f i c a t i o n s c h e m a R u l e b a s e d t r a n s f o r m a t i o n
  43. 4th Generation EDW Interpretation Prediction Diagnosis Design Planning Monitoring Debugging

    Repairing Instruction Control S t r a t e g y T a c t i c s O p e r a t i o n s
  44. Using, applying and measuring Big Data Big Data Big Data

    Predictive Analytics Predictive Analytics Outcomes EDW 4.0 EDW 4.0 E(A)TL
  45. Using, applying and measuring Big Data Predictive analytics Select predictions

    Define trackable actions Apply outcomes and actions to EDW 4 Accumulate campaign Big Data Descriptive analytics Select findings Combine with trackable actions Apply outcomes and actions to EDW 4 Run campaign Analyse campaign and performance of Big Data analytics
  46. Forecasts and results – from all perspectives -400 -300 -200

    -100 0 100 200 300 400 500 01/15 02/15 03/15 04/15 05/15 06/15 07/15 08/15 09/15 10/15 11/15 12/15 01/16 02/16 03/16 04/16 05/16 06/16 Cambriano Big Data Campaign 2015-2016 Forecast Actual Strategy BD Costs Benefit Values Relativity Dimensions Hierarchies Structures Past Future
  47. Using, applying and measuring •Combining Big Data analytics with Data

    Warehousing 4.0 •Planning and managing initiatives •Measuring, analysing and reporting the effectiveness of business initiatives •Measuring, analysing and reporting the tangible contribution of the Big Data analytics process to the creation of business value
  48. Big Data and Core Statistics A multi-faceted data theatre for

    ad-hoc, speculative and immediate operational analytics
  49. Internal digital data Data Supply Framework External digital data Data

    logistics Operational Data Store Data Warehouse Analytics Data Store Data Marts Statistical Analysis Business Intelligence Scenarios Data logistics Primary data flow Secondary data flow Operational applications Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu Cambriano Energy 2015 - http://www.cambriano.es
  50. DSF 4.0 Data Value Chains Published by goodstrat.com Martyn Richard

    Jones 2015 – martynjones.eu Cambriano Energy 2015 - http://www.cambriano.es DATA INFORMATION KNOWLEDGE Requires context Requires interpretation Requires wisdom Relevant Correct Usable Irrelevant Incorrect Useless Meaningless Misleading Wrong Value? Value? Value?         
  51. DSF 4.0 Data Assets in MOSCOW Published by goodstrat.com Martyn

    Richard Jones 2015 – martynjones.eu Cambriano Energy 2015 - http://www.cambriano.es RISK ASSET SECURE BAU Assurance Highest High Medium/Low Very low/None MUST SHOULD COULD WON’T Yes Yes Maybe Maybe/No Yes Yes Yes Maybe/No Yes Yes Yes Maybe/No
  52. DSF 4.0 Data Assets in MOSCOW Published by goodstrat.com Martyn

    Richard Jones 2015 – martynjones.eu Cambriano Energy 2015 - http://www.cambriano.es RISK ASSET SECURE BAU Assurance Highest High Medium/Low Very low/None MUST SHOULD COULD WON’T Yes Yes Maybe Maybe/No Yes Yes Yes Maybe/No Yes Yes Yes Maybe/No
  53. DSF 4.0 Data Supply Framework External digital data Data logistics

    Operational Data Store Data Warehouse Analytics Data Store Data Marts Statistical Analysis Business Intelligence Scenarios Data logistics Primary data flow Secondary data flow Operational applications Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu Cambriano Energy 2015 - http://www.cambriano.es OLTP Applications ‘What if’ analysis MIS / Reporting Visualisation Publication º All digital data
  54. Internal digital data DSF 4.0 Data Supply Framework External digital

    data Data logistics Operational Data Store Data Warehouse Analytics Data Store Data Marts Statistical Analysis Business Intelligence Scenarios Data logistics Primary data flow Secondary data flow Operational applications Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu Cambriano Energy 2015 - http://www.cambriano.es All information consumers º All digital data
  55. Internal digital data External digital data Primary data flow Secondary

    data flow Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu Cambriano Energy 2015 - http://www.cambriano.es º Statistics Data Science Big Data Small Data Smart Data This Data That Data That department Messing with data Map Fatten Retrospect Reports Alerts Visualisation Analytics This department The other department Map Reduce DSF 4.0 Data Supply Framework
  56. DSF 4.0 Data Supply Framework Operational Data Store Data Warehouse

    Business Intelligence Data logistics Operational applications Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu Cambriano Energy 2015 - http://www.cambriano.es All information and data consumers All information consumers All digital data All data processing, enrichment and information creation
  57. EDW ADS DM DM DM Statistical analysis ETL T/ETL ET(A)L

    Staging & Reduction Signal Appliance Message Adapter Message Queue Infrastructure Data Write back Message Adapter Message Queue OLTP Staging ODS ETL T/ETL Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 TL DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Message Adapter Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu Cambriano Energy 2015 - http://www.cambriano.es
  58. EDW ADS DM DM DM Statistical analysis ETL T/ETL ET(A)L

    Staging & Reduction Signal Appliance Message Adapter Message Queue Infrastructure Data Write back Message Adapter Message Queue OLTP Staging ODS ETL T/ETL Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 TL DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Message Adapter Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu Cambriano Energy 2015 - http://www.cambriano.es
  59. EDW ADS DM DM DM Statistical analysis ETL T/ETL ET(A)L

    Staging & Reduction Signal Appliance Message Adapter Message Queue Infrastructure Data Write back Message Adapter Message Queue OLTP Staging ODS ETL T/ETL Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 TL DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Message Adapter Data Sources – This element covers all the current sources, varieties and volumes of data available which may be used to support processes of 'challenge identification', 'option definition', decision making, including statistical analysis and scenario generation. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  60. EDW ADS DM DM DM Statistical analysis ETL T/ETL ET(A)L

    Staging & Reduction Signal Appliance Message Adapter Message Queue Infrastructure Data Write back Message Adapter Message Queue OLTP Staging ODS ETL T/ETL Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 TL DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Message Adapter Core Data Warehousing – This is a suggested evolution path of the DW 2.0 model. It faithfully extends the Inmon paradigm to not only include unstructured and complex data but also the information and outcomes derived from statistical analysis performed outside of the Core Data Warehousing landscape. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  61. EDW ADS DM DM DM Statistical analysis ETL T/ETL ET(A)L

    Staging & Reduction Signal Appliance Message Adapter Message Queue Infrastructure Data Write back Message Adapter Message Queue OLTP Staging ODS ETL T/ETL Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 TL DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Message Adapter Core Statistics – This element covers the core body of statistical competence, especially but not only with regards to evolving data volumes, data velocity and speed, data quality and data variety. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  62. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DW 3.0 Information Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu INTO THE ZONE!
  63. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Complex Data – This is unstructured or highly complexly structured data contained in documents and other complex data artefacts, such as multimedia documents. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  64. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Event Data – This is an aspect of Enterprise Process Data, and typically at a fine-grained level of abstraction. Here are the business process logs, the internet web activity logs and other similar sources of event data. The volumes generated by these sources will tend to be higher than other volumes of data, and are those that are currently associated with the Big Data term, covering as it does that masses of information generated by tracking even the most minor piece of 'behavioural data' from, for example, someone casually surfing a web site. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  65. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Infrastructure Data – This aspect includes data which could well be described as signal data. Continuous high velocity streams of potentially highly volatile data that might be processed through complex event correlation and analysis components. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  66. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Event Applicance – This puts the dynamic data collation, selection and reduction functionality as close to the point of event data generation as physically possible. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  67. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Signal Applicance – This puts the dynamic data collation, selection and reduction functionality as close to the point of continuous streaming data generation as physically possible. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  68. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DW 3.0 Information Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Distributed Inter Process Communication – Different forms of messaging allow high volumes of data to be transmitted in near real time. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  69. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Staging and Reduction – Traditional data staging combined with in-line data reduction. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  70. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter ET(A)L – Extending ETL to include data analytics components tightly integrated into parallel ETL job streams. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  71. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter ADS – The Analytics Data Store. 1. Statistics oriented 2. Integrated by focus area 3. Variable volatility 4. Time variant Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  72. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Statistical Analysis – Qualitative analysis. Diagnostic analysis, predictive analysis, speculative analysis, data mining, data exploration, modelling. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  73. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DSF 4.0 Data Supply Framework Core Data Warehousing Core Statistics Data Sources Message Adapter Scenarios and outcomes – 1. Snapshots of outcomes of scenario analysis as the process of analyzing possible future events by generating alternative possible outcomes. 2. Captured outcomes of statistical analysis. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Martyn Richard Jones 2015 – martynjones.eu
  74. ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message

    Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 DSF 4.0 Data Supply Framework Martyn Richard Jones 2015 – martynjones.eu Core Data Warehousing Core Statistics Data Sources Message Adapter Write back – The ability to append data, update data and enrich data within the Analytics Data Store, and to provide scenario data to the Core Data Warehousing. Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com
  75. DSF 4-0 – Core Statistics: Analytics Data Store Martyn Richard

    Jones 2015 – martynjones.eu ADS Statistical analysis ET(A)L Staging & Reduction Signal Appliance Message Adapter Message Queue Infrastructure Data Write back Complex data Event Data Event Appliance Scenario 1 Scenario 2 Scenario 3 Core Data Warehousing Core Statistics Data Sources Message Adapter Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com
  76. DSF 4.0 – Analytics Data Store Martyn Richard Jones 2015

    – martynjones.eu Cambriano Energy 2015 - http://www.cambriano.es Published by goodstrat.com Distributed File System Non-relational distributed file storage / NoSQL DFS (Including ‘refractoring’ of Unix primitives) Unix File Store POSIX compliant Document DBMS Graph DBMS Key-Value DBMS In-memory Column Oriented Relational DBMS Relational DBMS (MPP/SMP/Hybrid) Object DBMS POSIX compliant Unix / Linux primitives Relational DBMS
  77. DSF 4.0 – What’s important? Cambriano Energy 2015 - http://www.cambriano.es

    Data Warehouse Martyn Richard Jones 2015 – martynjones.eu Published by goodstrat.com Business Intelligence Operational Data Store Analytics Data Store Statistical Analysis Dark Data Big Data Internet of Things Knowledge Management Structured Intellectual Capital Cloud
  78. Summary A good place to end, for now

  79. Summary • Consider everything • Question everything • Never stop

    hypothesising • Never stop testing • For every initiative have a business imperative • Make continuous engagement and involvement a goal
  80. Muchas gracias Many thanks Big Data Spain 2015

  81. Big Data, Analytics and 4th Generation Data Warehousing Big Data

    Spain 2015