Become an Equilibrista: Find the Right Balance in The Analytics Tech Ecosystem

Slide 1

Slide 1 text

Francesco Tisiot - @FTisiot Analytics Tech Lead Become an Equilibrista Find the Right Balance in The Analytics Tech Ecosystem

Slide 2

Slide 2 text

Verona, Italy http://ritt.md/ftisiot Over10 Years in Analytics ft@rittmanmead.com @FTisiot Oracle ACE Director ITOUG Board President Francesco Tisiot Analytics Tech Lead

Slide 3

Slide 3 text

Data Engineering Analytics Data Science www.rittmanmead.com info@rittmanmead.com @rittmanmead

Slide 4

Slide 4 text

A bit of History

Slide 5

Slide 5 text

Excel Hell

Slide 6

Slide 6 text

https://www.rittmanmead.com/blog/2018/11/where-are-my-users-coming-from-analysing-obiee-connections-methods/

Slide 7

Slide 7 text

Unique Source of Truth

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

Calculated from Scratch Calculated from Scratch AGREEM ENT

Slide 12

Slide 12 text

Sourced from Certiﬁed Datasource Calculated from Scratch

Slide 13

Slide 13 text

Approach to Data Certiﬁcation

Slide 14

Slide 14 text

Top-Down Approach Data Model Deﬁnition Implementation Analysis Centralized BI OBIEE - RPD LOOKER

Slide 15

Slide 15 text

Bottom-Up Approach Datasource Creation Analysis Datasource Publication Self Service Oracle Data Visualization Tableau

Slide 16

Slide 16 text

Top-Down vs Bottom-Up KPI Duplication Security Time To KPI Reusability Performance

Slide 17

Slide 17 text

Reusability / Security DataSource A DataSource B Shared Dimension(s) Centralised BI

Slide 18

Slide 18 text

Reusability / Security DataSource A DataSource B Dim X Self Service BI Dim X

Slide 19

Slide 19 text

Self-Service BI

Slide 20

Slide 20 text

Self Service Centralised

Slide 21

Slide 21 text

The Right Tool: OAC

Slide 22

Slide 22 text

Sourcing from Central System Select "Dim Account"."Account Code", "Fact Churn"."Churn Prediction" from "Churn" No Formula Recalculation No Join, Filtering, Aggregation Knowledge Security Inheritance No Code Duplication

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

Hybrid

Slide 25

Slide 25 text

Self Service Centralised

Slide 26

Slide 26 text

Awareness

Slide 27

Slide 27 text

Documentation

Slide 28

Slide 28 text

RPD Metadata Dictionary

Slide 29

Slide 29 text

Suggestions

Slide 30

Slide 30 text

Complexity Key Factors Security Data Volumes Interest

Slide 31

Slide 31 text

Key Factors Centralised Self Service Complexity Low High Interest Narrow/Unique Wide/Repeated Security Easy Complex Data Volumes Small Big

Slide 32

Slide 32 text

Self Service Centralised ?

Slide 33

Slide 33 text

Control

Slide 34

Slide 34 text

Keeping Track Usage Tracking!

Slide 35

Slide 35 text

Usage Tracking SELECT DISTINCT USER_NAME, NVL(QUERY_SRC_CD, 'RPD ODBC') SOURCE, TRUNC(START_TS) TS FROM S_NQ_ACCT WHERE AND ( QUERY_SRC_CD IS NULL OR UPPER(QUERY_SRC_CD) IN ('SOAP', 'RAWSQL') ) AND QUERY_TEXT NOT LIKE '{CALL%' ORDER BY 3 DESC;

Slide 36

Slide 36 text

Self Service?

Slide 37

Slide 37 text

Calculation

Slide 38

Slide 38 text

”Dim Issue Status"."Issue Status”…> ”Fact Issue KPI"."# of Issues"…> Metadata

Slide 39

Slide 39 text

https://www.youtube.com/watch?v=mQY2vslle_8 REST Endpoints

Slide 40

Slide 40 text

https://www.youtube.com/watch?v=mQY2vslle_8 REST Endpoints curl “https:///datasetsvc/ public/api/v4/datasets” \ -u ‘:'

Slide 41

Slide 41 text

{ "provider": "managed", "requiresImpersonation": false, "semicolonDelimited": false, "customDelimiter": false, "name": "train", "namespace": "francesco.tisiot@rittmanmead.com", "business-model": "\"\"", "description": "Uploaded from train.csv.", "display-name": "train", "locale": "sys", "file-length": 460676, "type": "text", "data-last-modified": "2019-04-08T08:19:34.481+0000", "private-connection": false, "mime-type": "application/csv", "owner-guid": "francesco.tisiot@rittmanmead.com", "metadata-last-modified": "2019-04-08T08:22:20.928+0000", "bdp-profile": "{\"profile\":\"784c4626-ebd6-4a01-b5dc-1a3ccd0fee1a\"}", "is-temporary": false } REST Endpoints - CSV

Slide 42

Slide 42 text

{ …, "name": "OPEN_INCIDENTS", "namespace": "francesco.tisiot@rittmanmead.com", "business-model": "\"\"", "indexable": "BOTH", "description": "External Data Set", "display-name": "OPEN_INCIDENTS", …, "data-last-modified": "2019-03-19T16:07:22.732+0000", "connection-namespace": "francesco.tisiot@rittmanmead.com", "private-connection": false, "connection-name": "LGI", "provider-type": "db", "provider-name": "oracle", "owner-guid": "francesco.tisiot@rittmanmead.com", … } REST Endpoints - DB

Slide 43

Slide 43 text

REST Endpoints - DB Details api/v4/datasets/{dataset-uri} {dataset-uri} = ‘’.’’

Slide 44

Slide 44 text

REST Endpoints - DB Details

Slide 45

Slide 45 text

{ "id": "LGI", "leaf": false, "otype": "Schema", "name": "LGI", "children": [ { "id": "LGI.OPEN_INCIDENTS", "leaf": false, "otype": "dbtable", "name": "OPEN_INCIDENTS", "allColumns": [ { "datatype": "number", "name": "OPEN_INCIDENTS", "is-supported": "TRUE", "otype": "Column", "leaf": true, "id": "LGI.OPEN_INCIDENTS.OPEN_INCIDENTS" }, { "datatype": "varchar(30)", "name": "ISO_YEAR_WEEK", "is-supported": "TRUE", "otype": "Column", "leaf": true, "id": "LGI.OPEN_INCIDENTS.ISO_YEAR_WEEK" }, REST Endpoints - DB Details

Slide 46

Slide 46 text

REST Endpoints - Dataﬂows api/v4/dataflows/{dataflow-uri} {dataflow-uri} = ‘’.’’

Slide 47

Slide 47 text

{ "datagen-name": "Clean Wine Dataflow", "namespace": "francesco.tisiot@rittmanmead.com", "definition": { "steps": [ { "stepId": "InputDataset_0", "description": "winemag-data_first150k", "type": "InputDataset", "stepDisplayName": "winemag-data_first150k", "stepDescription": "", "datasetRef": "ds_0", "qualifiedTable": "XSA(ds_0).\"Columns\"", "datasetDescription": "Uploaded from winemag-data_first150k.xlsx.", "datasetType": "excel", "datasetDisplayName": "winemag-data_first150k", "inputType": "all", "parameters": [], "promptForInputSource": [] }, REST Endpoints - Dataﬂow Detail

Slide 48

Slide 48 text

{ "stepId": "AddColumns_0", "description": "Add Columns", "type": "AddColumns", "stepDisplayName": "", "stepDescription": "", "columns": [ { "name": "TRAIN OR TEST", "expression": "case when RAND() < 0.8 then 'Train' else 'Test' end", "datatype": "varchar(4000)", "aggrule": "none", "srcexpression": "case when RAND() < 0.8 then 'Train' else 'Test' end", "transformationtype": "Calculation" }, { "name": "Length of Description", "expression": "CHAR_LENGTH(\"description\")", "datatype": "integer", "aggrule": "sum", "srcexpression": "CHAR_LENGTH(XSA('francesco.tisiot@rittmanmead.com'.'__CLIENT_FLOW_DATASET__'). \"Columns\".\"description\")", "transformationtype": "Calculation" } ] } REST Endpoints - Dataﬂow Detail

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

https://www.oracle.com/solutions/business-analytics/data-visualization/extensions.html

Slide 51

Slide 51 text

Data Governance Extension

Slide 52

Slide 52 text

Displays only objects visible to you!

Slide 53

Slide 53 text

BAR File

Slide 54

Slide 54 text

BAR File

Slide 55

Slide 55 text

DEMO

Slide 56

Slide 56 text

Oracle Data Catalog http://ritt.md/oow19-datacatalog Inventories of Data Assets

Slide 57

Slide 57 text

Oracle Data Catalog Inventories of Data Assets http://ritt.md/oow19-datacatalog

Slide 58

Slide 58 text

RM Metadata Explorer

Slide 59

Slide 59 text

RM Metadata Explorer Use Cases Data Lineage GDPR DataFlow -> ODI Mapping Metadata Search Tool Data Capital

Slide 60

Slide 60 text

Self Service Analytics

Slide 61

Slide 61 text

Centralized Reporting

Slide 62

Slide 62 text

Measure

Slide 63

Slide 63 text

Tech Days 2020 Milan 29th Jan Rome 31st Jan

Slide 64

Slide 64 text

Francesco Tisiot BI Tech Lead Become an Equilibrista Find the Right Balance in The Analytics Tech Ecosystem