Become an Equilibrista: Find the Right Balance in The Analytics Tech Ecosystem

Transcript

1. Francesco Tisiot - @FTisiot Analytics Tech Lead Become an Equilibrista Find the Right Balance in The Analytics Tech Ecosystem
2. Verona, Italy http://ritt.md/ftisiot Over10 Years in Analytics ft@rittmanmead.com @FTisiot Oracle ACE Director ITOUG Board President Francesco Tisiot Analytics Tech Lead
3. Data Engineering Analytics Data Science www.rittmanmead.com info@rittmanmead.com @rittmanmead
4. A bit of History
5. Excel Hell
6. https://www.rittmanmead.com/blog/2018/11/where-are-my-users-coming-from-analysing-obiee-connections-methods/
7. Unique Source of Truth
11. Calculated from Scratch Calculated from Scratch AGREEM ENT
12. Sourced from Certified Datasource Calculated from Scratch
13. Approach to Data Certification
14. Top-Down Approach Data Model Definition Implementation Analysis Centralized BI OBIEE - RPD LOOKER
15. Bottom-Up Approach Datasource Creation Analysis Datasource Publication Self Service Oracle Data Visualization Tableau
16. Top-Down vs Bottom-Up KPI Duplication Security Time To KPI Reusability Performance
17. Reusability / Security DataSource A DataSource B Shared Dimension(s) Centralised BI
18. Reusability / Security DataSource A DataSource B Dim X Self Service BI Dim X
19. Self-Service BI
20. Self Service Centralised
21. The Right Tool: OAC
22. Sourcing from Central System Select "Dim Account"."Account Code", "Fact Churn"."Churn Prediction" from "Churn" No Formula Recalculation No Join, Filtering, Aggregation Knowledge Security Inheritance No Code Duplication
24. Hybrid
25. Self Service Centralised
26. Awareness
27. Documentation
28. RPD Metadata Dictionary
29. Suggestions
30. Complexity Key Factors Security Data Volumes Interest
31. Key Factors Centralised Self Service Complexity Low High Interest Narrow/Unique Wide/Repeated Security Easy Complex Data Volumes Small Big
32. Self Service Centralised ?
33. Control
34. Keeping Track Usage Tracking!
35. Usage Tracking SELECT DISTINCT USER_NAME, NVL(QUERY_SRC_CD, 'RPD ODBC') SOURCE, TRUNC(START_TS) TS FROM S_NQ_ACCT WHERE AND ( QUERY_SRC_CD IS NULL OR UPPER(QUERY_SRC_CD) IN ('SOAP', 'RAWSQL') ) AND QUERY_TEXT NOT LIKE '{CALL%' ORDER BY 3 DESC;
36. Self Service?
37. Calculation
38. <saw:report ...> <saw:criteria xsi:type=“…” subjectArea=“&quot;Jira Reporting&quot;"> <saw:columns> <saw:column …> <saw:columnFormula> <sawx:expr xsi:type=“…”>”Dim Issue Status"."Issue Status”</…> <saw:column …> <saw:columnFormula> <sawx:expr xsi:type=“…”>”Fact Issue KPI"."# of Issues"</…> Metadata
39. https://www.youtube.com/watch?v=mQY2vslle_8 REST Endpoints
40. https://www.youtube.com/watch?v=mQY2vslle_8 REST Endpoints curl “https://<OAC>/datasetsvc/ public/api/v4/datasets” \ -u ‘<USER>:<PWD>'
41. { "provider": "managed", "requiresImpersonation": false, "semicolonDelimited": false, "customDelimiter": false, "name": "train", "namespace": "francesco.tisiot@rittmanmead.com", "business-model": "\"\"", "description": "Uploaded from train.csv.", "display-name": "train", "locale": "sys", "file-length": 460676, "type": "text", "data-last-modified": "2019-04-08T08:19:34.481+0000", "private-connection": false, "mime-type": "application/csv", "owner-guid": "francesco.tisiot@rittmanmead.com", "metadata-last-modified": "2019-04-08T08:22:20.928+0000", "bdp-profile": "{\"profile\":\"784c4626-ebd6-4a01-b5dc-1a3ccd0fee1a\"}", "is-temporary": false } REST Endpoints - CSV
42. { …, "name": "OPEN_INCIDENTS", "namespace": "francesco.tisiot@rittmanmead.com", "business-model": "\"\"", "indexable": "BOTH", "description": "External Data Set", "display-name": "OPEN_INCIDENTS", …, "data-last-modified": "2019-03-19T16:07:22.732+0000", "connection-namespace": "francesco.tisiot@rittmanmead.com", "private-connection": false, "connection-name": "LGI", "provider-type": "db", "provider-name": "oracle", "owner-guid": "francesco.tisiot@rittmanmead.com", … } REST Endpoints - DB
43. REST Endpoints - DB Details api/v4/datasets/{dataset-uri} {dataset-uri} = ‘<USERNAME>’.’<DATASET_NAME>’
44. REST Endpoints - DB Details
45. { "id": "LGI", "leaf": false, "otype": "Schema", "name": "LGI", "children": [ { "id": "LGI.OPEN_INCIDENTS", "leaf": false, "otype": "dbtable", "name": "OPEN_INCIDENTS", "allColumns": [ { "datatype": "number", "name": "OPEN_INCIDENTS", "is-supported": "TRUE", "otype": "Column", "leaf": true, "id": "LGI.OPEN_INCIDENTS.OPEN_INCIDENTS" }, { "datatype": "varchar(30)", "name": "ISO_YEAR_WEEK", "is-supported": "TRUE", "otype": "Column", "leaf": true, "id": "LGI.OPEN_INCIDENTS.ISO_YEAR_WEEK" }, REST Endpoints - DB Details
46. REST Endpoints - Dataflows api/v4/dataflows/{dataflow-uri} {dataflow-uri} = ‘<USERNAME>’.’<DATAFLOW_NAME>’
47. { "datagen-name": "Clean Wine Dataflow", "namespace": "francesco.tisiot@rittmanmead.com", "definition": { "steps": [ { "stepId": "InputDataset_0", "description": "winemag-data_first150k", "type": "InputDataset", "stepDisplayName": "winemag-data_first150k", "stepDescription": "", "datasetRef": "ds_0", "qualifiedTable": "XSA(ds_0).\"Columns\"", "datasetDescription": "Uploaded from winemag-data_first150k.xlsx.", "datasetType": "excel", "datasetDisplayName": "winemag-data_first150k", "inputType": "all", "parameters": [], "promptForInputSource": [] }, REST Endpoints - Dataflow Detail
48. { "stepId": "AddColumns_0", "description": "Add Columns", "type": "AddColumns", "stepDisplayName": "", "stepDescription": "", "columns": [ { "name": "TRAIN OR TEST", "expression": "case when RAND() < 0.8 then 'Train' else 'Test' end", "datatype": "varchar(4000)", "aggrule": "none", "srcexpression": "case when RAND() < 0.8 then 'Train' else 'Test' end", "transformationtype": "Calculation" }, { "name": "Length of Description", "expression": "CHAR_LENGTH(\"description\")", "datatype": "integer", "aggrule": "sum", "srcexpression": "CHAR_LENGTH(XSA('francesco.tisiot@rittmanmead.com'.'__CLIENT_FLOW_DATASET__'). \"Columns\".\"description\")", "transformationtype": "Calculation" } ] } REST Endpoints - Dataflow Detail
50. https://www.oracle.com/solutions/business-analytics/data-visualization/extensions.html
51. Data Governance Extension
52. Displays only objects visible to you!
53. BAR File
54. BAR File
55. DEMO
56. Oracle Data Catalog http://ritt.md/oow19-datacatalog Inventories of Data Assets
57. Oracle Data Catalog Inventories of Data Assets http://ritt.md/oow19-datacatalog
58. RM Metadata Explorer
59. RM Metadata Explorer Use Cases Data Lineage GDPR DataFlow -> ODI Mapping Metadata Search Tool Data Capital
60. Self Service Analytics
61. Centralized Reporting
62. Measure
63. Tech Days 2020 Milan 29th Jan Rome 31st Jan
64. Francesco Tisiot BI Tech Lead Become an Equilibrista Find the Right Balance in The Analytics Tech Ecosystem