Become an Equilibrista: Find the Right Balance in The Analytics Tech Ecosystem

Transcript

1. Francesco Tisiot - @FTisiot Analytics Tech Lead Become an Equilibrista

   Find the Right Balance in The Analytics Tech Ecosystem

2. Verona, Italy http://ritt.md/ftisiot Over10 Years in Analytics ft@rittmanmead.com @FTisiot Oracle

   ACE Director ITOUG Board President Francesco Tisiot Analytics Tech Lead

3. Data Engineering Analytics Data Science www.rittmanmead.com info@rittmanmead.com @rittmanmead

4. A bit of History

5. Excel Hell

6. https://www.rittmanmead.com/blog/2018/11/where-are-my-users-coming-from-analysing-obiee-connections-methods/

7. Unique Source of Truth

8. None
9. None
10. None

11. Calculated from Scratch Calculated from Scratch AGREEM ENT

12. Sourced from Certified Datasource Calculated from Scratch

13. Approach to Data Certification

14. Top-Down Approach Data Model Definition Implementation Analysis Centralized BI OBIEE

    - RPD LOOKER

15. Bottom-Up Approach Datasource Creation Analysis Datasource Publication Self Service Oracle

    Data Visualization Tableau

16. Top-Down vs Bottom-Up KPI Duplication Security Time To KPI Reusability

    Performance

17. Reusability / Security DataSource A DataSource B Shared Dimension(s) Centralised

    BI

18. Reusability / Security DataSource A DataSource B Dim X Self

    Service BI Dim X

19. Self-Service BI

20. Self Service Centralised

21. The Right Tool: OAC

22. Sourcing from Central System Select "Dim Account"."Account Code", "Fact Churn"."Churn

    Prediction" from "Churn" No Formula Recalculation No Join, Filtering, Aggregation Knowledge Security Inheritance No Code Duplication
23. None

24. Hybrid

25. Self Service Centralised

26. Awareness

27. Documentation

28. RPD Metadata Dictionary

29. Suggestions

30. Complexity Key Factors Security Data Volumes Interest

31. Key Factors Centralised Self Service Complexity Low High Interest Narrow/Unique

    Wide/Repeated Security Easy Complex Data Volumes Small Big

32. Self Service Centralised ?

33. Control

34. Keeping Track Usage Tracking!

35. Usage Tracking SELECT DISTINCT USER_NAME, NVL(QUERY_SRC_CD, 'RPD ODBC') SOURCE, TRUNC(START_TS)

    TS FROM S_NQ_ACCT WHERE AND ( QUERY_SRC_CD IS NULL OR UPPER(QUERY_SRC_CD) IN ('SOAP', 'RAWSQL') ) AND QUERY_TEXT NOT LIKE '{CALL%' ORDER BY 3 DESC;

36. Self Service?

37. Calculation

38. <saw:report ...> <saw:criteria xsi:type=“…” subjectArea=“&quot;Jira Reporting&quot;"> <saw:columns> <saw:column …> <saw:columnFormula>

    <sawx:expr xsi:type=“…”>”Dim Issue Status"."Issue Status”</…> <saw:column …> <saw:columnFormula> <sawx:expr xsi:type=“…”>”Fact Issue KPI"."# of Issues"</…> Metadata

39. https://www.youtube.com/watch?v=mQY2vslle_8 REST Endpoints

40. https://www.youtube.com/watch?v=mQY2vslle_8 REST Endpoints curl “https://<OAC>/datasetsvc/ public/api/v4/datasets” \ -u ‘<USER>:<PWD>'

41. { "provider": "managed", "requiresImpersonation": false, "semicolonDelimited": false, "customDelimiter": false, "name":

    "train", "namespace": "francesco.tisiot@rittmanmead.com", "business-model": "\"\"", "description": "Uploaded from train.csv.", "display-name": "train", "locale": "sys", "file-length": 460676, "type": "text", "data-last-modified": "2019-04-08T08:19:34.481+0000", "private-connection": false, "mime-type": "application/csv", "owner-guid": "francesco.tisiot@rittmanmead.com", "metadata-last-modified": "2019-04-08T08:22:20.928+0000", "bdp-profile": "{\"profile\":\"784c4626-ebd6-4a01-b5dc-1a3ccd0fee1a\"}", "is-temporary": false } REST Endpoints - CSV

42. { …, "name": "OPEN_INCIDENTS", "namespace": "francesco.tisiot@rittmanmead.com", "business-model": "\"\"", "indexable": "BOTH",

    "description": "External Data Set", "display-name": "OPEN_INCIDENTS", …, "data-last-modified": "2019-03-19T16:07:22.732+0000", "connection-namespace": "francesco.tisiot@rittmanmead.com", "private-connection": false, "connection-name": "LGI", "provider-type": "db", "provider-name": "oracle", "owner-guid": "francesco.tisiot@rittmanmead.com", … } REST Endpoints - DB

43. REST Endpoints - DB Details api/v4/datasets/{dataset-uri} {dataset-uri} = ‘<USERNAME>’.’<DATASET_NAME>’

44. REST Endpoints - DB Details

45. { "id": "LGI", "leaf": false, "otype": "Schema", "name": "LGI", "children":

    [ { "id": "LGI.OPEN_INCIDENTS", "leaf": false, "otype": "dbtable", "name": "OPEN_INCIDENTS", "allColumns": [ { "datatype": "number", "name": "OPEN_INCIDENTS", "is-supported": "TRUE", "otype": "Column", "leaf": true, "id": "LGI.OPEN_INCIDENTS.OPEN_INCIDENTS" }, { "datatype": "varchar(30)", "name": "ISO_YEAR_WEEK", "is-supported": "TRUE", "otype": "Column", "leaf": true, "id": "LGI.OPEN_INCIDENTS.ISO_YEAR_WEEK" }, REST Endpoints - DB Details

46. REST Endpoints - Dataflows api/v4/dataflows/{dataflow-uri} {dataflow-uri} = ‘<USERNAME>’.’<DATAFLOW_NAME>’

47. { "datagen-name": "Clean Wine Dataflow", "namespace": "francesco.tisiot@rittmanmead.com", "definition": { "steps":

    [ { "stepId": "InputDataset_0", "description": "winemag-data_first150k", "type": "InputDataset", "stepDisplayName": "winemag-data_first150k", "stepDescription": "", "datasetRef": "ds_0", "qualifiedTable": "XSA(ds_0).\"Columns\"", "datasetDescription": "Uploaded from winemag-data_first150k.xlsx.", "datasetType": "excel", "datasetDisplayName": "winemag-data_first150k", "inputType": "all", "parameters": [], "promptForInputSource": [] }, REST Endpoints - Dataflow Detail

48. { "stepId": "AddColumns_0", "description": "Add Columns", "type": "AddColumns", "stepDisplayName": "",

    "stepDescription": "", "columns": [ { "name": "TRAIN OR TEST", "expression": "case when RAND() < 0.8 then 'Train' else 'Test' end", "datatype": "varchar(4000)", "aggrule": "none", "srcexpression": "case when RAND() < 0.8 then 'Train' else 'Test' end", "transformationtype": "Calculation" }, { "name": "Length of Description", "expression": "CHAR_LENGTH(\"description\")", "datatype": "integer", "aggrule": "sum", "srcexpression": "CHAR_LENGTH(XSA('francesco.tisiot@rittmanmead.com'.'__CLIENT_FLOW_DATASET__'). \"Columns\".\"description\")", "transformationtype": "Calculation" } ] } REST Endpoints - Dataflow Detail
49. None

50. https://www.oracle.com/solutions/business-analytics/data-visualization/extensions.html

51. Data Governance Extension

52. Displays only objects visible to you!

53. BAR File

54. BAR File

55. DEMO

56. Oracle Data Catalog http://ritt.md/oow19-datacatalog Inventories of Data Assets

57. Oracle Data Catalog Inventories of Data Assets http://ritt.md/oow19-datacatalog

58. RM Metadata Explorer

59. RM Metadata Explorer Use Cases Data Lineage GDPR DataFlow ->

    ODI Mapping Metadata Search Tool Data Capital

60. Self Service Analytics

61. Centralized Reporting

62. Measure

63. Tech Days 2020 Milan 29th Jan Rome 31st Jan

64. Francesco Tisiot BI Tech Lead Become an Equilibrista Find the

    Right Balance in The Analytics Tech Ecosystem