Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Become an Equilibrista: Find the Right Balance in The Analytics Tech Ecosystem

A23789f299ed06fe7d9f1c6940440bfa?s=47 FTisiot
December 03, 2019

Become an Equilibrista: Find the Right Balance in The Analytics Tech Ecosystem

A23789f299ed06fe7d9f1c6940440bfa?s=128

FTisiot

December 03, 2019
Tweet

Transcript

  1. Francesco Tisiot - @FTisiot Analytics Tech Lead Become an Equilibrista

    Find the Right Balance in The Analytics Tech Ecosystem
  2. Verona, Italy http://ritt.md/ftisiot Over10 Years in Analytics ft@rittmanmead.com @FTisiot Oracle

    ACE Director ITOUG Board President Francesco Tisiot Analytics Tech Lead
  3. Data Engineering Analytics Data Science www.rittmanmead.com info@rittmanmead.com @rittmanmead

  4. A bit of History

  5. Excel Hell

  6. https://www.rittmanmead.com/blog/2018/11/where-are-my-users-coming-from-analysing-obiee-connections-methods/

  7. Unique Source of Truth

  8. None
  9. None
  10. None
  11. Calculated from Scratch Calculated from Scratch AGREEM ENT

  12. Sourced from Certified Datasource Calculated from Scratch

  13. Approach to Data Certification

  14. Top-Down Approach Data Model Definition Implementation Analysis Centralized BI OBIEE

    - RPD LOOKER
  15. Bottom-Up Approach Datasource Creation Analysis Datasource Publication Self Service Oracle

    Data Visualization Tableau
  16. Top-Down vs Bottom-Up KPI Duplication Security Time To KPI Reusability

    Performance
  17. Reusability / Security DataSource A DataSource B Shared Dimension(s) Centralised

    BI
  18. Reusability / Security DataSource A DataSource B Dim X Self

    Service BI Dim X
  19. Self-Service BI

  20. Self Service Centralised

  21. The Right Tool: OAC

  22. Sourcing from Central System Select "Dim Account"."Account Code", "Fact Churn"."Churn

    Prediction" from "Churn" No Formula Recalculation No Join, Filtering, Aggregation Knowledge Security Inheritance No Code Duplication
  23. None
  24. Hybrid

  25. Self Service Centralised

  26. Awareness

  27. Documentation

  28. RPD Metadata Dictionary

  29. Suggestions

  30. Complexity Key Factors Security Data Volumes Interest

  31. Key Factors Centralised Self Service Complexity Low High Interest Narrow/Unique

    Wide/Repeated Security Easy Complex Data Volumes Small Big
  32. Self Service Centralised ?

  33. Control

  34. Keeping Track Usage Tracking!

  35. Usage Tracking SELECT DISTINCT USER_NAME, NVL(QUERY_SRC_CD, 'RPD ODBC') SOURCE, TRUNC(START_TS)

    TS FROM S_NQ_ACCT WHERE AND ( QUERY_SRC_CD IS NULL OR UPPER(QUERY_SRC_CD) IN ('SOAP', 'RAWSQL') ) AND QUERY_TEXT NOT LIKE '{CALL%' ORDER BY 3 DESC;
  36. Self Service?

  37. Calculation

  38. <saw:report ...> <saw:criteria xsi:type=“…” subjectArea=“&quot;Jira Reporting&quot;"> <saw:columns> <saw:column …> <saw:columnFormula>

    <sawx:expr xsi:type=“…”>”Dim Issue Status"."Issue Status”</…> <saw:column …> <saw:columnFormula> <sawx:expr xsi:type=“…”>”Fact Issue KPI"."# of Issues"</…> Metadata
  39. https://www.youtube.com/watch?v=mQY2vslle_8 REST Endpoints

  40. https://www.youtube.com/watch?v=mQY2vslle_8 REST Endpoints curl “https://<OAC>/datasetsvc/ public/api/v4/datasets” \ -u ‘<USER>:<PWD>'

  41. { "provider": "managed", "requiresImpersonation": false, "semicolonDelimited": false, "customDelimiter": false, "name":

    "train", "namespace": "francesco.tisiot@rittmanmead.com", "business-model": "\"\"", "description": "Uploaded from train.csv.", "display-name": "train", "locale": "sys", "file-length": 460676, "type": "text", "data-last-modified": "2019-04-08T08:19:34.481+0000", "private-connection": false, "mime-type": "application/csv", "owner-guid": "francesco.tisiot@rittmanmead.com", "metadata-last-modified": "2019-04-08T08:22:20.928+0000", "bdp-profile": "{\"profile\":\"784c4626-ebd6-4a01-b5dc-1a3ccd0fee1a\"}", "is-temporary": false } REST Endpoints - CSV
  42. { …, "name": "OPEN_INCIDENTS", "namespace": "francesco.tisiot@rittmanmead.com", "business-model": "\"\"", "indexable": "BOTH",

    "description": "External Data Set", "display-name": "OPEN_INCIDENTS", …, "data-last-modified": "2019-03-19T16:07:22.732+0000", "connection-namespace": "francesco.tisiot@rittmanmead.com", "private-connection": false, "connection-name": "LGI", "provider-type": "db", "provider-name": "oracle", "owner-guid": "francesco.tisiot@rittmanmead.com", … } REST Endpoints - DB
  43. REST Endpoints - DB Details api/v4/datasets/{dataset-uri} {dataset-uri} = ‘<USERNAME>’.’<DATASET_NAME>’

  44. REST Endpoints - DB Details

  45. { "id": "LGI", "leaf": false, "otype": "Schema", "name": "LGI", "children":

    [ { "id": "LGI.OPEN_INCIDENTS", "leaf": false, "otype": "dbtable", "name": "OPEN_INCIDENTS", "allColumns": [ { "datatype": "number", "name": "OPEN_INCIDENTS", "is-supported": "TRUE", "otype": "Column", "leaf": true, "id": "LGI.OPEN_INCIDENTS.OPEN_INCIDENTS" }, { "datatype": "varchar(30)", "name": "ISO_YEAR_WEEK", "is-supported": "TRUE", "otype": "Column", "leaf": true, "id": "LGI.OPEN_INCIDENTS.ISO_YEAR_WEEK" }, REST Endpoints - DB Details
  46. REST Endpoints - Dataflows api/v4/dataflows/{dataflow-uri} {dataflow-uri} = ‘<USERNAME>’.’<DATAFLOW_NAME>’

  47. { "datagen-name": "Clean Wine Dataflow", "namespace": "francesco.tisiot@rittmanmead.com", "definition": { "steps":

    [ { "stepId": "InputDataset_0", "description": "winemag-data_first150k", "type": "InputDataset", "stepDisplayName": "winemag-data_first150k", "stepDescription": "", "datasetRef": "ds_0", "qualifiedTable": "XSA(ds_0).\"Columns\"", "datasetDescription": "Uploaded from winemag-data_first150k.xlsx.", "datasetType": "excel", "datasetDisplayName": "winemag-data_first150k", "inputType": "all", "parameters": [], "promptForInputSource": [] }, REST Endpoints - Dataflow Detail
  48. { "stepId": "AddColumns_0", "description": "Add Columns", "type": "AddColumns", "stepDisplayName": "",

    "stepDescription": "", "columns": [ { "name": "TRAIN OR TEST", "expression": "case when RAND() < 0.8 then 'Train' else 'Test' end", "datatype": "varchar(4000)", "aggrule": "none", "srcexpression": "case when RAND() < 0.8 then 'Train' else 'Test' end", "transformationtype": "Calculation" }, { "name": "Length of Description", "expression": "CHAR_LENGTH(\"description\")", "datatype": "integer", "aggrule": "sum", "srcexpression": "CHAR_LENGTH(XSA('francesco.tisiot@rittmanmead.com'.'__CLIENT_FLOW_DATASET__'). \"Columns\".\"description\")", "transformationtype": "Calculation" } ] } REST Endpoints - Dataflow Detail
  49. None
  50. https://www.oracle.com/solutions/business-analytics/data-visualization/extensions.html

  51. Data Governance Extension

  52. Displays only objects visible to you!

  53. BAR File

  54. BAR File

  55. DEMO

  56. Oracle Data Catalog http://ritt.md/oow19-datacatalog Inventories of Data Assets

  57. Oracle Data Catalog Inventories of Data Assets http://ritt.md/oow19-datacatalog

  58. RM Metadata Explorer

  59. RM Metadata Explorer Use Cases Data Lineage GDPR DataFlow ->

    ODI Mapping Metadata Search Tool Data Capital
  60. Self Service Analytics

  61. Centralized Reporting

  62. Measure

  63. Tech Days 2020 Milan 29th Jan Rome 31st Jan

  64. Francesco Tisiot BI Tech Lead Become an Equilibrista Find the

    Right Balance in The Analytics Tech Ecosystem