Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Become an Equilibrista: Find the Right Balance in The Analytics Tech Ecosystem
FTisiot
December 03, 2019
Technology
0
140
Become an Equilibrista: Find the Right Balance in The Analytics Tech Ecosystem
FTisiot
December 03, 2019
Tweet
Share
More Decks by FTisiot
See All by FTisiot
Apache Kafka JDBC Source Connector: What could go wrong?
ftisiot
0
26
Solve the Knapsack problem with recursive queries and PostgreSQL
ftisiot
0
42
Build an Open Source Data Pipeline
ftisiot
0
170
Get to know Apache Kafka with Jupyter Notebooks
ftisiot
0
410
Fix Your Strings in PostgreSQL
ftisiot
0
200
Practical Tips & Tricks for Apache Kafka Messages Integration
ftisiot
0
140
Track Database Changes with Apache Kafka
ftisiot
0
200
Apache Kafka and Flink: Stateful Streaming Data Pipelines made easy with SQL
ftisiot
0
720
Kickstart your Kafka with Faker Data
ftisiot
0
200
Other Decks in Technology
See All in Technology
マイクロサービス宣言から8年 振り返りとこれから / Eight Years After the Microservices Declaration A Look Back and A Look Ahead
eisuke
2
160
2年で10→70人へ! スタートアップの 情報セキュリティ課題と施策
miekobayashi
1
630
メドレー エンジニア採用資料/ Medley Engineer Guide
medley
3
5.1k
「一通りできるようになった」その先の話
hitomi___kt
0
120
SPA・SSGでSSRのようなOGP対応!
simo123
2
150
Cloudflare Workersで動くOG画像生成器
aiji42
1
490
WebLogic Server for OCI 概要
oracle4engineer
PRO
3
880
日本ディープラーニング協会主催 NeurIPS 2022 技術報告会講演資料
tdailab
0
1.1k
オンプレk8sとEKSの並行運用の実際
ch1aki
0
280
スクラム導入して変わったチーム、組織のありかた
yumechi
0
190
02_プロトタイピングの進め方
kouzoukaikaku
0
510
Periodic Multi-Agent Path Planning
hziwara
0
110
Featured
See All Featured
How to Ace a Technical Interview
jacobian
270
21k
Mobile First: as difficult as doing things right
swwweet
213
7.8k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
13
5.4k
Building Better People: How to give real-time feedback that sticks.
wjessup
346
17k
For a Future-Friendly Web
brad_frost
166
7.8k
Automating Front-end Workflow
addyosmani
1351
200k
How GitHub (no longer) Works
holman
298
140k
Pencils Down: Stop Designing & Start Developing
hursman
114
10k
Web Components: a chance to create the future
zenorocha
304
40k
Code Review Best Practice
trishagee
50
11k
The Cult of Friendly URLs
andyhume
69
5.1k
Web development in the modern age
philhawksworth
197
9.6k
Transcript
Francesco Tisiot - @FTisiot Analytics Tech Lead Become an Equilibrista
Find the Right Balance in The Analytics Tech Ecosystem
Verona, Italy http://ritt.md/ftisiot Over10 Years in Analytics
[email protected]
@FTisiot Oracle
ACE Director ITOUG Board President Francesco Tisiot Analytics Tech Lead
Data Engineering Analytics Data Science www.rittmanmead.com
[email protected]
@rittmanmead
A bit of History
Excel Hell
https://www.rittmanmead.com/blog/2018/11/where-are-my-users-coming-from-analysing-obiee-connections-methods/
Unique Source of Truth
None
None
None
Calculated from Scratch Calculated from Scratch AGREEM ENT
Sourced from Certified Datasource Calculated from Scratch
Approach to Data Certification
Top-Down Approach Data Model Definition Implementation Analysis Centralized BI OBIEE
- RPD LOOKER
Bottom-Up Approach Datasource Creation Analysis Datasource Publication Self Service Oracle
Data Visualization Tableau
Top-Down vs Bottom-Up KPI Duplication Security Time To KPI Reusability
Performance
Reusability / Security DataSource A DataSource B Shared Dimension(s) Centralised
BI
Reusability / Security DataSource A DataSource B Dim X Self
Service BI Dim X
Self-Service BI
Self Service Centralised
The Right Tool: OAC
Sourcing from Central System Select "Dim Account"."Account Code", "Fact Churn"."Churn
Prediction" from "Churn" No Formula Recalculation No Join, Filtering, Aggregation Knowledge Security Inheritance No Code Duplication
None
Hybrid
Self Service Centralised
Awareness
Documentation
RPD Metadata Dictionary
Suggestions
Complexity Key Factors Security Data Volumes Interest
Key Factors Centralised Self Service Complexity Low High Interest Narrow/Unique
Wide/Repeated Security Easy Complex Data Volumes Small Big
Self Service Centralised ?
Control
Keeping Track Usage Tracking!
Usage Tracking SELECT DISTINCT USER_NAME, NVL(QUERY_SRC_CD, 'RPD ODBC') SOURCE, TRUNC(START_TS)
TS FROM S_NQ_ACCT WHERE AND ( QUERY_SRC_CD IS NULL OR UPPER(QUERY_SRC_CD) IN ('SOAP', 'RAWSQL') ) AND QUERY_TEXT NOT LIKE '{CALL%' ORDER BY 3 DESC;
Self Service?
Calculation
<saw:report ...> <saw:criteria xsi:type=“…” subjectArea=“"Jira Reporting""> <saw:columns> <saw:column …> <saw:columnFormula>
<sawx:expr xsi:type=“…”>”Dim Issue Status"."Issue Status”</…> <saw:column …> <saw:columnFormula> <sawx:expr xsi:type=“…”>”Fact Issue KPI"."# of Issues"</…> Metadata
https://www.youtube.com/watch?v=mQY2vslle_8 REST Endpoints
https://www.youtube.com/watch?v=mQY2vslle_8 REST Endpoints curl “https://<OAC>/datasetsvc/ public/api/v4/datasets” \ -u ‘<USER>:<PWD>'
{ "provider": "managed", "requiresImpersonation": false, "semicolonDelimited": false, "customDelimiter": false, "name":
"train", "namespace": "
[email protected]
", "business-model": "\"\"", "description": "Uploaded from train.csv.", "display-name": "train", "locale": "sys", "file-length": 460676, "type": "text", "data-last-modified": "2019-04-08T08:19:34.481+0000", "private-connection": false, "mime-type": "application/csv", "owner-guid": "
[email protected]
", "metadata-last-modified": "2019-04-08T08:22:20.928+0000", "bdp-profile": "{\"profile\":\"784c4626-ebd6-4a01-b5dc-1a3ccd0fee1a\"}", "is-temporary": false } REST Endpoints - CSV
{ …, "name": "OPEN_INCIDENTS", "namespace": "
[email protected]
", "business-model": "\"\"", "indexable": "BOTH",
"description": "External Data Set", "display-name": "OPEN_INCIDENTS", …, "data-last-modified": "2019-03-19T16:07:22.732+0000", "connection-namespace": "
[email protected]
", "private-connection": false, "connection-name": "LGI", "provider-type": "db", "provider-name": "oracle", "owner-guid": "
[email protected]
", … } REST Endpoints - DB
REST Endpoints - DB Details api/v4/datasets/{dataset-uri} {dataset-uri} = ‘<USERNAME>’.’<DATASET_NAME>’
REST Endpoints - DB Details
{ "id": "LGI", "leaf": false, "otype": "Schema", "name": "LGI", "children":
[ { "id": "LGI.OPEN_INCIDENTS", "leaf": false, "otype": "dbtable", "name": "OPEN_INCIDENTS", "allColumns": [ { "datatype": "number", "name": "OPEN_INCIDENTS", "is-supported": "TRUE", "otype": "Column", "leaf": true, "id": "LGI.OPEN_INCIDENTS.OPEN_INCIDENTS" }, { "datatype": "varchar(30)", "name": "ISO_YEAR_WEEK", "is-supported": "TRUE", "otype": "Column", "leaf": true, "id": "LGI.OPEN_INCIDENTS.ISO_YEAR_WEEK" }, REST Endpoints - DB Details
REST Endpoints - Dataflows api/v4/dataflows/{dataflow-uri} {dataflow-uri} = ‘<USERNAME>’.’<DATAFLOW_NAME>’
{ "datagen-name": "Clean Wine Dataflow", "namespace": "
[email protected]
", "definition": { "steps":
[ { "stepId": "InputDataset_0", "description": "winemag-data_first150k", "type": "InputDataset", "stepDisplayName": "winemag-data_first150k", "stepDescription": "", "datasetRef": "ds_0", "qualifiedTable": "XSA(ds_0).\"Columns\"", "datasetDescription": "Uploaded from winemag-data_first150k.xlsx.", "datasetType": "excel", "datasetDisplayName": "winemag-data_first150k", "inputType": "all", "parameters": [], "promptForInputSource": [] }, REST Endpoints - Dataflow Detail
{ "stepId": "AddColumns_0", "description": "Add Columns", "type": "AddColumns", "stepDisplayName": "",
"stepDescription": "", "columns": [ { "name": "TRAIN OR TEST", "expression": "case when RAND() < 0.8 then 'Train' else 'Test' end", "datatype": "varchar(4000)", "aggrule": "none", "srcexpression": "case when RAND() < 0.8 then 'Train' else 'Test' end", "transformationtype": "Calculation" }, { "name": "Length of Description", "expression": "CHAR_LENGTH(\"description\")", "datatype": "integer", "aggrule": "sum", "srcexpression": "CHAR_LENGTH(XSA('
[email protected]
'.'__CLIENT_FLOW_DATASET__'). \"Columns\".\"description\")", "transformationtype": "Calculation" } ] } REST Endpoints - Dataflow Detail
None
https://www.oracle.com/solutions/business-analytics/data-visualization/extensions.html
Data Governance Extension
Displays only objects visible to you!
BAR File
BAR File
DEMO
Oracle Data Catalog http://ritt.md/oow19-datacatalog Inventories of Data Assets
Oracle Data Catalog Inventories of Data Assets http://ritt.md/oow19-datacatalog
RM Metadata Explorer
RM Metadata Explorer Use Cases Data Lineage GDPR DataFlow ->
ODI Mapping Metadata Search Tool Data Capital
Self Service Analytics
Centralized Reporting
Measure
Tech Days 2020 Milan 29th Jan Rome 31st Jan
Francesco Tisiot BI Tech Lead Become an Equilibrista Find the
Right Balance in The Analytics Tech Ecosystem