DevOps and OBIEE: Do it Before it's too late!

Transcript

1. info@rittmanmead.com www.rittmanmead.com @rittmanmead !1 DevOps and OBIEE: Do it Before it's Too Late!
2. info@rittmanmead.com www.rittmanmead.com @rittmanmead !2 Verona, Italy Rittman Mead Blog 10 Years Experience in BI/Analytics francesco.tisiot@rittmanmead.com @FTisiot Oracle ACE No Cappuccino After 11 Knight Photo by Matt Hoffman on Unsplash Francesco Tisiot BI Tech Lead at Rittman Mead
3. info@rittmanmead.com www.rittmanmead.com @rittmanmead !3 Photo by Pascal van de Vendel on Unsplash Mode 1 Analytics IT Driven Long Cycle Times Centralised
4. info@rittmanmead.com www.rittmanmead.com @rittmanmead !4 Photo by Kyle Henderson on Unsplash Mode 2 Analytics Fast Business Driven Data Discovery
5. info@rittmanmead.com www.rittmanmead.com @rittmanmead !5 Photo by Jovica Smileski on Unsplash Centralised • Unique Source of Truth • KPIs Definition • Development • Test • Release
6. info@rittmanmead.com www.rittmanmead.com @rittmanmead !6 Photo by Chris Sabor on Unsplash TTD Development Isolation Manual Deployment Manual Testing
7. info@rittmanmead.com www.rittmanmead.com @rittmanmead !7 Photo by chage tang on Unsplash Agile DevOps
8. info@rittmanmead.com www.rittmanmead.com @rittmanmead !8 https://www.01net.it/sviluppare-software-metodo-agile/ Collaboration Frequent Delivery Focus Iteration
9. info@rittmanmead.com www.rittmanmead.com @rittmanmead !9 DevOps https://www.niceideas.ch/roller2/badtrash/entry/devops-explained
10. info@rittmanmead.com www.rittmanmead.com @rittmanmead !10 https://www.niceideas.ch/roller2/badtrash/entry/devops-explained Waste Before it’s Too Late?
11. info@rittmanmead.com www.rittmanmead.com @rittmanmead !11 Before it’s Too Late? https://www.niceideas.ch/roller2/badtrash/entry/devops-explained Risk
12. info@rittmanmead.com www.rittmanmead.com @rittmanmead !12 Before it’s Too Late! The Results are Wrong! Production is Pointing to UAT! We Locked the DB User! Which RPD can we Promote? Photo by R. Mac Wheeler on Unsplash
13. info@rittmanmead.com www.rittmanmead.com @rittmanmead !13 OBIEE *1996 - ✝2018 Photo by Jerry Kiesewetter on Unsplash Before it’s Too Late!
14. info@rittmanmead.com www.rittmanmead.com @rittmanmead !14 Agenda • Default Situation • Code Versioning • Development Isolation • Testing • Promoting • Process Automation
15. info@rittmanmead.com www.rittmanmead.com @rittmanmead !15 RPD Source Catalog OBIEE Security Config Artefacts
16. info@rittmanmead.com www.rittmanmead.com @rittmanmead !16 The Original OBIEE Sins • No Code Versioning • Concurrent Online RPD Editing • Manual Regression Testing • Limited Scripting • Manual Deployment Photo by David Clode on Unsplash
17. info@rittmanmead.com www.rittmanmead.com @rittmanmead !17 Photo by Erwan Hesry on Unsplash DevOps is a Methodology
18. info@rittmanmead.com www.rittmanmead.com @rittmanmead !18 Photo by Fidel Fernando on Unsplash Start Simple - Code Versioning • RPD • Catalog • Security • Config Files • Artefacts
19. info@rittmanmead.com www.rittmanmead.com @rittmanmead !19 Gitflow Method Master Branch -> Production Develop Branch -> Development Features -> New Developments
20. info@rittmanmead.com www.rittmanmead.com @rittmanmead !20 RPD • RPD Binary • MDS XML • How to Merge?
21. info@rittmanmead.com www.rittmanmead.com @rittmanmead !21 Catalog • Set of XML Files • Stored as .catalog? • What about .atr? • How to deploy?
22. info@rittmanmead.com www.rittmanmead.com @rittmanmead !22 Security • system-jazn-data in 11g • bar file in 12c (contains system-jazn-data) • WLST Scripting (infrastructure as code)
23. info@rittmanmead.com www.rittmanmead.com @rittmanmead !23 Config Files • Automatically promoted? • Distinct files per environment
24. info@rittmanmead.com www.rittmanmead.com @rittmanmead !24 Photo by raquel raclette on Unsplash Development Isolation Local Instances VMs Docker
25. info@rittmanmead.com www.rittmanmead.com @rittmanmead !25 Photo by Matthew Henry on Unsplash Original Situation Online RPD/Catalog Editing No History No Change Control No Automated Regression Testing
26. info@rittmanmead.com www.rittmanmead.com @rittmanmead !26 Photo by Aaron Burden on Unsplash Start Simple • Offline RPD • Dev Server for Catalog No test of the RPD Before Merging!
27. info@rittmanmead.com www.rittmanmead.com @rittmanmead !27 Photo by James & Carol Lee on Unsplash And Evolve… • Fully Separated Dev Envs • Content pushed/pulled from Git
28. info@rittmanmead.com www.rittmanmead.com @rittmanmead !28 Photo by James & Carol Lee on Unsplash And Evolve… • Automated Environment Creation • Docker ‣ https:/ /github.com/gianniceresa/docker-images
29. info@rittmanmead.com www.rittmanmead.com @rittmanmead !29 OBIEE + DB OBIEE F1 OBIEE F2 DB F1_BIPLATFORM F2_BIPLATFORM Docker Options
30. info@rittmanmead.com www.rittmanmead.com @rittmanmead !30 Photo by Timothy Rhyne on Unsplash Testing
31. info@rittmanmead.com www.rittmanmead.com @rittmanmead !31 RPD Consistency Check Validates RPD Logic Errors Warnings
32. info@rittmanmead.com www.rittmanmead.com @rittmanmead !32 RPD Command Line Consistency Check validaterpd -R <repositoryName> -O/C/X <outputfile> Text CSV XML
33. info@rittmanmead.com www.rittmanmead.com @rittmanmead !33 RPD Understanding Differences comparerpd -C <modifiedRPD> -G <originalRPD> -O/F <outputfile> Validating Differences
34. info@rittmanmead.com www.rittmanmead.com @rittmanmead !34 Catalog Consistency Check runcat.cmd -cmd validate -items Report -links Report -offline <path_to_catalog> Valid XMLs No Broken Links
35. info@rittmanmead.com www.rittmanmead.com @rittmanmead !35 Photo by Vincent van Zalinge on Unsplash Regression Testing Before After
36. info@rittmanmead.com www.rittmanmead.com @rittmanmead !36 Photo by Tanner Larson on Unsplash Testing Suite • Static • Comprehensive • Security Sensitive Separate Catalog Folder
37. info@rittmanmead.com www.rittmanmead.com @rittmanmead !37 BVT Baseline Validation Tool • Automated Testing • Set of Prebuilt Test ‣ Logic SQL (Discontinued) ‣ Catalog ‣ Reports ‣ Dashboards ‣ UI ‣ BI Publisher ‣ Visual Analyser
38. info@rittmanmead.com www.rittmanmead.com @rittmanmead !38 BVT Baseline Validation Tool • Deployment ‣ Deployment Name ‣ Connections (analytics/bip/va) ‣ User • Test ‣ Set of Tests to run ‣ Catalog Folder • Results ‣ Results Folder textconfig.xml
39. info@rittmanmead.com www.rittmanmead.com @rittmanmead !39 BVT Baseline Validation Tool • Thresholds ‣ UI Comparison
40. info@rittmanmead.com www.rittmanmead.com @rittmanmead !40 BVT Baseline Validation Tool • Baseline • Changes • Compare Results bin\obibvt.bat -config testconfig.xml -deployment PRE bin\obibvt.bat -config testconfig.xml -deployment AFTER bin\obibvt.bat -compareresults Results\PRE Results\AFTER -config testconfig.xml
41. info@rittmanmead.com www.rittmanmead.com @rittmanmead !41 Photo by Jordan Whitt on Unsplash Code Promotion
42. info@rittmanmead.com www.rittmanmead.com @rittmanmead !42 Promoting • Push ‣ RPD ‣ Catalog ‣ Security ‣ Configuration ‣ Custom Files/Images
43. info@rittmanmead.com www.rittmanmead.com @rittmanmead !43 Photo by Rob Schreckhise on Unsplash Do you Remember the Password?
44. info@rittmanmead.com www.rittmanmead.com @rittmanmead !44 RPD • UploadRPD • Patch RPD - Change Connection Pools datamodel.cmd uploadrpd -I <file> -D -U <user> -SI ssi -S <server> -N <port> admintool /command <filename> datamodel.sh updateConnectionpool -SI sis -U <user> -P <password> -C <filejson> patchrpd ... -C <curr_rpd> -G <origin_rpd> -I <patch_file> -O <dest_rpd>
45. info@rittmanmead.com www.rittmanmead.com @rittmanmead !45 Catalog • File System Copy • Webservices /shared /system WebCatalogService.pasteItem2(<.catalog_file>,…)
46. info@rittmanmead.com www.rittmanmead.com @rittmanmead !46 Security • WLST scripts • system-jazn wlst.sh <wlst_script.py> Config • File System Copy
47. info@rittmanmead.com www.rittmanmead.com @rittmanmead !47 Custom Files AnalyticsRes - File System Copy
48. info@rittmanmead.com www.rittmanmead.com @rittmanmead !48 Photo by Armando Tobar on Unsplash Sticking Everything Together
49. info@rittmanmead.com www.rittmanmead.com @rittmanmead !49 Automation Tools Jenkins Gitlab
50. info@rittmanmead.com www.rittmanmead.com @rittmanmead b !50 Environment Creation New Feature/New Ticket in Jira New Git Branch Jenkins Flow Docker OBIEE Image RPD+Catalog Deployment Development Baseline Creation
51. info@rittmanmead.com www.rittmanmead.com @rittmanmead !51 Finish Feature Finish Feature (Commit to work Branch) RPD & Catalog Validation Jenkins Flow Baseline Validation Tool - Against Baseline Pre-Changes Push to Dev Review/Approve Fix
52. info@rittmanmead.com www.rittmanmead.com @rittmanmead b !52 Merge Into Dev Commit to Dev RPD & Catalog Validation Jenkins Flow Baseline Validation Tool Push to Test Review/Approve RPD & Catalog Merge - Manual Intervention when Conflicts Fix
53. info@rittmanmead.com www.rittmanmead.com @rittmanmead !53 Production Release Commit to Master Jenkins Flow Automatic Push to UAT/Production
54. info@rittmanmead.com www.rittmanmead.com @rittmanmead !54 Rittman Mead BI Developer Toolkit https://www.rittmanmead.com/blog/2017/02/concurrent-rpd-development-with-git/ • Accelleration Framework • Set of pre-packaged functions ‣ Git versioning ‣ Testing Suite • Written in Python - Easy to Integrate ‣ OBIEE Webservices, command line, file system • Git/SVN Versioning ‣ Binary RPD ‣ .catalog files
55. info@rittmanmead.com www.rittmanmead.com @rittmanmead !55 Rittman Mead BI Developer Toolkit • Creation of Testing Suite - Parameter Modification python modify_reports.py /shared/Test /shared/Test_Suite/2014_2015 "Date Period" Year 2014 2015 • Regression Testing ‣ Comparison of LSQL and Results ‣ Parameter Modification ‣ Push Results to Database ‣ Prebuilt RPD/catalog
56. info@rittmanmead.com www.rittmanmead.com @rittmanmead !56 Developer Toolkits available for ‣ BI ‣ ODI ‣ Informatica ‣ Essbase Apply DevOps to the Whole BI Cycle!