DevOps and OBIEE: Do it Before it's too late!

Transcript

1. info@rittmanmead.com www.rittmanmead.com @rittmanmead !1 DevOps and OBIEE: Do it Before

   it's Too Late!

2. info@rittmanmead.com www.rittmanmead.com @rittmanmead !2 Verona, Italy Rittman Mead Blog 10

   Years Experience in BI/Analytics francesco.tisiot@rittmanmead.com @FTisiot Oracle ACE No Cappuccino After 11 Knight Photo by Matt Hoffman on Unsplash Francesco Tisiot BI Tech Lead at Rittman Mead

3. info@rittmanmead.com www.rittmanmead.com @rittmanmead !3 Photo by Pascal van de Vendel

   on Unsplash Mode 1 Analytics IT Driven Long Cycle Times Centralised

4. info@rittmanmead.com www.rittmanmead.com @rittmanmead !4 Photo by Kyle Henderson on Unsplash

   Mode 2 Analytics Fast Business Driven Data Discovery

5. info@rittmanmead.com www.rittmanmead.com @rittmanmead !5 Photo by Jovica Smileski on Unsplash

   Centralised • Unique Source of Truth • KPIs Definition • Development • Test • Release

6. info@rittmanmead.com www.rittmanmead.com @rittmanmead !6 Photo by Chris Sabor on Unsplash

   TTD Development Isolation Manual Deployment Manual Testing

7. info@rittmanmead.com www.rittmanmead.com @rittmanmead !7 Photo by chage tang on Unsplash

   Agile DevOps

8. info@rittmanmead.com www.rittmanmead.com @rittmanmead !8 https://www.01net.it/sviluppare-software-metodo-agile/ Collaboration Frequent Delivery Focus Iteration

9. info@rittmanmead.com www.rittmanmead.com @rittmanmead !9 DevOps https://www.niceideas.ch/roller2/badtrash/entry/devops-explained

10. info@rittmanmead.com www.rittmanmead.com @rittmanmead !10 https://www.niceideas.ch/roller2/badtrash/entry/devops-explained Waste Before it’s Too Late?

11. info@rittmanmead.com www.rittmanmead.com @rittmanmead !11 Before it’s Too Late? https://www.niceideas.ch/roller2/badtrash/entry/devops-explained Risk

12. info@rittmanmead.com www.rittmanmead.com @rittmanmead !12 Before it’s Too Late! The Results

    are Wrong! Production is Pointing to UAT! We Locked the DB User! Which RPD can we Promote? Photo by R. Mac Wheeler on Unsplash

13. info@rittmanmead.com www.rittmanmead.com @rittmanmead !13 OBIEE *1996 - ✝2018 Photo by

    Jerry Kiesewetter on Unsplash Before it’s Too Late!

14. info@rittmanmead.com www.rittmanmead.com @rittmanmead !14 Agenda • Default Situation • Code

    Versioning • Development Isolation • Testing • Promoting • Process Automation

15. info@rittmanmead.com www.rittmanmead.com @rittmanmead !15 RPD Source Catalog OBIEE Security Config

    Artefacts

16. info@rittmanmead.com www.rittmanmead.com @rittmanmead !16 The Original OBIEE Sins • No

    Code Versioning • Concurrent Online RPD Editing • Manual Regression Testing • Limited Scripting • Manual Deployment Photo by David Clode on Unsplash

17. info@rittmanmead.com www.rittmanmead.com @rittmanmead !17 Photo by Erwan Hesry on Unsplash

    DevOps is a Methodology

18. info@rittmanmead.com www.rittmanmead.com @rittmanmead !18 Photo by Fidel Fernando on Unsplash

    Start Simple - Code Versioning • RPD • Catalog • Security • Config Files • Artefacts

19. info@rittmanmead.com www.rittmanmead.com @rittmanmead !19 Gitflow Method Master Branch -> Production

    Develop Branch -> Development Features -> New Developments

20. info@rittmanmead.com www.rittmanmead.com @rittmanmead !20 RPD • RPD Binary • MDS

    XML • How to Merge?

21. info@rittmanmead.com www.rittmanmead.com @rittmanmead !21 Catalog • Set of XML Files

    • Stored as .catalog? • What about .atr? • How to deploy?

22. info@rittmanmead.com www.rittmanmead.com @rittmanmead !22 Security • system-jazn-data in 11g •

    bar file in 12c (contains system-jazn-data) • WLST Scripting (infrastructure as code)

23. info@rittmanmead.com www.rittmanmead.com @rittmanmead !23 Config Files • Automatically promoted? •

    Distinct files per environment

24. info@rittmanmead.com www.rittmanmead.com @rittmanmead !24 Photo by raquel raclette on Unsplash

    Development Isolation Local Instances VMs Docker

25. info@rittmanmead.com www.rittmanmead.com @rittmanmead !25 Photo by Matthew Henry on Unsplash

    Original Situation Online RPD/Catalog Editing No History No Change Control No Automated Regression Testing

26. info@rittmanmead.com www.rittmanmead.com @rittmanmead !26 Photo by Aaron Burden on Unsplash

    Start Simple • Offline RPD • Dev Server for Catalog No test of the RPD Before Merging!

27. info@rittmanmead.com www.rittmanmead.com @rittmanmead !27 Photo by James & Carol Lee

    on Unsplash And Evolve… • Fully Separated Dev Envs • Content pushed/pulled from Git

28. info@rittmanmead.com www.rittmanmead.com @rittmanmead !28 Photo by James & Carol Lee

    on Unsplash And Evolve… • Automated Environment Creation • Docker ‣ https:/ /github.com/gianniceresa/docker-images

29. info@rittmanmead.com www.rittmanmead.com @rittmanmead !29 OBIEE + DB OBIEE F1 OBIEE

    F2 DB F1_BIPLATFORM F2_BIPLATFORM Docker Options

30. info@rittmanmead.com www.rittmanmead.com @rittmanmead !30 Photo by Timothy Rhyne on Unsplash

    Testing

31. info@rittmanmead.com www.rittmanmead.com @rittmanmead !31 RPD Consistency Check Validates RPD Logic

    Errors Warnings

32. info@rittmanmead.com www.rittmanmead.com @rittmanmead !32 RPD Command Line Consistency Check validaterpd

    -R <repositoryName> -O/C/X <outputfile> Text CSV XML

33. info@rittmanmead.com www.rittmanmead.com @rittmanmead !33 RPD Understanding Differences comparerpd -C <modifiedRPD>

    -G <originalRPD> -O/F <outputfile> Validating Differences

34. info@rittmanmead.com www.rittmanmead.com @rittmanmead !34 Catalog Consistency Check runcat.cmd -cmd validate

    -items Report -links Report -offline <path_to_catalog> Valid XMLs No Broken Links

35. info@rittmanmead.com www.rittmanmead.com @rittmanmead !35 Photo by Vincent van Zalinge on

    Unsplash Regression Testing Before After

36. info@rittmanmead.com www.rittmanmead.com @rittmanmead !36 Photo by Tanner Larson on Unsplash

    Testing Suite • Static • Comprehensive • Security Sensitive Separate Catalog Folder

37. info@rittmanmead.com www.rittmanmead.com @rittmanmead !37 BVT Baseline Validation Tool • Automated

    Testing • Set of Prebuilt Test ‣ Logic SQL (Discontinued) ‣ Catalog ‣ Reports ‣ Dashboards ‣ UI ‣ BI Publisher ‣ Visual Analyser

38. info@rittmanmead.com www.rittmanmead.com @rittmanmead !38 BVT Baseline Validation Tool • Deployment

    ‣ Deployment Name ‣ Connections (analytics/bip/va) ‣ User • Test ‣ Set of Tests to run ‣ Catalog Folder • Results ‣ Results Folder textconfig.xml

39. info@rittmanmead.com www.rittmanmead.com @rittmanmead !39 BVT Baseline Validation Tool • Thresholds

    ‣ UI Comparison

40. info@rittmanmead.com www.rittmanmead.com @rittmanmead !40 BVT Baseline Validation Tool • Baseline

    • Changes • Compare Results bin\obibvt.bat -config testconfig.xml -deployment PRE bin\obibvt.bat -config testconfig.xml -deployment AFTER bin\obibvt.bat -compareresults Results\PRE Results\AFTER -config testconfig.xml

41. info@rittmanmead.com www.rittmanmead.com @rittmanmead !41 Photo by Jordan Whitt on Unsplash

    Code Promotion

42. info@rittmanmead.com www.rittmanmead.com @rittmanmead !42 Promoting • Push ‣ RPD ‣

    Catalog ‣ Security ‣ Configuration ‣ Custom Files/Images

43. info@rittmanmead.com www.rittmanmead.com @rittmanmead !43 Photo by Rob Schreckhise on Unsplash

    Do you Remember the Password?

44. info@rittmanmead.com www.rittmanmead.com @rittmanmead !44 RPD • UploadRPD • Patch RPD

    - Change Connection Pools datamodel.cmd uploadrpd -I <file> -D -U <user> -SI ssi -S <server> -N <port> admintool /command <filename> datamodel.sh updateConnectionpool -SI sis -U <user> -P <password> -C <filejson> patchrpd ... -C <curr_rpd> -G <origin_rpd> -I <patch_file> -O <dest_rpd>

45. info@rittmanmead.com www.rittmanmead.com @rittmanmead !45 Catalog • File System Copy •

    Webservices /shared /system WebCatalogService.pasteItem2(<.catalog_file>,…)

46. info@rittmanmead.com www.rittmanmead.com @rittmanmead !46 Security • WLST scripts • system-jazn

    wlst.sh <wlst_script.py> Config • File System Copy

47. info@rittmanmead.com www.rittmanmead.com @rittmanmead !47 Custom Files AnalyticsRes - File System

    Copy

48. info@rittmanmead.com www.rittmanmead.com @rittmanmead !48 Photo by Armando Tobar on Unsplash

    Sticking Everything Together

49. info@rittmanmead.com www.rittmanmead.com @rittmanmead !49 Automation Tools Jenkins Gitlab

50. info@rittmanmead.com www.rittmanmead.com @rittmanmead b !50 Environment Creation New Feature/New Ticket

    in Jira New Git Branch Jenkins Flow Docker OBIEE Image RPD+Catalog Deployment Development Baseline Creation

51. info@rittmanmead.com www.rittmanmead.com @rittmanmead !51 Finish Feature Finish Feature (Commit to

    work Branch) RPD & Catalog Validation Jenkins Flow Baseline Validation Tool - Against Baseline Pre-Changes Push to Dev Review/Approve Fix

52. info@rittmanmead.com www.rittmanmead.com @rittmanmead b !52 Merge Into Dev Commit to

    Dev RPD & Catalog Validation Jenkins Flow Baseline Validation Tool Push to Test Review/Approve RPD & Catalog Merge - Manual Intervention when Conflicts Fix

53. info@rittmanmead.com www.rittmanmead.com @rittmanmead !53 Production Release Commit to Master Jenkins

    Flow Automatic Push to UAT/Production

54. info@rittmanmead.com www.rittmanmead.com @rittmanmead !54 Rittman Mead BI Developer Toolkit https://www.rittmanmead.com/blog/2017/02/concurrent-rpd-development-with-git/

    • Accelleration Framework • Set of pre-packaged functions ‣ Git versioning ‣ Testing Suite • Written in Python - Easy to Integrate ‣ OBIEE Webservices, command line, file system • Git/SVN Versioning ‣ Binary RPD ‣ .catalog files

55. info@rittmanmead.com www.rittmanmead.com @rittmanmead !55 Rittman Mead BI Developer Toolkit •

    Creation of Testing Suite - Parameter Modification python modify_reports.py /shared/Test /shared/Test_Suite/2014_2015 "Date Period" Year 2014 2015 • Regression Testing ‣ Comparison of LSQL and Results ‣ Parameter Modification ‣ Push Results to Database ‣ Prebuilt RPD/catalog

56. info@rittmanmead.com www.rittmanmead.com @rittmanmead !56 Developer Toolkits available for ‣ BI

    ‣ ODI ‣ Informatica ‣ Essbase Apply DevOps to the Whole BI Cycle!