Tweet
Tweet

Slide 1

Slide 1 text

Data as Code: Exploring Data and Database Portability Latin America Oracle User Group Community Tour '23 July 29 - August 16, 2023 @ViscosityNA www.viscosityna.com Sean Scott Oracle ACE Director DoK Community Ambassador Managing Principal Consultant @oraclesean linkedin.com/in/soscott/ sean.scott@viscosityna.com

Slide 2

Slide 2 text

The Intersectionality of Cloud Native and "Groundhog Day" @ViscosityNA www.viscosityna.com Latin America Community Tour '23 - Uruguay August 14, 2023 Sean Scott Oracle ACE Director DoK Community Ambassador Managing Principal Consultant @oraclesean linkedin.com/in/soscott/ sean.scott@viscosityna.com

Slide 3

Slide 3 text

DATABASE RELIABILITY ENGINEERING ⁘ DEVOPS & AUTOMATION HIGH AVAILABILITY ⁘ BUSINESS CONTINUITY ⁘ DISASTER RECOVERY MODERNIZATION ⁘ OBSERVABILITY ⁘ ENGINEERED SYSTEMS AutoUpgrade ⁘ Zero Downtime Migrations ⁘ Patching Real Application Clusters ⁘ Data Guard ⁘ Sharding Docker/Containers ⁘ Terraform ⁘ Ansible Exadata ⁘ Oracle Database Appliance AHF ⁘ TFA ⁘ GIMR ⁘ CHA @ViscosityNA www.viscosityna.com Sean Scott Oracle ACE Director DoK Community Ambassador Managing Principal Consultant @oraclesean linkedin.com/in/soscott/ sean.scott@viscosityna.com

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

An Open Community for Data on Kubernetes https://dok.community

Slide 6

Slide 6 text

@ViscosityNA www.viscosityna.com Oracle on Docker Running Oracle Databases in Linux Containers Download a free sample chapter: https:/ /oraclesean.com 20% Discount Code: OracleDocker https:/ /link.springer.com

Slide 7

Slide 7 text

@ViscosityNA www.viscosityna.com "Databases are Special™"

Slide 8

Slide 8 text

@ViscosityNA www.viscosityna.com "Databases are Special™" DBA DEV

Slide 9

Slide 9 text

@ViscosityNA www.viscosityna.com Databases are Special™

Slide 10

Slide 10 text

@ViscosityNA www.viscosityna.com As creators, buyers, and influencers, developers have become instrumental in deciding what products pass muster. Ultimately, the C-suite may call the shot, but the developer is often guiding its aim. Boston Consulting Group "The Growing Influence of Developers in Enterprise Tech Sales" https://www.bcg.com/publications/2022/developers-influence-in-enterprise-tech-sales

Slide 11

Slide 11 text

@ViscosityNA www.viscosityna.com www.viscosityna.com @ViscosityNA 6:00 AM Phil w akes up Encounters N ed 7:00 AM Tapes segm ent 7:20 AM Blizzard! 9:00 AM

Slide 12

Slide 12 text

@ViscosityNA www.viscosityna.com www.viscosityna.com @ViscosityNA Phil Connors: main 6:00 AM Phil w akes up Encounters N ed 7:00 AM Tapes segm ent 7:20 AM Blizzard! 9:00 AM

Slide 13

Slide 13 text

@ViscosityNA www.viscosityna.com Feature Request Customers are complaining that the Phil microservice is narcissistic and misogynistic. Redesign Phil to be a decent human being.

Slide 14

Slide 14 text

@ViscosityNA www.viscosityna.com

Slide 15

Slide 15 text

@ViscosityNA www.viscosityna.com www.viscosityna.com @ViscosityNA Phil Connors: main └─ day-2 6:00 AM Phil w akes up Encounters N ed 7:00 AM Tapes segm ent 7:20 AM Blizzard! 9:00 AM

Slide 16

Slide 16 text

@ViscosityNA www.viscosityna.com www.viscosityna.com @ViscosityNA Phil Connors: main └─ day-2 └─ bank-robber 6:00 AM Phil w akes up Encounters N ed 7:00 AM Tapes segm ent 7:20 AM Blizzard! 9:00 AM

Slide 17

Slide 17 text

@ViscosityNA www.viscosityna.com www.viscosityna.com @ViscosityNA Phil Connors: main └─ day-2 └─ bank-robber └─ piano-student 6:00 AM Phil w akes up Encounters N ed 7:00 AM Tapes segm ent 7:20 AM Blizzard! 9:00 AM

Slide 18

Slide 18 text

@ViscosityNA www.viscosityna.com www.viscosityna.com @ViscosityNA Phil Connors: main └─ day-2 └─ bank-robber └─ piano-student └─ redemption 6:00 AM Phil w akes up Encounters N ed 7:00 AM Tapes segm ent 7:20 AM Blizzard! 9:00 AM

Slide 19

Slide 19 text

@ViscosityNA www.viscosityna.com www.viscosityna.com @ViscosityNA Phil Connors: main └─ day-2 └─ bank-robber └─ piano-student └─ redemption 6:00 AM Phil w akes up Encounters N ed 7:00 AM Tapes segm ent 7:20 AM Blizzard! 9:00 AM Punxsutawney_TEST Feb 2 FUNNY

Slide 20

Slide 20 text

@ViscosityNA www.viscosityna.com www.viscosityna.com @ViscosityNA 6:00 AM Phil w akes up Encounters N ed 7:00 AM Tapes segm ent 7:20 AM Blizzard! 9:00 AM Punxsutawney_TEST Feb 3 NOT FUNNY Phil Connors: main └─ day-2 └─ bank-robber └─ piano-student └─ redemption

Slide 21

Slide 21 text

@ViscosityNA www.viscosityna.com www.viscosityna.com @ViscosityNA Phil Connors: main └─ day-2 └─ bank-robber └─ piano-student └─ redemption 6:00 AM Alarm doesn't go off! N ed is PO TU S! 7:00 AM M ultiple groundhogs? 7:20 AM M iam i Beach? 9:00 AM Punxsutawney_TEST Polluted Feb 2 BIZARRE

Slide 22

Slide 22 text

@ViscosityNA www.viscosityna.com Problems • Every test further pollutes data. • Restoring databases is operationally difficult, requiring: • Downtime • Specialized skills/DBA intervention • Every database is a production database • Making databases idempotent is technically difficult

Slide 23

Slide 23 text

DBA DEVELOPER

Slide 24

Slide 24 text

@ViscosityNA www.viscosityna.com Common Solutions • Build exceptions/elasticity into CI/CD pipelines • Technically difficult • Requires upkeep • Not absolute • Assume certain errors can be ignored • Technology solution ($$$$$)

Slide 25

Slide 25 text

@ViscosityNA www.viscosityna.com But... Data is Code

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

@ViscosityNA www.viscosityna.com Data is Code Every record in every database is created in one of three ways: • INSERT • UPDATE • DELETE

Slide 28

Slide 28 text

@ViscosityNA www.viscosityna.com Heterogenous replication = synchronous SQL Oracle Tx Log SQL MySQL

Slide 29

Slide 29 text

@ViscosityNA www.viscosityna.com Heterogenous replication = synchronous SQL Oracle Tx Log SQL MySQL Optional Transformation

Slide 30

Slide 30 text

@ViscosityNA www.viscosityna.com Databases are Special™

Slide 31

Slide 31 text

@ViscosityNA www.viscosityna.com Databases are Special™

Slide 32

Slide 32 text

@ViscosityNA www.viscosityna.com Databases are Special™

Slide 33

Slide 33 text

@ViscosityNA www.viscosityna.com "[F]or a long time, DBAs were in the business of crafting silos and snowflakes. Their tools were different, their hardware was different, and their languages were different." Laine Campbell, Charity Majors Database Reliability Engineering O'Reilly

Slide 34

Slide 34 text

@ViscosityNA www.viscosityna.com "The days in which this model can prove itself to be effective and sustainable are numbered." Laine Campbell, Charity Majors Database Reliability Engineering O'Reilly

Slide 35

Slide 35 text

@ViscosityNA www.viscosityna.com "Our systems are no more or less important than any other components serving the needs of the business. [T]he components of database clusters are not sacred." Laine Campbell, Charity Majors Database Reliability Engineering O'Reilly

Slide 36

Slide 36 text

@ViscosityNA www.viscosityna.com Databases are Special™ Software Configuration Data Not a Database Database

Slide 37

Slide 37 text

@ViscosityNA www.viscosityna.com Databases are Special™ Software Configuration Data

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

@ViscosityNA www.viscosityna.com What is Cloud Native?

Slide 40

Slide 40 text

Containers are like lightweight Virtual Machines, right?

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

@ViscosityNA viscosityna.com

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

@ViscosityNA www.viscosityna.com An Image is the game board and rules; Containers host and persist games; The superimposed Layer saves moves; Deleting a Container removes its Layer.

Slide 46

Slide 46 text

@ViscosityNA www.viscosityna.com

Slide 47

Slide 47 text

@ViscosityNA www.viscosityna.com Stalemate!

Slide 48

Slide 48 text

@ViscosityNA www.viscosityna.com

Slide 49

Slide 49 text

@ViscosityNA www.viscosityna.com

Slide 50

Slide 50 text

@ViscosityNA www.viscosityna.com Database Containers DB Software Configuration Data Image Container Layer

Slide 51

Slide 51 text

@ViscosityNA www.viscosityna.com Database Containers DB Software Configuration Data Container Local Disk Container Layer

Slide 52

Slide 52 text

@ViscosityNA www.viscosityna.com CONTAINER_NAME=punxsutawney_test mkdir -p /data/${CONTAINER_NAME}/data docker volume create \ --opt type=none \ --opt o=bind \ --opt device=/data/${CONTAINER_NAME}/data \ ${CONTAINER_NAME}_data

Slide 53

Slide 53 text

@ViscosityNA www.viscosityna.com An Image is the game board and rules. Containers host and persist games.

Slide 54

Slide 54 text

@ViscosityNA www.viscosityna.com Database Image Rule #1: If there is a database, start it. If not, create a new database.

Slide 55

Slide 55 text

@ViscosityNA www.viscosityna.com Database Container Startup docker run -d \ --name ${CONTAINER_NAME} \ -e ORACLE_SID=ORCL \ -v /data/${CONTAINER_NAME}:/opt/oracle/oradata \ oracle/database:19.3.0-ee

Slide 56

Slide 56 text

@ViscosityNA www.viscosityna.com Clone the Data Directory cp -rp /data/punxsutawney/ \ /data/${CONTAINER_NAME}

Slide 57

Slide 57 text

@ViscosityNA www.viscosityna.com Clone the Data Directory cp -rp /data/punxsutawney/ \ /data/${CONTAINER_NAME} cp -rp /NFS/gold_data/punxsutawney/ \ /data/${CONTAINER_NAME}

Slide 58

Slide 58 text

@ViscosityNA www.viscosityna.com Clone the Data Directory cp -rp /data/punxsutawney/ \ /data/${CONTAINER_NAME} cp -rp /NFS/gold_data/punxsutawney/ \ /data/${CONTAINER_NAME} cp -rp /NFS/gold_data/punxsutawney/2023-02-02.0600/ \ /data/${CONTAINER_NAME}

Slide 59

Slide 59 text

@ViscosityNA www.viscosityna.com If data can be codified, data is versionable.

Slide 60

Slide 60 text

@ViscosityNA www.viscosityna.com Versioned Data 2023-02-02 Wake up campers! Phil? Phil Connors? Six more weeks of winter? Blizzard is coming!

Slide 61

Slide 61 text

@ViscosityNA www.viscosityna.com Versioned Data 2023-02-02 ARMORED_TRUCKS: NULL LAKES_AND_RIVERS: NULL IMPRESSIVE_PIANO: NULL ICE_SCULPTING: NULL select * from knowledge where fname = 'Phil' and lname = 'Connors';

Slide 62

Slide 62 text

@ViscosityNA www.viscosityna.com Versioned Data 2023-02-02 + phil: main 2023-02-02 + phil: day-2 2023-02-02 + phil: redemption

Slide 63

Slide 63 text

@ViscosityNA www.viscosityna.com www.viscosityna.com @ViscosityNA

Slide 64

Slide 64 text

@ViscosityNA www.viscosityna.com Apply lean principles to versioned data.

Slide 65

Slide 65 text

@ViscosityNA www.viscosityna.com Versioned Data 2023-02-02 day-2.sql day-x.sql + +

Slide 66

Slide 66 text

@ViscosityNA www.viscosityna.com Versioned Data 2023-02-02 day-2.sql day-x.sql + +

Slide 67

Slide 67 text

@ViscosityNA www.viscosityna.com Entrypoints • Generally adapted but not an OCI standard • Implementation varies by image, vendor/author • Setup only • Setup & startup

Slide 68

Slide 68 text

@ViscosityNA www.viscosityna.com Versioned Data 2023-02-02 day-2.sql day-x.sql + + day-2.sql day-x.sql Phil Microservice Rita Microservice + +

Slide 69

Slide 69 text

@ViscosityNA www.viscosityna.com Save ancestor data as a dataset.

Slide 70

Slide 70 text

@ViscosityNA www.viscosityna.com Apply versioned SQL to create descendants.

Slide 71

Slide 71 text

@ViscosityNA www.viscosityna.com www.viscosityna.com @ViscosityNA Phil Connors: main └─ day-2 └─ bank-robber └─ piano-student └─ redemption 6:00 AM Phil w akes up Encounters N ed 7:00 AM Tape segm ent 7:20 AM Blizzard! 9:00 AM Punxsutawney_TEST Feb 2 FUNNY

Slide 72

Slide 72 text

@ViscosityNA www.viscosityna.com www.viscosityna.com @ViscosityNA Phil Connors: main 6:00 AM Phil w akes up Punxsutawney_TEST Feb 3 PHIL & RITA LIVE HAPPILY EVER AFTER

Slide 73

Slide 73 text

No content

Slide 74

Slide 74 text

@ViscosityNA www.viscosityna.com Data as Code: Standardizes data Decouples DBAs from the development lifecycle Accelerates & streamlines development Assures test reliability and quality

Slide 75

Slide 75 text

@ViscosityNA www.viscosityna.com Contact Me sean.scott@viscosityna.com https://linktr.ee/oraclesean