Tweet
Tweet

Slide 1

Slide 1 text

Aren’t we forgetting someone? Dan North @tastapod

Slide 2

Slide 2 text

@tastapod The Challenge

Slide 3

Slide 3 text

@tastapod Can we treat the “database as code”?

Slide 4

Slide 4 text

@tastapod Can we treat the “database as code”? Servers as code

Slide 5

Slide 5 text

@tastapod Can we treat the “database as code”? Servers as code Network and DNS as code

Slide 6

Slide 6 text

@tastapod Can we treat the “database as code”? Servers as code Network and DNS as code Release management as code

Slide 7

Slide 7 text

@tastapod Can we treat the “database as code”? Servers as code Network and DNS as code Configuration as code Release management as code

Slide 8

Slide 8 text

@tastapod Can we treat the “database as code”? Servers as code Network and DNS as code Configuration as code Release management as code Database as code

Slide 9

Slide 9 text

@tastapod Database as code

Slide 10

Slide 10 text

@tastapod Database as code All changes scripted and automated

Slide 11

Slide 11 text

@tastapod Database as code All changes scripted and automated All changes under version control

Slide 12

Slide 12 text

@tastapod Database as code All changes scripted and automated All changes under version control Ability to release on demand

Slide 13

Slide 13 text

@tastapod Database as code All changes scripted and automated All changes under version control Ability to release on demand DBAs integrated with Dev and Ops

Slide 14

Slide 14 text

@tastapod The Context

Slide 15

Slide 15 text

@tastapod This is your architecture

Slide 16

Slide 16 text

@tastapod This is your architecture

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products

Slide 19

Slide 19 text

id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products relation (table)

Slide 20

Slide 20 text

id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) relation (table)

Slide 21

Slide 21 text

id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) record (row) relation (table)

Slide 22

Slide 22 text

id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) record (row) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors

Slide 23

Slide 23 text

id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) record (row) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key

Slide 24

Slide 24 text

id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) record (row) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key index

Slide 25

Slide 25 text

id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key UPDATE products SET name = … stored proc index record (row)

Slide 26

Slide 26 text

id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key UPDATE products SET name = … stored proc constraint index record (row)

Slide 27

Slide 27 text

id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key UPDATE products SET name = … stored proc constraint trigger index record (row)

Slide 28

Slide 28 text

id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key UPDATE products SET name = … stored proc constraint trigger index table_size = … heap_size = … free_space = … last_access = … heuristics record (row)

Slide 29

Slide 29 text

structural data application data reference data procs, triggers Databases contain data

Slide 30

Slide 30 text

structural data application data reference data procs, triggers Lots and lots of data!

Slide 31

Slide 31 text

@tastapod The Constraints

Slide 32

Slide 32 text

@tastapod Database changes come from multiple directions

Slide 33

Slide 33 text

@tastapod Database changes come from multiple directions Application changes

Slide 34

Slide 34 text

@tastapod Database changes come from multiple directions Application changes Data growth

Slide 35

Slide 35 text

@tastapod Database changes come from multiple directions Application changes Data growth Policy changes

Slide 36

Slide 36 text

@tastapod Database changes come from multiple directions Application changes Performance needs Data growth Policy changes

Slide 37

Slide 37 text

@tastapod Database changes come from multiple directions Application changes Performance needs Data growth Upgrades and patches Policy changes

Slide 38

Slide 38 text

@tastapod Database changes come from multiple directions Application changes Performance needs Data growth Upgrades and patches Policy changes Dev

Slide 39

Slide 39 text

@tastapod Database changes come from multiple directions Application changes Performance needs Policy changes Dev Business Data growth Upgrades and patches

Slide 40

Slide 40 text

@tastapod Database changes come from multiple directions Application changes Performance needs Data growth Upgrades and patches Dev Ops Policy changes Business

Slide 41

Slide 41 text

@tastapod DBAs have been bridging Dev and Ops for years! Dev Ops

Slide 42

Slide 42 text

@tastapod DBAs have been bridging Dev and Ops for years! Dev Ops but not how we would define it…

Slide 43

Slide 43 text

reference data structural data application data procs, triggers Multiple data lifecycles

Slide 44

Slide 44 text

reference data structural data application data procs, triggers Multiple data lifecycles changes with app changes with app

Slide 45

Slide 45 text

reference data changes with business structural data application data procs, triggers Multiple data lifecycles changes with app changes with app

Slide 46

Slide 46 text

reference data changes with business structural data application data procs, triggers Multiple data lifecycles changes with app changes with app changes with usage

Slide 47

Slide 47 text

@tastapod Application data is long-lived

Slide 48

Slide 48 text

@tastapod Application data is long-lived Outlives releases

Slide 49

Slide 49 text

@tastapod Application data is long-lived Outlives releases Outlives applications

Slide 50

Slide 50 text

@tastapod Application data is long-lived Outlives releases Outlives applications Outlives people!

Slide 51

Slide 51 text

@tastapod Database is often an integration point

Slide 52

Slide 52 text

@tastapod Database is often an integration point

Slide 53

Slide 53 text

@tastapod Database is often an integration point

Slide 54

Slide 54 text

@tastapod Database is often an integration point

Slide 55

Slide 55 text

@tastapod Database mistakes can have a high impact

Slide 56

Slide 56 text

@tastapod Database mistakes can have a high impact Performance impact

Slide 57

Slide 57 text

@tastapod Database mistakes can have a high impact Cascading failures Performance impact

Slide 58

Slide 58 text

@tastapod Database mistakes can have a high impact Cascading failures Performance impact Coincident failures

Slide 59

Slide 59 text

@tastapod Database mistakes can have a high impact Cascading failures Performance impact Data loss Coincident failures

Slide 60

Slide 60 text

@tastapod Database mistakes can have a high impact Cascading failures Performance impact Data loss Data corruption Coincident failures

Slide 61

Slide 61 text

@tastapod Database mistakes can have a high impact Cascading failures Performance impact Data loss Data corruption Detection can be distant in space and time, and cost Coincident failures

Slide 62

Slide 62 text

@tastapod DevOps relies on minimising impact impact likelihood

Slide 63

Slide 63 text

@tastapod DevOps relies on minimising impact impact likelihood MTBF

Slide 64

Slide 64 text

@tastapod DevOps relies on minimising impact impact likelihood MTBF MTTR

Slide 65

Slide 65 text

@tastapod DevOps relies on minimising impact impact likelihood We used to obsess about this MTBF MTTR

Slide 66

Slide 66 text

@tastapod DevOps relies on minimising impact impact likelihood We used to obsess about this DevOps cares about this MTBF MTTR

Slide 67

Slide 67 text

@tastapod

Slide 68

Slide 68 text

DBAs are conservative and change-cautious @tastapod

Slide 69

Slide 69 text

DevOps is relentless and change-eager DBAs are conservative and change-cautious @tastapod

Slide 70

Slide 70 text

DevOps is relentless and change-eager This is the culture clash! DBAs are conservative and change-cautious @tastapod

Slide 71

Slide 71 text

@tastapod The Approach

Slide 72

Slide 72 text

@tastapod The Status Quo

Slide 73

Slide 73 text

@tastapod The Status Quo Most changes are manual

Slide 74

Slide 74 text

@tastapod The Status Quo Most changes are manual Driven by tickets or change requests

Slide 75

Slide 75 text

@tastapod The Status Quo Most changes are manual Driven by tickets or change requests Carefully planned and scrutinised

Slide 76

Slide 76 text

@tastapod The Status Quo Most changes are manual Driven by tickets or change requests Carefully planned and scrutinised Fragmented audit trail

Slide 77

Slide 77 text

@tastapod The Status Quo Most changes are manual Driven by tickets or change requests Carefully planned and scrutinised Fragmented audit trail Expensive and time-consuming to repeat

Slide 78

Slide 78 text

@tastapod Database-as-code #1: Migrations

Slide 79

Slide 79 text

@tastapod Database-as-code #1: Migrations current

Slide 80

Slide 80 text

@tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current

Slide 81

Slide 81 text

@tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new

Slide 82

Slide 82 text

@tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new VCS

Slide 83

Slide 83 text

@tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new dbdeploy (RIP) VCS

Slide 84

Slide 84 text

@tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new dbdeploy (RIP) rails VCS

Slide 85

Slide 85 text

@tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new dbdeploy (RIP) rails flyway VCS

Slide 86

Slide 86 text

@tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new dbdeploy (RIP) rails flyway liquibase VCS

Slide 87

Slide 87 text

@tastapod Code-as-code: A comparison ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new VCS

Slide 88

Slide 88 text

@tastapod VCS Code-as-code: A comparison - old_code… … + new_code… … current new

Slide 89

Slide 89 text

@tastapod VCS Code-as-code: A comparison - old_code… … + new_code… … current new Developer edits the code

Slide 90

Slide 90 text

@tastapod VCS Code-as-code: A comparison - old_code… … + new_code… … current new Developer edits the code VCS figures out the script

Slide 91

Slide 91 text

@tastapod VCS Code-as-code: A comparison - old_code… … + new_code… … current new Developer edits the code VCS figures out the script VCS can create new state

Slide 92

Slide 92 text

@tastapod VCS Database-as-code #2: DBVCS? ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new Developer edits the database VCS figures out the script VCS can create new state

Slide 93

Slide 93 text

@tastapod VCS Database-as-code #2: DBVCS? ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new Red Gate is close but commercial. No OSS equivalent Developer edits the database VCS figures out the script VCS can create new state

Slide 94

Slide 94 text

@tastapod The Transition

Slide 95

Slide 95 text

@tastapod Engage with your DBA

Slide 96

Slide 96 text

@tastapod Engage with your DBA Their context is broader than yours

Slide 97

Slide 97 text

@tastapod Engage with your DBA Their context is broader than yours They already have a foot in both camps

Slide 98

Slide 98 text

@tastapod Engage with your DBA Their context is broader than yours They already have a foot in both camps Elevate the constraint

Slide 99

Slide 99 text

@tastapod Engage with your DBA Their context is broader than yours They already have a foot in both camps Elevate the constraint Don’t “DevOps” the DBAs!

Slide 100

Slide 100 text

@tastapod Engage with your DBA Their context is broader than yours They already have a foot in both camps Elevate the constraint Don’t “DevOps” the DBAs! Now, who else are we forgetting?

Slide 101

Slide 101 text

@tastapod Thanks for listening! Dan North @tastapod dan@dannorth.net http:/ /dannorth.net