Aren't we forgetting someone?

How do DBAs fit into the world of DevOps? We usually think of DevOps as a collaboration between developers and operations or support engineers so it’s easy to forget the database admins who occupy both the development and operations space.

Organisationally they are often separate, sometimes geographically as well as organisationally. How does the database fit into the DevOps or Continuous Delivery story, both technologically and culturally?

Dan North has been helping a number of large organisations at various stages on their journey to continuous delivery and in every case their legacy database dependencies are a major stumbling block. Where do you start unpicking the database? How can you tell if it is working? Is the Matrix just a very big Oracle instance?

Transcript

1. Aren’t we forgetting someone? Dan North @tastapod
2. @tastapod The Challenge
3. @tastapod Can we treat the “database as code”?
4. @tastapod Can we treat the “database as code”? Servers as code
5. @tastapod Can we treat the “database as code”? Servers as code Network and DNS as code
6. @tastapod Can we treat the “database as code”? Servers as code Network and DNS as code Release management as code
7. @tastapod Can we treat the “database as code”? Servers as code Network and DNS as code Configuration as code Release management as code
8. @tastapod Can we treat the “database as code”? Servers as code Network and DNS as code Configuration as code Release management as code Database as code
9. @tastapod Database as code
10. @tastapod Database as code All changes scripted and automated
11. @tastapod Database as code All changes scripted and automated All changes under version control
12. @tastapod Database as code All changes scripted and automated All changes under version control Ability to release on demand
13. @tastapod Database as code All changes scripted and automated All changes under version control Ability to release on demand DBAs integrated with Dev and Ops
14. @tastapod The Context
15. @tastapod This is your architecture
16. @tastapod This is your architecture
18. id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products
19. id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products relation (table)
20. id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) relation (table)
21. id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) record (row) relation (table)
22. id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) record (row) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors
23. id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) record (row) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key
24. id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) record (row) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key index
25. id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key UPDATE products SET name = … stored proc index record (row)
26. id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key UPDATE products SET name = … stored proc constraint index record (row)
27. id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key UPDATE products SET name = … stored proc constraint trigger index record (row)
28. id name language 1 chef ruby 2 puppet ruby 3 ansible python 4 salt python products field (column) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key UPDATE products SET name = … stored proc constraint trigger index table_size = … heap_size = … free_space = … last_access = … heuristics record (row)
29. structural data application data reference data procs, triggers Databases contain data
30. structural data application data reference data procs, triggers Lots and lots of data!
31. @tastapod The Constraints
32. @tastapod Database changes come from multiple directions
33. @tastapod Database changes come from multiple directions Application changes
34. @tastapod Database changes come from multiple directions Application changes Data growth
35. @tastapod Database changes come from multiple directions Application changes Data growth Policy changes
36. @tastapod Database changes come from multiple directions Application changes Performance needs Data growth Policy changes
37. @tastapod Database changes come from multiple directions Application changes Performance needs Data growth Upgrades and patches Policy changes
38. @tastapod Database changes come from multiple directions Application changes Performance needs Data growth Upgrades and patches Policy changes Dev
39. @tastapod Database changes come from multiple directions Application changes Performance needs Policy changes Dev Business Data growth Upgrades and patches
40. @tastapod Database changes come from multiple directions Application changes Performance needs Data growth Upgrades and patches Dev Ops Policy changes Business
41. @tastapod DBAs have been bridging Dev and Ops for years! Dev Ops
42. @tastapod DBAs have been bridging Dev and Ops for years! Dev Ops but not how we would define it…
43. reference data structural data application data procs, triggers Multiple data lifecycles
44. reference data structural data application data procs, triggers Multiple data lifecycles changes with app changes with app
45. reference data changes with business structural data application data procs, triggers Multiple data lifecycles changes with app changes with app
46. reference data changes with business structural data application data procs, triggers Multiple data lifecycles changes with app changes with app changes with usage
47. @tastapod Application data is long-lived
48. @tastapod Application data is long-lived Outlives releases
49. @tastapod Application data is long-lived Outlives releases Outlives applications
50. @tastapod Application data is long-lived Outlives releases Outlives applications Outlives people!
51. @tastapod Database is often an integration point
52. @tastapod Database is often an integration point
53. @tastapod Database is often an integration point
54. @tastapod Database is often an integration point
55. @tastapod Database mistakes can have a high impact
56. @tastapod Database mistakes can have a high impact Performance impact
57. @tastapod Database mistakes can have a high impact Cascading failures Performance impact
58. @tastapod Database mistakes can have a high impact Cascading failures Performance impact Coincident failures
59. @tastapod Database mistakes can have a high impact Cascading failures Performance impact Data loss Coincident failures
60. @tastapod Database mistakes can have a high impact Cascading failures Performance impact Data loss Data corruption Coincident failures
61. @tastapod Database mistakes can have a high impact Cascading failures Performance impact Data loss Data corruption Detection can be distant in space and time, and cost Coincident failures
62. @tastapod DevOps relies on minimising impact impact likelihood
63. @tastapod DevOps relies on minimising impact impact likelihood MTBF
64. @tastapod DevOps relies on minimising impact impact likelihood MTBF MTTR
65. @tastapod DevOps relies on minimising impact impact likelihood We used to obsess about this MTBF MTTR
66. @tastapod DevOps relies on minimising impact impact likelihood We used to obsess about this DevOps cares about this MTBF MTTR
67. @tastapod
68. DBAs are conservative and change-cautious @tastapod
69. DevOps is relentless and change-eager DBAs are conservative and change-cautious @tastapod
70. DevOps is relentless and change-eager This is the culture clash! DBAs are conservative and change-cautious @tastapod
71. @tastapod The Approach
72. @tastapod The Status Quo
73. @tastapod The Status Quo Most changes are manual
74. @tastapod The Status Quo Most changes are manual Driven by tickets or change requests
75. @tastapod The Status Quo Most changes are manual Driven by tickets or change requests Carefully planned and scrutinised
76. @tastapod The Status Quo Most changes are manual Driven by tickets or change requests Carefully planned and scrutinised Fragmented audit trail
77. @tastapod The Status Quo Most changes are manual Driven by tickets or change requests Carefully planned and scrutinised Fragmented audit trail Expensive and time-consuming to repeat
78. @tastapod Database-as-code #1: Migrations
79. @tastapod Database-as-code #1: Migrations current
80. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current
81. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new
82. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new VCS
83. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new dbdeploy (RIP) VCS
84. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new dbdeploy (RIP) rails VCS
85. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new dbdeploy (RIP) rails flyway VCS
86. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new dbdeploy (RIP) rails flyway liquibase VCS
87. @tastapod Code-as-code: A comparison ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new VCS
88. @tastapod VCS Code-as-code: A comparison - old_code… … + new_code… … current new
89. @tastapod VCS Code-as-code: A comparison - old_code… … + new_code… … current new Developer edits the code
90. @tastapod VCS Code-as-code: A comparison - old_code… … + new_code… … current new Developer edits the code VCS figures out the script
91. @tastapod VCS Code-as-code: A comparison - old_code… … + new_code… … current new Developer edits the code VCS figures out the script VCS can create new state
92. @tastapod VCS Database-as-code #2: DBVCS? ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new Developer edits the database VCS figures out the script VCS can create new state
93. @tastapod VCS Database-as-code #2: DBVCS? ALTER TABLE … ADD COLUMN … … INSERT INTO ref_data… … current new Red Gate is close but commercial. No OSS equivalent Developer edits the database VCS figures out the script VCS can create new state
94. @tastapod The Transition
95. @tastapod Engage with your DBA
96. @tastapod Engage with your DBA Their context is broader than yours
97. @tastapod Engage with your DBA Their context is broader than yours They already have a foot in both camps
98. @tastapod Engage with your DBA Their context is broader than yours They already have a foot in both camps Elevate the constraint
99. @tastapod Engage with your DBA Their context is broader than yours They already have a foot in both camps Elevate the constraint Don’t “DevOps” the DBAs!
100. @tastapod Engage with your DBA Their context is broader than yours They already have a foot in both camps Elevate the constraint Don’t “DevOps” the DBAs! Now, who else are we forgetting?
101. @tastapod Thanks for listening! Dan North @tastapod dan@dannorth.net http:/ /dannorth.net