Aren't we forgetting someone?

Aren't we forgetting someone?

How do DBAs fit into the world of DevOps? We usually think of DevOps as a collaboration between developers and operations or support engineers so it’s easy to forget the database admins who occupy both the development and operations space.

Organisationally they are often separate, sometimes geographically as well as organisationally. How does the database fit into the DevOps or Continuous Delivery story, both technologically and culturally?

Dan North has been helping a number of large organisations at various stages on their journey to continuous delivery and in every case their legacy database dependencies are a major stumbling block. Where do you start unpicking the database? How can you tell if it is working? Is the Matrix just a very big Oracle instance?

08145ecb1ce091d9dd3c328ea2a707fb?s=128

Daniel Terhorst-North

February 04, 2016
Tweet

Transcript

  1. Aren’t we forgetting someone? Dan North @tastapod

  2. @tastapod The Challenge

  3. @tastapod Can we treat the “database as code”?

  4. @tastapod Can we treat the “database as code”? Servers as

    code
  5. @tastapod Can we treat the “database as code”? Servers as

    code Network and DNS as code
  6. @tastapod Can we treat the “database as code”? Servers as

    code Network and DNS as code Release management as code
  7. @tastapod Can we treat the “database as code”? Servers as

    code Network and DNS as code Configuration as code Release management as code
  8. @tastapod Can we treat the “database as code”? Servers as

    code Network and DNS as code Configuration as code Release management as code Database as code
  9. @tastapod Database as code

  10. @tastapod Database as code All changes scripted and automated

  11. @tastapod Database as code All changes scripted and automated All

    changes under version control
  12. @tastapod Database as code All changes scripted and automated All

    changes under version control Ability to release on demand
  13. @tastapod Database as code All changes scripted and automated All

    changes under version control Ability to release on demand DBAs integrated with Dev and Ops
  14. @tastapod The Context

  15. @tastapod This is your architecture

  16. @tastapod This is your architecture

  17. None
  18. id name language 1 chef ruby 2 puppet ruby 3

    ansible python 4 salt python products
  19. id name language 1 chef ruby 2 puppet ruby 3

    ansible python 4 salt python products relation (table)
  20. id name language 1 chef ruby 2 puppet ruby 3

    ansible python 4 salt python products field (column) relation (table)
  21. id name language 1 chef ruby 2 puppet ruby 3

    ansible python 4 salt python products field (column) record (row) relation (table)
  22. id name language 1 chef ruby 2 puppet ruby 3

    ansible python 4 salt python products field (column) record (row) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors
  23. id name language 1 chef ruby 2 puppet ruby 3

    ansible python 4 salt python products field (column) record (row) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key
  24. id name language 1 chef ruby 2 puppet ruby 3

    ansible python 4 salt python products field (column) record (row) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key index
  25. id name language 1 chef ruby 2 puppet ruby 3

    ansible python 4 salt python products field (column) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key UPDATE products SET name = … stored proc index record (row)
  26. id name language 1 chef ruby 2 puppet ruby 3

    ansible python 4 salt python products field (column) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key UPDATE products SET name = … stored proc constraint index record (row)
  27. id name language 1 chef ruby 2 puppet ruby 3

    ansible python 4 salt python products field (column) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key UPDATE products SET name = … stored proc constraint trigger index record (row)
  28. id name language 1 chef ruby 2 puppet ruby 3

    ansible python 4 salt python products field (column) relation (table) id name product_id 10 Puppet Labs 2 11 Chef inc. 1 12 SaltStack 4 13 Red Hat 3 vendors foreign key UPDATE products SET name = … stored proc constraint trigger index table_size = … heap_size = … free_space = … last_access = … heuristics record (row)
  29. structural data application data reference data procs, triggers Databases contain

    data
  30. structural data application data reference data procs, triggers Lots and

    lots of data!
  31. @tastapod The Constraints

  32. @tastapod Database changes come from multiple directions

  33. @tastapod Database changes come from multiple directions Application changes

  34. @tastapod Database changes come from multiple directions Application changes Data

    growth
  35. @tastapod Database changes come from multiple directions Application changes Data

    growth Policy changes
  36. @tastapod Database changes come from multiple directions Application changes Performance

    needs Data growth Policy changes
  37. @tastapod Database changes come from multiple directions Application changes Performance

    needs Data growth Upgrades and patches Policy changes
  38. @tastapod Database changes come from multiple directions Application changes Performance

    needs Data growth Upgrades and patches Policy changes Dev
  39. @tastapod Database changes come from multiple directions Application changes Performance

    needs Policy changes Dev Business Data growth Upgrades and patches
  40. @tastapod Database changes come from multiple directions Application changes Performance

    needs Data growth Upgrades and patches Dev Ops Policy changes Business
  41. @tastapod DBAs have been bridging Dev and Ops for years!

    Dev Ops
  42. @tastapod DBAs have been bridging Dev and Ops for years!

    Dev Ops but not how we would define it…
  43. reference data structural data application data procs, triggers Multiple data

    lifecycles
  44. reference data structural data application data procs, triggers Multiple data

    lifecycles changes with app changes with app
  45. reference data changes with business structural data application data procs,

    triggers Multiple data lifecycles changes with app changes with app
  46. reference data changes with business structural data application data procs,

    triggers Multiple data lifecycles changes with app changes with app changes with usage
  47. @tastapod Application data is long-lived

  48. @tastapod Application data is long-lived Outlives releases

  49. @tastapod Application data is long-lived Outlives releases Outlives applications

  50. @tastapod Application data is long-lived Outlives releases Outlives applications Outlives

    people!
  51. @tastapod Database is often an integration point

  52. @tastapod Database is often an integration point

  53. @tastapod Database is often an integration point

  54. @tastapod Database is often an integration point

  55. @tastapod Database mistakes can have a high impact

  56. @tastapod Database mistakes can have a high impact Performance impact

  57. @tastapod Database mistakes can have a high impact Cascading failures

    Performance impact
  58. @tastapod Database mistakes can have a high impact Cascading failures

    Performance impact Coincident failures
  59. @tastapod Database mistakes can have a high impact Cascading failures

    Performance impact Data loss Coincident failures
  60. @tastapod Database mistakes can have a high impact Cascading failures

    Performance impact Data loss Data corruption Coincident failures
  61. @tastapod Database mistakes can have a high impact Cascading failures

    Performance impact Data loss Data corruption Detection can be distant in space and time, and cost Coincident failures
  62. @tastapod DevOps relies on minimising impact impact likelihood

  63. @tastapod DevOps relies on minimising impact impact likelihood MTBF

  64. @tastapod DevOps relies on minimising impact impact likelihood MTBF MTTR

  65. @tastapod DevOps relies on minimising impact impact likelihood We used

    to obsess about this MTBF MTTR
  66. @tastapod DevOps relies on minimising impact impact likelihood We used

    to obsess about this DevOps cares about this MTBF MTTR
  67. @tastapod

  68. DBAs are conservative and change-cautious @tastapod

  69. DevOps is relentless and change-eager DBAs are conservative and change-cautious

    @tastapod
  70. DevOps is relentless and change-eager This is the culture clash!

    DBAs are conservative and change-cautious @tastapod
  71. @tastapod The Approach

  72. @tastapod The Status Quo

  73. @tastapod The Status Quo Most changes are manual

  74. @tastapod The Status Quo Most changes are manual Driven by

    tickets or change requests
  75. @tastapod The Status Quo Most changes are manual Driven by

    tickets or change requests Carefully planned and scrutinised
  76. @tastapod The Status Quo Most changes are manual Driven by

    tickets or change requests Carefully planned and scrutinised Fragmented audit trail
  77. @tastapod The Status Quo Most changes are manual Driven by

    tickets or change requests Carefully planned and scrutinised Fragmented audit trail Expensive and time-consuming to repeat
  78. @tastapod Database-as-code #1: Migrations

  79. @tastapod Database-as-code #1: Migrations current

  80. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN …

    … INSERT INTO ref_data… … current
  81. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN …

    … INSERT INTO ref_data… … current new
  82. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN …

    … INSERT INTO ref_data… … current new VCS
  83. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN …

    … INSERT INTO ref_data… … current new dbdeploy (RIP) VCS
  84. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN …

    … INSERT INTO ref_data… … current new dbdeploy (RIP) rails VCS
  85. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN …

    … INSERT INTO ref_data… … current new dbdeploy (RIP) rails flyway VCS
  86. @tastapod Database-as-code #1: Migrations ALTER TABLE … ADD COLUMN …

    … INSERT INTO ref_data… … current new dbdeploy (RIP) rails flyway liquibase VCS
  87. @tastapod Code-as-code: A comparison ALTER TABLE … ADD COLUMN …

    … INSERT INTO ref_data… … current new VCS
  88. @tastapod VCS Code-as-code: A comparison - old_code… … + new_code…

    … current new
  89. @tastapod VCS Code-as-code: A comparison - old_code… … + new_code…

    … current new Developer edits the code
  90. @tastapod VCS Code-as-code: A comparison - old_code… … + new_code…

    … current new Developer edits the code VCS figures out the script
  91. @tastapod VCS Code-as-code: A comparison - old_code… … + new_code…

    … current new Developer edits the code VCS figures out the script VCS can create new state
  92. @tastapod VCS Database-as-code #2: DBVCS? ALTER TABLE … ADD COLUMN

    … … INSERT INTO ref_data… … current new Developer edits the database VCS figures out the script VCS can create new state
  93. @tastapod VCS Database-as-code #2: DBVCS? ALTER TABLE … ADD COLUMN

    … … INSERT INTO ref_data… … current new Red Gate is close but commercial. No OSS equivalent Developer edits the database VCS figures out the script VCS can create new state
  94. @tastapod The Transition

  95. @tastapod Engage with your DBA

  96. @tastapod Engage with your DBA Their context is broader than

    yours
  97. @tastapod Engage with your DBA Their context is broader than

    yours They already have a foot in both camps
  98. @tastapod Engage with your DBA Their context is broader than

    yours They already have a foot in both camps Elevate the constraint
  99. @tastapod Engage with your DBA Their context is broader than

    yours They already have a foot in both camps Elevate the constraint Don’t “DevOps” the DBAs!
  100. @tastapod Engage with your DBA Their context is broader than

    yours They already have a foot in both camps Elevate the constraint Don’t “DevOps” the DBAs! Now, who else are we forgetting?
  101. @tastapod Thanks for listening! Dan North @tastapod dan@dannorth.net http:/ /dannorth.net