Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deploying Drupal: Patterns and Antipatterns

Deploying Drupal: Patterns and Antipatterns

A session about the evolution of deployment process from FTP to config management, GIT workflows and automation from Drupal Dev Days Seville. Originally held at Drupal Mountain Camp in Davos.

Branislav Bujisic

March 22, 2017
Tweet

More Decks by Branislav Bujisic

Other Decks in Programming

Transcript

  1. Deploying Drupal: Patterns and Antipatterns

  2. Hi. Branislav. Drupal guy.

  3. None
  4. None
  5. None
  6. None
  7. None
  8. Continuous Deployment Cloud Hosting

  9. Always Be Compiling Why would I compile my PHP application?

  10. interpreted language (no compiling)

  11. Deployment before life became complicated

  12. • Open your favourite
 FTP client
 Deployment before life became

    complicated
  13. • Open your favourite
 FTP client
 • Replace files on

    server with your updated PHP Deployment before life became complicated
  14. • Open your favourite
 FTP client
 • Replace files on

    server with your updated PHP
 • Hit F5 in your browser
 • Life is good! Deployment before life became complicated
  15. http://www.mysite.com/login.php?user=admin&pass=admin Warning: fopen(/../users-and-passwords.txt) [function.fopen]: failed to open stream. No such

    file or directory in /home/ my_site/public_html/login.php on line 56 Warning: stream_get_contents() expects parameter 1 to be resource, boolean given in /home/my_site/public_html/ login.php on line 57 Warning: fclose() supplied argument is not a valid stream resource in /home/my_site/public_html/login.php on line 58
  16. ;php.ini display_errors = Off

  17. Every solution we introduce to solve any problem,
 no matter

    how trivial, increases development complexity. The curse of being a web developer
  18. Revision Control 1982 RCS (Revision Control System ) 1986 CVS

    (Concurrent Version System ) 1989 Sir Tim Berners-Lee invents W eb 2005 Git 2011 Git @ drupal.org 2001 Drupal release
  19. Git • It replaced well established FTP workflows. • It

    introduced new and strange commands and concepts. • Due to steep learning curve, only a fraction of it was used.
  20. Early Git Workflow git add -A git commit -m “whatever”

    git pull origin master git push origin master ssh root@myserver.com cd /var/www/mysite git pull origin master
  21. Why is it an Antipattern? • Power of Git lies

    in branching • Publicly accessible .git directory is a serious security risk • SSH to server for every single pull? • Boring task that can easily be automated!
  22. post-receive hook #!/bin/sh GIT_WORK_TREE=/var/www/my_project git checkout -f Local git push

    Git /home/git/my_project.git Web root /var/www
  23. Things to Consider • Post-receive hook can do much more

    than simply copying the working tree. • Control access to repositories. • Multiple sites on a single server, all owned by git, all run by apache user is not a good idea. 23
  24. SA-CORE-2014-005 - Drupal core - SQL injection

  25. Code review diff --git a/ds_bootstrap_layouts.module b/ds_bootstrap_layouts.module index b17b762..78dbe9d 100644 ---

    a/ds_bootstrap_layouts.module +++ b/ds_bootstrap_layouts.module @@ -128,7 +128,20 @@ function ds_bootstrap_layouts_ds_layout_info() { 'form' => TRUE, 'image' => TRUE, ); - + + $layouts['bootstrap_7_5_stacked'] = array( + 'label' => t('Bootstrap Two Columns: 7-5 stacked'), + 'path' => $twocol . '/bootstrap_7_5_stacked', + 'regions' => array( + 'top' => t('Top'), + 'left' => t('Left'), + 'right' => t('Right'), + 'bottom' => t('Bottom'), + ), + 'form' => TRUE, + 'image' => TRUE, + ); +
  26. Code review git pull origin master git fetch origin feature-branch

    git checkout feature-branch git pull git diff master > my-diff-file.patch git checkout -b feature-branch git add -A git commit -m “Stuff” git push origin feature-branch
  27. None
  28. Why is it an Antipattern? • it is just time

    consuming and boring! • The need for inline code commenting. • 100’s comments in issue queue. • There are some better tools for that, such as Gerrit, Github or Gitlab
  29. Pull Request Workflow be54c33 Created… f1a971a Fixed… 1ca3582 Updated… fork

    Fork repository Main
 repository GitHub Local pull push pull request Local Local
  30. None
  31. Manual deployment after code review fork Fork repository Main
 repository

    GitHub Local pull pull request push post-receive Web root Git repo Hosting
 Environment
  32. diff --git a/CHANGELOG.txt b/CHANGELOG.txt index 4853101..c015fb4 100644 --- a/CHANGELOG.txt +++

    b/CHANGELOG.txt @@ -1,4 +1,53 @@ +Drupal 7.54, 2017-02-01 +----------------------- +- Modules are now able to define theme engines (API ad + https://www.drupal.org/node/2826480). +- Logging of searches can now be disabled (new option + interface). +- Added menu tree render structure to (pre-)process ho theme_menu_tree() + (API addition: https://www.drupal.org/node/2827134). +- Added new function for determining whether an HTTPS + (API addition: https://www.drupal.org/node/2824590). +- Fixed incorrect default value for short and medium d + type configuration page. +- File validation error message is now removed after s valid + file. +- Numerous bug fixes. +- Numerous API documentation improvements. +- Additional performance improvements. +- Additional automated test coverage. + +Drupal 7.53, 2016-12-07 +----------------------- +- Fixed drag and drop support on newer Chrome/IE 11+ v update + when jQuery is updated to 1.7-1.11.0. + +Drupal 7.52, 2016-11-16 Meanwhile in Planet Drupal • Somebody updated core from 7.50 to 7.54 • 4501 lines of diff • Impossible to review • Entire git history soon becomes clogged and useless after a dozen of core+module updates.
  33. Meanwhile in Planet Drupal Patching (hacking) core and contrib Make

    a change in core/contrib Maintain a list of hacked files Never perform an update again Make a patch file Commit the patch file Maintain a list of patches Update a module git apply -v name_of.patch
  34. Drush make • Built to be a packaging tool •

    …but became a patching tool • Downloads Drupal core, modules and themes from d.o. • Downloads libraries in .tar.gz or .zip format • Fetches and applies patches
  35. The anatomy of makefile core = 7.x api = 2

    defaults[projects][subdir] = contrib projects[drupal][version] = “7.52" projects[ctools][version] = “1.10” ; remote patch projects[ctools][patch][] = “https://www.drupal.org/.../2839602_1.patch”
  36. The anatomy of YAML flavoured
 makefile core: 7.x api: 2

    defaults: projects: subdir: “contrib” projects: drupal: version: “7.52” ctools: version: “1.10”
 patch: - “https://www.drupal.org/.../2839602_1.patch"
  37. drush make project.make -y • Downloads Drupal • Downloads ctools

    into /sites/all/modules/contrib • Downloads and applies a remote patch
  38. Updating a module defaults[projects][subdir] = contrib projects[drupal][version] = “7.52” -projects[ctools][version]

    = “1.10” +projects[ctools][version] = “1.12” ; remote patch projects[ctools][patch][] = “https://www.drupal.org/.../2839602_1.patch" Commit the change Deploy to server drush make project.make -y
  39. Things to Consider • Prepare a backup strategy if things

    go wrong. • Running drush make on live server can and will result in site instability. • Drush make does not install Drupal or contrib modules. • Drush make does not run database update. • Drush make does not handle dependencies.
  40. Composer • Dependency management • Autoloader • Keeps the code

    updated • Feels nerdy! • Universal PHP tool
  41. The anatomy of composer.json file
 (Drupal-relevant parts) “require”: {
 "drupal/core":

    "~8.0"
 } “requre-dev”: {
 "behat/mink": "~1.7"
 } “scripts”: {
 "post-install-cmd": [
 "DrupalProject\\…\\ScriptHandler::createRequiredFiles"
 ]
 } “extra”: {
 “arbitrary”: “data”
 }
  42. Adding a module to project - Download Address module -

    Download Entity module - Download State Machine module - Download Inline Entity Form module - Download Profile module - Download needed libraries - Update composer.json and composer.lock composer require "drupal/commerce 2.x-dev" git add composer.json composer.lock git commit -m “Add Drupal Commerce” git push origin master
  43. Deployment to the web host - Download Address module -

    Download Entity module - Download State Machine module - Download Inline Entity Form module - Download Profile module - Download needed libraries composer install
  44. Composer isn’t Drush • Does not do patching. • Not

    aware of Drupal’s directory structure. • Drupal.org has its own package repository. • There’s some new syntax to be learned. • Not a speed champion!
  45. The fuss of setting up the project Getting the core

    composer create-project drupal/drupal my-d8-site 8.2.* Wait for 131 seconds in average 0 35 70 105 140 Drush Composer (warm caches) Composer (cold caches) 131s 31s 12s
  46. The fuss of setting up the project Setting the right

    repository composer config repositories.drupal composer https://packages.drupal.org/8 "repositories": { "drupal": { "type": "composer", "url": "https://packages.drupal.org/8" } }
  47. The fuss of setting up the project Making sure modules

    are downloaded to appropriate folders "extra": {
 "installer-paths": {
 "modules/contrib/{$name}": ["type:drupal-module"],
 "profiles/contrib/{$name}": ["type:drupal-profile"],
 "themes/contrib/{$name}": ["type:drupal-theme"]
 }
 } "require": {
 "composer/installers": "^1.0.20"
 }
  48. The fuss of setting up the project Enabling patching composer

    require cweagans/composer-patches "require": {
 "cweagans/composer-patches": "~1.0"
 }
  49. Applying patches “extra”: { "patches": { "drupal/admin_toolbar": { "Startup config":

    “https://www.drupal.org/...-2781745-3.patch”, "Local patch": "patches/my-patch.patch" } } }
  50. Deployment to the web host - Delete outdated code -

    Download new and updated code - Download dependencies - Apply patches composer install
  51. Why is it an antipattern? • It is not! •

    It is just time consuming and boring to set up the boilerplate • Somebody did it for us.
  52. Composer template
 for Drupal projects composer create-project drupal-composer/drupal-project • Installs

    Drupal in web directory • Uses composer-generated autoloader • Ships with functional .gitignore file • Composer-patches installed and set up • Composer set up to respect Drupal directory structure • Creates default writable versions of settings.php, services.yml and sites/ default/files directory • Installs drush and drupal console locally
  53. Drush Make or Composer

  54. Deployment Config

  55. None
  56. hook_update_N() Database mymodule.install in dev Git mymodule.install in live Database

  57. Features Database Feature
 in dev Git Feature
 in live Database

  58. It’s not all lollipops and unicorns! • Auto-increment indexes in

    Drupal 7 • Crazy terminology: recreate, revert? • A feature is just a module, let’s build some custom logic in it! • Circular dependency between features • Disable a feature, see what will happen… • WE DO NOT HAVE ANYTHING BETTER IN D7 :(
  59. Configuration Management (CMI*) * Obviously wrong acronym, previously used for

    Configuration Management Initiative, but got stuck.
  60. Database YAML files
 in dev Git YAML files
 in live

    Database drush cex drush cim
  61. Front-end (Fastest growing source of complexity) ?

  62. The need to compile the front end • Growing number

    of tools, including CSS compiling, JavaScript compiling, asset management, image optimisation etc. • Setting up local development stacks and keeping the toolset same across the large team can be challenging. • Should front end be compiled locally and then committed, or vice versa?
  63. The final goal is automation

  64. drush dl admin_toolbar;
 drush en admin_toolbar drush make;
 drush features-revert-all

    composer install;
 drush config-import
  65. A really simple D7 deploy script #!/usr/bin/env bash cd /var/www

    drush -y make project.make drush -y updatedb drush cc all drush -y features-revert-all grunt build
  66. A really simple D8 deploy script #!/usr/bin/env bash cd /var/www

    composer install drush -y updatedb drush cache-rebuild drush -y config-import grunt build
  67. #!/usr/bin/env bash GIT_WORK_TREE=/var/www git checkout -f cd /var/www composer install

    drush -y updatedb drush cache-rebuild drush -y config-import grunt build git push Local Web host A simple deployment workflow
  68. git push Local Web host push event Let’s make it

    complicated cd /var/www composer install drush -y updatedb drush cache-rebuild drush -y config-import grunt build
  69. Let’s make it even more complicated git push Local Test

    host
 /var/www/test push event Live host
 /var/www/live tag push
  70. OMG, security made it even more complicated! git push Local

    push event tag push
  71. Things to consider • Site instability during builds • Matter

    of security • Matter of flexibility • Matter of scaling
  72. Git push Assemble container Run build hook Stop serving requests

    Replace container Run deploy hook Serve requests Site instability during builds Serve requests from the old container until the new one was built.
  73. Matter of security Strong isolation of services on network, read

    only file system for PHP files, regular updates, minimum OS, single point of password access
  74. Matter of flexibility A dedicated working environment for each git

    branch. CLI and UI.
  75. Matter of scaling Every environment is a cluster. Commit YAML

    files to git repository to add or remove services, or use web UI to request more resources
  76. Well… That Escalated Quickly

  77. Thank you. branislav@platform.sh @bbujisic http://drupal.org/u/bbujisic