Deployment without the Hassle @ IPC 2016

The new feature is finally ready! All that’s left now is getting it to production. But who will have to get up at 3 am this time? Hopefully a downtime of one hour will be enough, but if anything goes wrong it might even take us two to get everything working again. If that sounds familiar, we have good news for you: it doesn’t have to be this way! With the right tools at your hand automating any kind of deployment will be as easy as pushing a button. No matter if you want to update your linux distribution, roll out your own application or make configuration changes – your operation system’s package management will empower you to manage even the most complex environment with ease.

Transcript

1. DEPLOYMENT WITHOUT THE HASSLE
2. Arne Blankerts Principal Consultant & Co-Founder thePHP.cc Sebastian Heuer Developer Advocate die kartenmacherei
3. OUR FEATURE IS DONE, LET'S DEPLOY!
6. [dave@ip-172-31-4-54 ~]$ cd /srv/application/ $ ssh -A dave@example.com [dave@ip-172-31-4-54 application]$ git pull [dave@ip-172-31-4-54 application]$ composer install From bitbucket.org:example.com/application 3c068b7..1560b43 develop -> origin/develop Updating 3c068b7..1560b43 composer.json | 12 + composer.lock | 22 ++ src/Application.php | 6 + Loading composer repositories with package information Installing dependencies (including require-dev) from lock file Your requirements could not be resolved to an installable set of packages.
7. COMPOSER INSTALL FAILS
8. BOB STARTS SHOUTING AT DAVE BECAUSE THE WEBSITE IS BROKEN
9. LET'S DEPLOY INTO A SEPARATE DIRECTORY AND USE A SYMLINK!
10. $ ssh -A dave@example.com [dave@ip-172-31-4-54 ~]$ git clone git@bitbucket.org:/example.com/application /srv/application-1.23.0/ [dave@ip-172-31-4-54 ~]$ composer install Cloning into 'application—1.23.0' remote: Counting objects: 644, done. remote: Compressing objects: 100% (34/34), done. remote: Total 644 (delta 9), reused 0 (delta 0), pack-reused 610 Receiving objects: 100% (644/644), 85.50 KiB | 0 bytes/s, done. Resolving deltas: 100% (369/369), done. Checking connectivity... done. Loading composer repositories with package information Updating dependencies (including require-dev) Nothing to install or update Writing lock file Generating autoload files [dave@ip-172-31-4-54 ~]$ cd /srv/application-1.23.0/
11. DEPLOYMENT IS FINISHED, SYMLINK IS UPDATED [dave@ip-172-31-4-54 ~]$ ln -sfn /srv/application-1.23.0 /srv/application [dave@ip-172-31-4-54 ~]$ ls -l /srv/ -rw-r—-r—- 1 dave dave 4096 Oct 05 12:37 application-1.22.3 -rw-r—-r—- 1 dave dave 4096 Oct 21 16:49 application-1.23.0 -rw-r—-r—- 1 dave dave 4096 Oct 21 16:49 application -> /srv/application-1.23.0
12. WEBSITE BEHAVES WEIRD, NOT REPRODUCIBLE ON A DEV MACHINE, MAGICALLY FIXES ITSELF AFTER A COUPLE OF MINUTES
13. REASON: REALPATH CACHE
14. –Rasmus Lerdorf 
 http://www.serverphorums.com/read.php?7,1233612 “If you are flipping a symlink on a live server which can happen in the middle of a request then your deploy mechanism is broken.”
15. SOMEBODY ELSE SHOULD DECIDE WHICH VERSION TO USE (WITHOUT THE NEED OF RESTARTING SERVICES OR INVALIDATING CACHES)
16. HTTPD APP V1 APP V2 PHP active version? v1
17. active version? PHP HTTPD APP V1 APP V2 v2
18. INSTALLATION AND ACTIVATION ARE DECOUPLED
19. APP V1 APP V2 active version? v2 PHP HTTPD
20. <?php $redis = new Redis; $redis->connect('127.0.0.1', 6379); $redis->select(0); $runWithVersion = $redis->get('version'); require '/var/www/php/application/' . $runWithVersion . '/bootstrap.php';
21. ACTIVATION ONLY AFTER INSTALLATION SUCCEEDED ON ALL SERVERS
22. NEXT DEPLOYMENT
23. WEB SERVER IS POINTED TO NEW VERSION
24. EXISTING SESSIONS BREAK
25. EXISTING SESSIONS SHOULD STICK TO THEIR INITIAL VERSION
26. APP V1 APP V2 existing session? v1 PHP HTTPD
27. APP V1 APP V2 active version? v2 PHP HTTPD
28. <?php $keys = array(); if (isset($_COOKIE['PHPSESSID'])) { $keys[] = $_COOKIE['PHPSESSID'] . '_version'; } $keys[] = 'default_version'; $redis = new Redis; $redis->connect('127.0.0.1', 6379); $redis->select(0); $runWithVersion = array_filter($redis->mGet($keys)); require '/var/www/php/application/' . $runWithVersion[0] . '/bootstrap.php';
29. NEXT DEPLOYMENT
30. WEB SERVER IS POINTED TO NEW VERSION
31. WEBSITE BEHAVES WEIRD, NOT REPRODUCIBLE ON A DEV MACHINE
32. REASON: COMPOSER HAS DEV-MASTER DEPENDENCIES, INSTALLED DIFFERENT VERSION
33. WE SHOULD NOT RUN BUILD PROCESSES ON THE TARGET SERVERS
34. WE NEED A BUILD SERVER
35. OUR CI SERVER CAN DO THE BUILDS!
36. HOW DO WE PUT THE BUILD RESULT ON OUR SERVERS?
37. WE USE RSYNC
38. THE NEW VERSION NEEDS A PHP EXTENSION WE DON'T HAVE ON OUR SERVERS YET
40. DEPLOYMENT IS BLOCKED, DEV WAITS FOR OPS
41. DEV: WE WANT TO BE ABLE TO MAKE INFRASTRUCTURE CHANGES OURSELVES
42. OPS GOES ON STRIKE
44. WHO IS RESPONSIBLE FOR WHAT?
45. OPS: MACHINES OS BASE CONFIGURATION
46. DEV: APPLICATION APPLICATION-SPECIFIC CONFIGURATION
47. OPS PROVIDES OS PACKAGES (BY ALLOWING US TO USE THE DISTRIBUTION'S REPOSITORY OR PROVIDING THEIR OWN)
48. ALL ENVIRONMENTS CAN USE THE SAME PACKAGE SOURCE
49. DEV IS ENABLED TO DEFINE REQUIRED TRUSTED PACKAGES SO THEY CAN BE INSTALLED DURING DEPLOYMENT
50. $ composer install Loading composer repositories with package information Updating dependencies (including require-dev) Your requirements could not be resolved to an installable set of packages. Problem 1 - The requested PHP extension ext-imagick * is missing from your system.
51. DON'T INVENT CUSTOM STUFF, THERE IS SOMETHING YOU CAN USE OUT OF THE BOX
52. OS PACKAGES ALLOW DEFINING DEPENDENT PACKAGES ALREADY
53. WE SHOULD PACKAGE OUR APPLICATION
54. HOW TO BUILD OS PACKAGES
55. .RPM
56. example-package-1.23.0/ build/ application/ src/ domain/ http/ Application.php vendor/ autoload.php web/ index.php application.spec Summary: example.com's e-commerce store frontend Name: example-package Version: 1.23.0 Release: vendor.1 Provides: example-package-%{version}-%{release} Requires: php-fpm >=7.0.11, example-library = 1.23.0 BuildRoot: %{_tmppath}/%{name}-%{version}-\ %{release}-root-%(%{__id_u} -n) BuildArch: noarch %description example.com's e-commerce store frontend PHP layer
57. example-package-1.23.0/ build/ application/ src/ domain/ http/ Application.php vendor/ autoload.php web/ index.php application.spec %install [ "$RPM_BUILD_ROOT" != "/" ] && rm -rf $RPM_BUILD_ROOT install -m 755 -d $RPM_BUILD_ROOT/srv/application cp -R build/application/* \ $RPM_BUILD_ROOT/srv/application %files %defattr(-,root,root) %dir /srv/application /srv/application/* %post %preun
58. $ rpmbuild -bb example-package-1.23.0/application.spec $ cd ~/projects/ Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.AZ7B61 […] Wrote: /home/dave/rpmbuild/RPMS/noarch/example-\ package-1.23.0-vendor.1.noarch.rpm + exit 0
59. .DEB
60. example-package-1.23.0/ DEBIAN/ control postinst preremove srv/ application/ src/ domain/ http/ Application.php vendor/ autoload.php web/ index.php Package: example-application-1.23.0 Version: 1 Maintainer: Dave Description: example.com's e-commerce store frontend Section: all Priority: optional Depends: php7-fpm (>=7.0.11), example-library-1.23.0 Architecture: all
61. $ fakeroot dpkg-deb —-build example-package-1.23.0/ dpkg-deb: building package 'example-application-1.23.0' in 'example-package-1.23.0.deb'. $ ls -l -rw-r—-r—- 1 dave dave 4096 Oct 21 16:49 example-package-1.23.0 -rw-r—-r—- 1 dave dave 910 Oct 23 10:32 example-package-1.23.0.deb $ cd ~/projects/
62. YOU CAN BUILD VARIOUS PACKAGES OUT OF ONE SOURCE (LIKE STATIC CONTENT AND PHP APPLICATION)
63. BUT WAIT, THERE'S MORE!
64. TARGET SERVERS DON'T NEED BUILD TOOLS LIKE COMPOSER OR NPM
65. INSTALLATION HAPPENS IN A TRANSACTION
66. HOOKS (PREINSTALL, POSTINSTALL, PREREMOVE, …)
67. CONFIGURATION MANAGEMENT
68. app v1.rpm app-nginx-config.rpm app v2.rpm app-nginx-config-dev.rpm app-nginx-config-prod.rpm provides provides requires requires
69. MULTIPLE PACKAGES CAN NOT OWN THE SAME FILE
70. USE PUPPET & CO
71. WHAT ABOUT DATABASE MIGRATIONS?
72. YOU DON'T "MIGRATE" THE DATABASE
73. NO ALTER TABLE
74. APP V1 TABLE V1 TABLE V2 APP V2
75. AUTOMATE EVERYTHING, MANUAL TRIGGERS SHOULD BE A BUSINESS DECISION
76. IMMUTABLE SERVER
77. Q&A Arne Blankerts @thephpcc
 @arneblankerts Sebastian Heuer @techdotkam @belanur