The Road to Continuous Delivery Zendcon 2015

Transcript

1. The Road to Continuous Delivery For PHP Applications, from FTP to Cloud CI/CD
2. Matt Land Code Samples: github.com/matt-land Slides: speakerdeck.com/mattland/ linkedin.com/in/matthewland CERTIFIED A R C H I T E CT 2 https://joind.in/15575
3. Disclaimers • This is going to be dev-ops heavy. • Portions are going to be AWS centric. • Mucho material included. • High level on the cloud patterns. • Low level on the important bits.
4. Goals • Discuss where the gains are, and how to get there. • Demystify migrating with best practice. • Demystify deployment strategies.
5. From FTP to CI/CD *or from somewhere in between Intro To CI/CD/CD Project Changes Build Tools Continuous Integration Deployment Models
6. Large Deployments
7. Large Deployments from an Agile Perspective
8. Large Deployments from an Agile Perspective • Bigger means more new moving parts, which is riskier. • Bigger means we spent more time working on it, meaning the time until delivery was longer. • Agile values trying lots of things, some of which are not going to work. • Agile values delivering things quickly. • Small & Frequent > Large & Infrequent
9. Background: Continuous Integration (CI) Shared source code control (git/svn) Maybe partial build process Automated tests with some coverage Deployment is still done manually Offers: Immediate feedback to merge issues Immediate feedback for test failures Testing contract is established Smaller, frequent low risk deployments minimum ~ several stories
10. Background: Continuous Delivery (CD) Continuous Integration plus: On commit, deployment is triggered to a ‘production like’ environment. Automated tests runs in that ‘production like’ environment. Features are immediately delivered back to stakeholders for feedback. Deployment is still done semi-manually. Offers: Faster feedback cycles. Time in transit for stories can drop. Minimizes risk of deployment errors or code that didn’t pass. CAN RELEASE RELIABLY AT ANY TIME minimum ~ one story
11. Background: Continuous Deployment (CD) Continuous Delivery plus: RELEASE TO PROD IS AUTOMATIC √ ON ACCEPTANCE √ ON TEST SUITE PASSING Offers: Deployment several times per hour. Smallest deployments, lowest risk. Potential User Fatigue. Feature Toggles. minimum ~ one line of code
12. Continuous Something…
13. Continuous Something…
14. Benefits of full transition to CI/CD High degree of confidence everything works, every deploy Ability to release as frequently as appropriate Shorter time story life from request to deployment Better security: managed libraries get updates more frequently Easy for anyone to add test coverage / run tests No tacit knowledge needed by team members to develop or deploy No more ‘prod credential holders’ needed on the team
15. Benefits of partial transition to CI/CD App becomes easier and easier to maintain. Deploys become less and less scary. Professional development in executing best practices. Try new techniques on current projects. Possible to undo prior mistakes or the sins of others. Never have to write another autoloader. Never fix a bug more than once. (Regression tests are awesome.)
16. The Human Part Agile, Product Managers, Scrum Masters, Backlogs, Ceremonies. CI/CD were developed to service the needs of these business behaviors and roles. The Agile umbrella is not required for CI/CD. Besides, its much harder to change how an organization works. The software part is comparatively easy. Do that part.
17. Downsides ‘Easier’ does not mean ‘not hard’. The full end to end solution takes a long time to build. Test suites take a long time to mature. Learning to write tests well is hard. Small projects can be converted in a day. Big projects may never completely convert.
18. Project Changes A skill dependency tree for converting a project Each improvement has immediate individual value too. (Agile)
19. Project Changes: Dependency Managers PHP: Composer, has composer.json JS: NPM, has package.json JS build tool: • Bower, .bowerrc or bower.json, can also run composer • Grunt, gruntfile.coffee or gulpfile.js • Gulp, gulpfile.js or gulpfile.coffee Pick your favorite. Covered again in build tools.
20. Project Changes: PHP Dependencies Most Front Controller Pattern Apps: index.php require_once __DIR__.’/src/vendor/autoloader.php’ PHPUnit: phpunit.xml contains bootstrap="vendor/bootstrap.php" Track composer.json, composer.phar and composer.lock. Add `vendor` to .gitignore. Developers call “composer update” , CI calls “Composer Install”
21. Project Changes: PHP Dependencies Load composer’s autoload.php in all app entry points. Move external libraries/dependencies into composer and test. Unlink old copies and remove old “require/include” statements. Remove any symlinks. Remove PEAR dependencies.* Turn your shared libraries into composer packages. Composer legacy libraries into your apps. Update your old autoloaders to stop offering to load legacy libraries. Remove all contents of /vendor from VCS, append /vendor to .gitignore. Update CI config to run `composer install` before deployment, add auth token.
22. Project Changes: JS Dependencies Update all third party JS to be managed by the JS manager. Update layouts to use the managed package versions. Split distribution files from site specific files. Delete all the crud. Add `node_modules` to .gitignore. Later in CI, use “npm-cache clean” and “npm-cache install npm”
23. Project Changes: Namespacing Optional: • Namespacing. But make sure to use it when creating the tests. • Case sensitivity. Fixing class name => filename and path name => namespace conventions can be worked around. Required: • Maximum one file per class.* (need to verify) • Each class across a library must have a unique name. • Remove all the include*/require* statements.
24. Project Changes: Require Optional: • Standalone scripts: update each file • Front Controller Pattern: Single app entry point across the application, usually an index.php • The only require/include statement across the application is a single “require_once ‘vendor/autoload.php” if possible. Required:* • Use Relative Paths for any paths for File IO or Require statements. Ex: __DIR__ . ’/../autoload.php’, __FILE__ . ‘/../../error.log’ • Note a leading slash in the string after PHP special constants.
25. Project Changes: Understanding Autoloading 1. Register my autoloader function(s) when my app instantiates 2. PHP needs to create an object or use a method in some class 3. PHP begins to instantiate the class 4. PHP checks get_declared_classes() or its C equivalent 5. PHP checks spl_autoload_functions() for any registered autoloaders 6. PHP asks each autoloaders to service the request, in order of the autoloader stack 7. PHP is able to instantiate the class or throws a Fatal error
26. Project Changes: Normal Autoloader Example Example without composer for class \Foo\Bar is in /src/Foo/Bar.php spl_autoload_register(function ($className) { if (strncmp($className, ‘Foo', 3) !== 0) { //DO NOT BE GREEDY return; //return early, return often } require_once(__DIR__.’/src/’.str_replace(“\\”, “/”, $className).’.php’; });
27. Project Changes: Using Classmap Autoloader Classmap is one of lesser known ways to specify a namespace in composer. Excellent for consuming legacy libraries that are ‘broken’. Use it when: • Class are snowflakes instead of class name => file name pairs. • The code base has case sensitivity inconsistencies. • The app doesn’t follow front controller pattern. • Renaming would break other dependent applications. (MVCS example) • Too many source files to fix. • Cannot modify source class files.
28. Project Changes: Classmap AL Example Let composer scan classes into /vendor/autoload_classmap.php { "autoload": { "classmap": ["path/to/my/src/"] } } Note: For new files, call `composer update` to recognize.
29. Project Changes: Packages From Repos composer.json for the library { "name": "company/package", "license": "proprietary", "type": "library" } composer.json for the app { "repositories": [{ "type": "vcs", "url":"ssh://git@github.com/my/project.git" }], “require”: { "company/package": "dev-master" }, "autoload": { "classmap": ["vendor/company/package"] }
30. Project Changes: Packages From Repos You still need to be able to modify & test the code being managed by composer as a dependency, but maintained by your team. Not recommended: You can modify files live in vendor, and push changes upstream. But composer cannot update libs when there are mods, and “require-dev” was not installed in /vendor, so testing is not possible. Recommended: (package swap) Move <lib> to <lib>2, symlink and tell IDE to ignore <lib>2. Add an ant target or shell script to swap the symlink and editable copy on demand.
31. Testing Frameworks: PHP • Add phpunit to `require-dev` section of composer.json • Add a testing namespace to composer.json • Create a test that uses the namespace • Add the following <?xml version="1.0" encoding="UTF-8"?> <phpunit colors="true" verbose="true" bootstrap="tests/bootstrap.php"> <testsuites> <testsuite name="My_Suite"> <directory>tests</directory> </testsuite> </testsuites> <logging> <log type="junit" target="junit.xml" logIncompleteSkipped="false"/> </logging> </phpunit> call “./vendor/bin/phpunit”
32. Testing Frameworks: JS • Add testing deps to package.json (mocha, mocha-junit-reporter) • Create a js test • Add the following to the JS build tool (gulpfile.js) var mocha = require('gulp-mocha') var gulp = require('gulp') gulp.task('test', function() { return gulp.src(['intro-to-testing/*.js']).pipe(mocha({ reporter: 'mocha-junit-reporter', reporterOptions: { mochaFile: './junit.xml' }, })); }); call “gulp test”
33. Project Changes: New Files
34. Project Changes: New Files Files covered so far:
35. Project Changes: New Files Test suite output used by CI. Alternate test suite output, in JSON format. Add both of these to .gitignore:
36. Project Changes: Composer CI • Identical versions of composer and all dependencies from local to CI • Check in `composer.phar` • Check in `composer.lock` • Add vendor to .gitignore • CI should always call ‘composer install’ • Deployment to prod does not run composer
37. Project Changes: New Files Additional Composer files that get checked in
38. Project Changes: Composer Auth • Anonymous will go over the API limit on github.com to retrieve just ZF2 libraries • Author user API limit is much higher • A ‘Personal Access Token’ allows composer to auth • Add a ‘personal access token’ for each CI project composer config github-oauth '{"github.com":"<token>"}'
39. Getting a personal access token from github.com
40. Project Changes: New Files Auth file that helps composer not exceed api limits
41. Build Tools
42. Build Tools Build tools automate tasks in a repeatable way. Examples: • Building Angular, Ember, Grunt, Gulp, SASS, Compass • Launching VMs and services (apache, mysql, selenium) • Launching tests and running against different environments (target servers) • Reloading development datasets and schemas • Rsync/scp files to our production servers
43. Build Tools ant - every CI will have docs for ant maven make gradle The tool doesn’t matter that much. Whichever makes you most effective (kind of the whole point). Tool chains can be mix-and-match for V1, but don’t use too many. Too many scripts = spaghetti deployments and black magic.
44. Build Tools: Apache Ant Build scripts are xml based. Variables and conditional logic are supported. Low barrier to entry, lots of documentation. A build.xml file is composed of targets. Settings can be put in a separate build.properties file. Targets can call other targets or targets in other build scripts. Targets can be executing on remote servers or local. Plays nice with CI servers.
45. Build Tools: Sample build.xml for ant <?xml version="1.0" encoding="UTF-8"?> <project name="demo" default="test" basedir="." > <target name="composer"> <exec dir="${basedir}" executable="./composer.phar" failonerror="true"> <arg line="install" /> </exec> </target> <target name="phpunit" description="Run unit tests with PHPUnit"> <exec dir="${basedir}" executable="./vendor/bin/phpunit" failonerror="true"> <arg line="--stderr --tap" /> </exec> </target> <target name="test" depends="composer,phpunit" /> </project> call “ant”
46. Build Tools: Sample Target <target name="mysql-bare-up" > <sshexec trust="true" host="${ssh.host}" port="${ssh.port}" username="${ssh.username}" keyfile="${ssh.key}" command="sudo service mysql start" failonerror="true"/> <property name="mysql.running" value="true"/> </target>
47. Project Changes: New Files The build files in the project.
48. Build Tools: Return Codes in Scripts PHP logic: True: [9, -4, 22.3, true, 1, ….]; False: [null, false, 0, ...]; OS return code logic: Success: $? == 0; Failure: $? == anything but zero `$?` is the last return code A php script succeeded if the script calls exit(0); or exit(); A php script should fail with: exit(1); or exit(‘anything’); An uncaught exception will also a fail. The exception code will be the return code.
49. Continuous Integration
50. Continuous Integration: Tasks Check out the newest commit (or branch and commit) Runs the build scripts Build script calls dependency managers Build script calls test suites Evaluates return codes at each step Evaluates test result files (junit.xml) Radiates success or failure Deploys code to production Bold is minimum req for CI. The rest indicate CD.
51. Continuous Integration: CI Platforms Travis CI - cloud based, simple and easy, integrates with Github, badges, $$$, good first party docs Jenkins/Hudson CI - self hosed WAR, open source, harder setup, messy UI, lots of stackoverflow docs. SaaS offerings available. Bamboo - self hosted or cloud, works with JIRA/BitBucket/ Hipchat. Good APIs Team City - self hosted. Jetbrains offering. Mentioned because we like PHPStorm Plenty of fish to choose from...
52. Continuous Integration: Hooks/Events There are different events in the life of a build. Different events should happen at different times. Each can have its own script or ant target. Common events: pre-deploy: checkout latest code, run dependency managers, compile assets, inject db credentials, and hostname deploy: copy the code to the production like target post-deploy: server specific stuff, kick apache, restart memcache Test suite launch happens after post-deploy. prod-post-deploy: happens only on prod, usually to inject prod credential
53. Project Changes: New Files All the ‘hooks’ files.
54. Project Changes: New Files Apache virtual host file. Config goes with the code onto any server.
55. Continuous Integration: Misc Inject config to test suites with environment variables. PHPUnit ::setup() should read them and setup selenium grid connections and baseurl. Selenium… docker : hostnames, dbs, build numbers. Auth.json for composer
56. Deployment Models
57. Deployment MVP • Automated • Maintains or can retrieve a list of environments • Connected to VCS • Can accomplish per target machine: • rsync (and tag with the commit) • advancement of symlink pointers • revert (reset symlink pointers) • verify actions completed successfully • Optional: All team members can self-serve Check out AWS PHP SDK.
58. Deployment V1: Pets
59. Deployment V1: Pets
60. Deployment V1: Pets
61. Deployment: Cows Are Not Pets Servers are ephemeral. They can and fail. Stop upgrading. Replace. If your servers are pets, think about saying goodbye. Move away from setup and over to provisioning. Provision from an AMI, apply a saltstack profile, deploy code.
62. Deployment V2: Cows
63. Deployment V2: Cows
64. Deployment V2: Cows
65. Deployment V2: Cows
66. Deployment V2: Cows
67. Deployment V2: Cows
68. Deployment V3: Containers • Solves the “Works local, not in prod” problem. • Container per service. • Extremely fast to provision. • Lower footprint than our Vagrant mono-container. • Developers check in a container. • CI tests run against the container, not the code. • A container is ‘promoted’ to production, vs code ‘deploy’.
69. Deployment V3: Testing Containers
70. Deployment V3: Deploying Containers
71. Deployment next: Serverless • Extremely low cloud operating cost: first 1 million free per month, $.20 per million requests • No servers or infrastructure to manage • Code has access to S3, RDS, dynamodb, Redshift • Can be triggered event or called by service aggregator • HTTP endpoints • Supports Node JS, Java 8, Python
72. Deployment next: Serverless Apache / Nginx Traffic ELB Group Service Response Aggregator Lambda Function Microservices RDS Redshift S3 f(n) f(n) f(n) f(n)
73. Deployment next: Serverless Apache / Nginx Traffic ELB Group Service Response Aggregator Lambda Function Microservices RDS Redshift S3 f(n) f(n) f(n) f(n) f(n) New version deployed, and replicates
74. Deployment next: Serverless Apache / Nginx Traffic ELB Group Service Response Aggregator Lambda Function Microservices RDS Redshift S3 f(n) f(n) f(n) f(n) f(n) Service Aggregator pointer moves
75. Deployment next: Serverless Apache / Nginx Traffic ELB Group Service Response Aggregator Lambda Function Microservices RDS Redshift S3 f(n) f(n) f(n) f(n) Old version drops from lambda platform
76. The Road to Continuous Delivery Matt Land Code Samples: github.com/matt-land Slides: speakerdeck.com/mattland/ linkedin.com/in/matthewland CERTIFIED A R C H I T E CT 2 https://joind.in/15575
77. How Much Testing
78. Test Anything Protocol
79. Fatigue Risk
80. The Four Kinds of Customer • B2A (to API): interacts several times per minute. Zero or Infinite fatigue risk. • B2B (to Business): interacts several times per month. Easily fatigued. • B2C (to Customer): interacts for one session per year. Zero fatigue risk. • B2D (to Department): interacts several times per day, high savvy. Variable fatigue risk.
81. Breaking the DevOps Dependency • We tried Embedded DevOps, and plan to try again • Currently high Self Serve, 1 devops per 10 engineers • Devs are snowflakes. Some will never have devops needs, some will need stuff every day. • Hip chat bot for us
83. Self Updating composer.par as part of ant
84. Continuous Integration: Try CI Lab (2 hours) • Create a greenfield project • Add composer.phar to the project • Composer init, add phpunit as a dependency • Create two namespaces under /src & /test in composer.json • Write some tests and create a phpunit.xml in / • Install a build tool and target that invokes phpunit in /vendor, and outputs using TAP • Push the project to github as a public project https://help.github.com/articles/create-a-repo/ • Connect it to a commercial CI server (Travis CI) • Add a .travis.yml file http://docs.travis-ci.com/user/languages/php/ • Fix config until the build passes • Add a README.md with build passing badges • Compare with https://github.com/matt-land/test-samples if stuck