The Road to Continuous Delivery Zendcon 2015

Slide 1

Slide 1 text

The Road to Continuous Delivery For PHP Applications, from FTP to Cloud CI/CD

Slide 2

Slide 2 text

Matt Land Code Samples: github.com/matt-land Slides: speakerdeck.com/mattland/ linkedin.com/in/matthewland CERTIFIED A R C H I T E CT 2 https://joind.in/15575

Slide 3

Slide 3 text

Disclaimers • This is going to be dev-ops heavy. • Portions are going to be AWS centric. • Mucho material included. • High level on the cloud patterns. • Low level on the important bits.

Slide 4

Slide 4 text

Goals • Discuss where the gains are, and how to get there. • Demystify migrating with best practice. • Demystify deployment strategies.

Slide 5

Slide 5 text

From FTP to CI/CD *or from somewhere in between Intro To CI/CD/CD Project Changes Build Tools Continuous Integration Deployment Models

Slide 6

Slide 6 text

Large Deployments

Slide 7

Slide 7 text

Large Deployments from an Agile Perspective

Slide 8

Slide 8 text

Large Deployments from an Agile Perspective • Bigger means more new moving parts, which is riskier. • Bigger means we spent more time working on it, meaning the time until delivery was longer. • Agile values trying lots of things, some of which are not going to work. • Agile values delivering things quickly. • Small & Frequent > Large & Infrequent

Slide 9

Slide 9 text

Background: Continuous Integration (CI) Shared source code control (git/svn) Maybe partial build process Automated tests with some coverage Deployment is still done manually Offers: Immediate feedback to merge issues Immediate feedback for test failures Testing contract is established Smaller, frequent low risk deployments minimum ~ several stories

Slide 10

Slide 10 text

Background: Continuous Delivery (CD) Continuous Integration plus: On commit, deployment is triggered to a ‘production like’ environment. Automated tests runs in that ‘production like’ environment. Features are immediately delivered back to stakeholders for feedback. Deployment is still done semi-manually. Offers: Faster feedback cycles. Time in transit for stories can drop. Minimizes risk of deployment errors or code that didn’t pass. CAN RELEASE RELIABLY AT ANY TIME minimum ~ one story

Slide 11

Slide 11 text

Background: Continuous Deployment (CD) Continuous Delivery plus: RELEASE TO PROD IS AUTOMATIC √ ON ACCEPTANCE √ ON TEST SUITE PASSING Offers: Deployment several times per hour. Smallest deployments, lowest risk. Potential User Fatigue. Feature Toggles. minimum ~ one line of code

Slide 12

Slide 12 text

Continuous Something…

Slide 13

Slide 13 text

Continuous Something…

Slide 14

Slide 14 text

Beneﬁts of full transition to CI/CD High degree of conﬁdence everything works, every deploy Ability to release as frequently as appropriate Shorter time story life from request to deployment Better security: managed libraries get updates more frequently Easy for anyone to add test coverage / run tests No tacit knowledge needed by team members to develop or deploy No more ‘prod credential holders’ needed on the team

Slide 15

Slide 15 text

Beneﬁts of partial transition to CI/CD App becomes easier and easier to maintain. Deploys become less and less scary. Professional development in executing best practices. Try new techniques on current projects. Possible to undo prior mistakes or the sins of others. Never have to write another autoloader. Never ﬁx a bug more than once. (Regression tests are awesome.)

Slide 16

Slide 16 text

The Human Part Agile, Product Managers, Scrum Masters, Backlogs, Ceremonies. CI/CD were developed to service the needs of these business behaviors and roles. The Agile umbrella is not required for CI/CD. Besides, its much harder to change how an organization works. The software part is comparatively easy. Do that part.

Slide 17

Slide 17 text

Downsides ‘Easier’ does not mean ‘not hard’. The full end to end solution takes a long time to build. Test suites take a long time to mature. Learning to write tests well is hard. Small projects can be converted in a day. Big projects may never completely convert.

Slide 18

Slide 18 text

Project Changes A skill dependency tree for converting a project Each improvement has immediate individual value too. (Agile)

Slide 19

Slide 19 text

Project Changes: Dependency Managers PHP: Composer, has composer.json JS: NPM, has package.json JS build tool: • Bower, .bowerrc or bower.json, can also run composer • Grunt, gruntfile.coffee or gulpfile.js • Gulp, gulpfile.js or gulpfile.coffee Pick your favorite. Covered again in build tools.

Slide 20

Slide 20 text

Project Changes: PHP Dependencies Most Front Controller Pattern Apps: index.php require_once __DIR__.’/src/vendor/autoloader.php’ PHPUnit: phpunit.xml contains bootstrap="vendor/bootstrap.php" Track composer.json, composer.phar and composer.lock. Add `vendor` to .gitignore. Developers call “composer update” , CI calls “Composer Install”

Slide 21

Slide 21 text

Project Changes: PHP Dependencies Load composer’s autoload.php in all app entry points. Move external libraries/dependencies into composer and test. Unlink old copies and remove old “require/include” statements. Remove any symlinks. Remove PEAR dependencies.* Turn your shared libraries into composer packages. Composer legacy libraries into your apps. Update your old autoloaders to stop offering to load legacy libraries. Remove all contents of /vendor from VCS, append /vendor to .gitignore. Update CI conﬁg to run `composer install` before deployment, add auth token.

Slide 22

Slide 22 text

Project Changes: JS Dependencies Update all third party JS to be managed by the JS manager. Update layouts to use the managed package versions. Split distribution files from site specific files. Delete all the crud. Add `node_modules` to .gitignore. Later in CI, use “npm-cache clean” and “npm-cache install npm”

Slide 23

Slide 23 text

Project Changes: Namespacing Optional: • Namespacing. But make sure to use it when creating the tests. • Case sensitivity. Fixing class name => ﬁlename and path name => namespace conventions can be worked around. Required: • Maximum one ﬁle per class.* (need to verify) • Each class across a library must have a unique name. • Remove all the include*/require* statements.

Slide 24

Slide 24 text

Project Changes: Require Optional: • Standalone scripts: update each ﬁle • Front Controller Pattern: Single app entry point across the application, usually an index.php • The only require/include statement across the application is a single “require_once ‘vendor/autoload.php” if possible. Required:* • Use Relative Paths for any paths for File IO or Require statements. Ex: __DIR__ . ’/../autoload.php’, __FILE__ . ‘/../../error.log’ • Note a leading slash in the string after PHP special constants.

Slide 25

Slide 25 text

Project Changes: Understanding Autoloading 1. Register my autoloader function(s) when my app instantiates 2. PHP needs to create an object or use a method in some class 3. PHP begins to instantiate the class 4. PHP checks get_declared_classes() or its C equivalent 5. PHP checks spl_autoload_functions() for any registered autoloaders 6. PHP asks each autoloaders to service the request, in order of the autoloader stack 7. PHP is able to instantiate the class or throws a Fatal error

Slide 26

Slide 26 text

Project Changes: Normal Autoloader Example Example without composer for class \Foo\Bar is in /src/Foo/Bar.php spl_autoload_register(function ($className) { if (strncmp($className, ‘Foo', 3) !== 0) { //DO NOT BE GREEDY return; //return early, return often } require_once(__DIR__.’/src/’.str_replace(“\\”, “/”, $className).’.php’; });

Slide 27

Slide 27 text

Project Changes: Using Classmap Autoloader Classmap is one of lesser known ways to specify a namespace in composer. Excellent for consuming legacy libraries that are ‘broken’. Use it when: • Class are snowflakes instead of class name => file name pairs. • The code base has case sensitivity inconsistencies. • The app doesn’t follow front controller pattern. • Renaming would break other dependent applications. (MVCS example) • Too many source files to fix. • Cannot modify source class files.

Slide 28

Slide 28 text

Project Changes: Classmap AL Example Let composer scan classes into /vendor/autoload_classmap.php { "autoload": { "classmap": ["path/to/my/src/"] } } Note: For new ﬁles, call `composer update` to recognize.

Slide 29

Slide 29 text

Project Changes: Packages From Repos composer.json for the library { "name": "company/package", "license": "proprietary", "type": "library" } composer.json for the app { "repositories": [{ "type": "vcs", "url":"ssh://[email protected]/my/project.git" }], “require”: { "company/package": "dev-master" }, "autoload": { "classmap": ["vendor/company/package"] }

Slide 30

Slide 30 text

Project Changes: Packages From Repos You still need to be able to modify & test the code being managed by composer as a dependency, but maintained by your team. Not recommended: You can modify ﬁles live in vendor, and push changes upstream. But composer cannot update libs when there are mods, and “require-dev” was not installed in /vendor, so testing is not possible. Recommended: (package swap) Move to 2, symlink and tell IDE to ignore 2. Add an ant target or shell script to swap the symlink and editable copy on demand.

Slide 31

Slide 31 text

Testing Frameworks: PHP • Add phpunit to `require-dev` section of composer.json • Add a testing namespace to composer.json • Create a test that uses the namespace • Add the following tests call “./vendor/bin/phpunit”

Slide 32

Slide 32 text

Testing Frameworks: JS • Add testing deps to package.json (mocha, mocha-junit-reporter) • Create a js test • Add the following to the JS build tool (gulpﬁle.js) var mocha = require('gulp-mocha') var gulp = require('gulp') gulp.task('test', function() { return gulp.src(['intro-to-testing/*.js']).pipe(mocha({ reporter: 'mocha-junit-reporter', reporterOptions: { mochaFile: './junit.xml' }, })); }); call “gulp test”

Slide 33

Slide 33 text

Project Changes: New Files

Slide 34

Slide 34 text

Project Changes: New Files Files covered so far:

Slide 35

Slide 35 text

Project Changes: New Files Test suite output used by CI. Alternate test suite output, in JSON format. Add both of these to .gitignore:

Slide 36

Slide 36 text

Project Changes: Composer CI • Identical versions of composer and all dependencies from local to CI • Check in `composer.phar` • Check in `composer.lock` • Add vendor to .gitignore • CI should always call ‘composer install’ • Deployment to prod does not run composer

Slide 37

Slide 37 text

Project Changes: New Files Additional Composer ﬁles that get checked in

Slide 38

Slide 38 text

Project Changes: Composer Auth • Anonymous will go over the API limit on github.com to retrieve just ZF2 libraries • Author user API limit is much higher • A ‘Personal Access Token’ allows composer to auth • Add a ‘personal access token’ for each CI project composer conﬁg github-oauth '{"github.com":""}'

Slide 39

Slide 39 text

Getting a personal access token from github.com

Slide 40

Slide 40 text

Project Changes: New Files Auth ﬁle that helps composer not exceed api limits

Slide 41

Slide 41 text

Build Tools

Slide 42

Slide 42 text

Build Tools Build tools automate tasks in a repeatable way. Examples: • Building Angular, Ember, Grunt, Gulp, SASS, Compass • Launching VMs and services (apache, mysql, selenium) • Launching tests and running against different environments (target servers) • Reloading development datasets and schemas • Rsync/scp ﬁles to our production servers

Slide 43

Slide 43 text

Build Tools ant - every CI will have docs for ant maven make gradle The tool doesn’t matter that much. Whichever makes you most effective (kind of the whole point). Tool chains can be mix-and-match for V1, but don’t use too many. Too many scripts = spaghetti deployments and black magic.

Slide 44

Slide 44 text

Build Tools: Apache Ant Build scripts are xml based. Variables and conditional logic are supported. Low barrier to entry, lots of documentation. A build.xml file is composed of targets. Settings can be put in a separate build.properties ﬁle. Targets can call other targets or targets in other build scripts. Targets can be executing on remote servers or local. Plays nice with CI servers.

Slide 45

Slide 45 text

Build Tools: Sample build.xml for ant call “ant”

Slide 46

Slide 46 text

Build Tools: Sample Target

Slide 47

Slide 47 text

Project Changes: New Files The build ﬁles in the project.

Slide 48

Slide 48 text

Build Tools: Return Codes in Scripts PHP logic: True: [9, -4, 22.3, true, 1, ….]; False: [null, false, 0, ...]; OS return code logic: Success: $? == 0; Failure: $? == anything but zero `$?` is the last return code A php script succeeded if the script calls exit(0); or exit(); A php script should fail with: exit(1); or exit(‘anything’); An uncaught exception will also a fail. The exception code will be the return code.

Slide 49

Slide 49 text

Continuous Integration

Slide 50

Slide 50 text

Continuous Integration: Tasks Check out the newest commit (or branch and commit) Runs the build scripts Build script calls dependency managers Build script calls test suites Evaluates return codes at each step Evaluates test result ﬁles (junit.xml) Radiates success or failure Deploys code to production Bold is minimum req for CI. The rest indicate CD.

Slide 51

Slide 51 text

Continuous Integration: CI Platforms Travis CI - cloud based, simple and easy, integrates with Github, badges, $$$, good first party docs Jenkins/Hudson CI - self hosed WAR, open source, harder setup, messy UI, lots of stackoverflow docs. SaaS offerings available. Bamboo - self hosted or cloud, works with JIRA/BitBucket/ Hipchat. Good APIs Team City - self hosted. Jetbrains offering. Mentioned because we like PHPStorm Plenty of fish to choose from...

Slide 52

Slide 52 text

Continuous Integration: Hooks/Events There are different events in the life of a build. Different events should happen at different times. Each can have its own script or ant target. Common events: pre-deploy: checkout latest code, run dependency managers, compile assets, inject db credentials, and hostname deploy: copy the code to the production like target post-deploy: server speciﬁc stuff, kick apache, restart memcache Test suite launch happens after post-deploy. prod-post-deploy: happens only on prod, usually to inject prod credential

Slide 53

Slide 53 text

Project Changes: New Files All the ‘hooks’ ﬁles.

Slide 54

Slide 54 text

Project Changes: New Files Apache virtual host ﬁle. Conﬁg goes with the code onto any server.

Slide 55

Slide 55 text

Continuous Integration: Misc Inject conﬁg to test suites with environment variables. PHPUnit ::setup() should read them and setup selenium grid connections and baseurl. Selenium… docker : hostnames, dbs, build numbers. Auth.json for composer

Slide 56

Slide 56 text

Deployment Models

Slide 57

Slide 57 text

Deployment MVP • Automated • Maintains or can retrieve a list of environments • Connected to VCS • Can accomplish per target machine: • rsync (and tag with the commit) • advancement of symlink pointers • revert (reset symlink pointers) • verify actions completed successfully • Optional: All team members can self-serve Check out AWS PHP SDK.

Slide 58

Slide 58 text

Deployment V1: Pets

Slide 59

Slide 59 text

Deployment V1: Pets

Slide 60

Slide 60 text

Deployment V1: Pets

Slide 61

Slide 61 text

Deployment: Cows Are Not Pets Servers are ephemeral. They can and fail. Stop upgrading. Replace. If your servers are pets, think about saying goodbye. Move away from setup and over to provisioning. Provision from an AMI, apply a saltstack proﬁle, deploy code.

Slide 62

Slide 62 text

Deployment V2: Cows

Slide 63

Slide 63 text

Deployment V2: Cows

Slide 64

Slide 64 text

Deployment V2: Cows

Slide 65

Slide 65 text

Deployment V2: Cows

Slide 66

Slide 66 text

Deployment V2: Cows

Slide 67

Slide 67 text

Deployment V2: Cows

Slide 68

Slide 68 text

Deployment V3: Containers • Solves the “Works local, not in prod” problem. • Container per service. • Extremely fast to provision. • Lower footprint than our Vagrant mono-container. • Developers check in a container. • CI tests run against the container, not the code. • A container is ‘promoted’ to production, vs code ‘deploy’.

Slide 69

Slide 69 text

Deployment V3: Testing Containers

Slide 70

Slide 70 text

Deployment V3: Deploying Containers

Slide 71

Slide 71 text

Deployment next: Serverless • Extremely low cloud operating cost: ﬁrst 1 million free per month, $.20 per million requests • No servers or infrastructure to manage • Code has access to S3, RDS, dynamodb, Redshift • Can be triggered event or called by service aggregator • HTTP endpoints • Supports Node JS, Java 8, Python

Slide 72

Slide 72 text

Deployment next: Serverless Apache / Nginx Trafﬁc ELB Group Service Response Aggregator Lambda Function Microservices RDS Redshift S3 f(n) f(n) f(n) f(n)

Slide 73

Slide 73 text

Deployment next: Serverless Apache / Nginx Trafﬁc ELB Group Service Response Aggregator Lambda Function Microservices RDS Redshift S3 f(n) f(n) f(n) f(n) f(n) New version deployed, and replicates

Slide 74

Slide 74 text

Deployment next: Serverless Apache / Nginx Trafﬁc ELB Group Service Response Aggregator Lambda Function Microservices RDS Redshift S3 f(n) f(n) f(n) f(n) f(n) Service Aggregator pointer moves

Slide 75

Slide 75 text

Deployment next: Serverless Apache / Nginx Trafﬁc ELB Group Service Response Aggregator Lambda Function Microservices RDS Redshift S3 f(n) f(n) f(n) f(n) Old version drops from lambda platform

Slide 76

Slide 76 text

The Road to Continuous Delivery Matt Land Code Samples: github.com/matt-land Slides: speakerdeck.com/mattland/ linkedin.com/in/matthewland CERTIFIED A R C H I T E CT 2 https://joind.in/15575

Slide 77

Slide 77 text

How Much Testing

Slide 78

Slide 78 text

Test Anything Protocol

Slide 79

Slide 79 text

Fatigue Risk

Slide 80

Slide 80 text

The Four Kinds of Customer • B2A (to API): interacts several times per minute. Zero or Inﬁnite fatigue risk. • B2B (to Business): interacts several times per month. Easily fatigued. • B2C (to Customer): interacts for one session per year. Zero fatigue risk. • B2D (to Department): interacts several times per day, high savvy. Variable fatigue risk.

Slide 81

Slide 81 text

Breaking the DevOps Dependency • We tried Embedded DevOps, and plan to try again • Currently high Self Serve, 1 devops per 10 engineers • Devs are snowﬂakes. Some will never have devops needs, some will need stuff every day. • Hip chat bot for us

Slide 82

Slide 82 text

No content

Slide 83

Slide 83 text

Self Updating composer.par as part of ant

Slide 84

Slide 84 text

Continuous Integration: Try CI Lab (2 hours) • Create a greenfield project • Add composer.phar to the project • Composer init, add phpunit as a dependency • Create two namespaces under /src & /test in composer.json • Write some tests and create a phpunit.xml in / • Install a build tool and target that invokes phpunit in /vendor, and outputs using TAP • Push the project to github as a public project https://help.github.com/articles/create-a-repo/ • Connect it to a commercial CI server (Travis CI) • Add a .travis.yml file http://docs.travis-ci.com/user/languages/php/ • Fix config until the build passes • Add a README.md with build passing badges • Compare with https://github.com/matt-land/test-samples if stuck