Automated Deployment with Phing

Given at ZendCon 2009

Transcript

1. AUTOMATED DEPLOYMENT WITH PHING Or: How I Will Replace You With A Very Small Shell Script
2. Who  Daniel Cousineau  The [Former] Hat Guy  Senior Software Applications Developer Texas A&M University Division of Student Affairs Department of IT  http://www.toosweettobesour.com/  @dcousineau Automated Deployment With Phing - Daniel Cousineau ZendCon '09 2
3. The Problem Automated Deployment With Phing - Daniel Cousineau ZendCon '09 3
4. Human beings make mistakes Automated Deployment With Phing - Daniel Cousineau ZendCon '09 4
5. We aren’t as accurate each repetition Automated Deployment With Phing - Daniel Cousineau ZendCon '09 5
6. Machines don’t have this problem Automated Deployment With Phing - Daniel Cousineau ZendCon '09 6
7. They do the same thing every time (supposedly) Automated Deployment With Phing - Daniel Cousineau ZendCon '09 7
8. At Any Given Time  New Code  New Configuration  New Database Schema  New Static Files Automated Deployment With Phing - Daniel Cousineau ZendCon '09 8
9. A Lot To Remember  Did you remember to upload ALL new files?  Did you remember to update your DB?  Did you remember to correct your config? Automated Deployment With Phing - Daniel Cousineau ZendCon '09 9
10. Even Worse  Did you clear your caches?  Did you delete that old file/plugin?  In the upload process, was your configuration overwritten?  Did you upload ALL the changed files?  When did you last upload? Automated Deployment With Phing - Daniel Cousineau ZendCon '09 10
11. The Solution Automated Deployment With Phing - Daniel Cousineau ZendCon '09 11
12. Automation!  We are programmers after all… Automated Deployment With Phing - Daniel Cousineau ZendCon '09 12
13. Don’t Do More Work Than You Have To! Automated Deployment With Phing - Daniel Cousineau ZendCon '09 13
14. Work Hard At Being Lazy! Automated Deployment With Phing - Daniel Cousineau ZendCon '09 14
15. What, Why, and When Automate? Automated Deployment With Phing - Daniel Cousineau ZendCon '09 15
16. What?  Automated deployment means a single command  Locks your live site  Uploads changed files  Clears caches and temporary files  Updates the database schema  Runs other cron tasks  Unlocks your live site Automated Deployment With Phing - Daniel Cousineau ZendCon '09 16
17. Why?  Deployment is tricky  Simplicity is outweighed by variety  Repetition degrades quality  She sells sea shells by the sea shore Automated Deployment With Phing - Daniel Cousineau ZendCon '09 17
18. When?  All the time!  Staging  Live  Probably best to use it on your dev box too! Automated Deployment With Phing - Daniel Cousineau ZendCon '09 18
19. The Basics Automated Deployment With Phing - Daniel Cousineau ZendCon '09 19
20. Tools of the Trade  Build System  Phing  Apache ANT  Capastrano  Plain PHP or BASH or BAT Files  File Transfer  Rsync (*NIX Environments)  Xcopy (Windows Environments)  Configuration Management  Database Migration  DBDeploy  Doctrine Automated Deployment With Phing - Daniel Cousineau ZendCon '09 20
21. Phing Philosophy  Build scripts contains "Targets"  Targets should be small and specialized.  Example Targets:  clean  Clear temporary and cached files  copy  Copy files to their intended destination  migrate  Upgrade the database schema Automated Deployment With Phing - Daniel Cousineau ZendCon '09 21
22. Phing Philosophy Cont.  Targets can have dependencies  Target "live" can depend on clean, copy, and migrate  Meaning, when we run the "live" target, it first runs clean, copy, then migrate  And any tasks they depend on Automated Deployment With Phing - Daniel Cousineau ZendCon '09 22
23. Installing Phing  pear channel-discover pear.phing.info  pear install phing/Phing  Want all the little dependencies?  pear config-set preferred_state alpha  pear install –alldeps phing/Phing  pear config-set preferred_state stable Automated Deployment With Phing - Daniel Cousineau ZendCon '09 23
24. Running Phing  $> phing –v  Lists Phing version  Phing expects to find a file "build.xml" in the current directory  build.xml defines the targets  You can use the "-f <filename>" flag to specify an alternate build file like  $> phing -f build-live.xml Automated Deployment With Phing - Daniel Cousineau ZendCon '09 24
25. Syntax  XML  If you don’t already know it, you have bigger problems  Variables  ${variablename}  Conventions  Psuedo-Namespaces using periods  ${namespace.variable.name} Automated Deployment With Phing - Daniel Cousineau ZendCon '09 25
26. Basic Conventions  build.xml  Central repository of your top-level tasks  build-*.xml  Can be included into build.xml.  Usually for grouping by target (dev, staging, live) or task (migrate, test, etc.)  build.properties  Technically an INI file that contains variables to be included by build files. Automated Deployment With Phing - Daniel Cousineau ZendCon '09 26
27. Basic build.xml <?xml version="1.0" encoding="UTF-8"?> <project name="ZendCon" default="clean" basedir="."> <property file="build.properties" /> <target name="staging"> <phing phingfile="build-staging.xml" target="deploy" /> </target> </project> Automated Deployment With Phing - Daniel Cousineau ZendCon '09 27
28. Basic build.properties source.directory = /src ## Database Configuration db.user = username db.pass = password db.host = localhost db.name = sample Automated Deployment With Phing - Daniel Cousineau ZendCon '09 28
29. File Transfer Automated Deployment With Phing - Daniel Cousineau ZendCon '09 29
30. Techniques  Built in Phing Copy Task  Cross Platform  Slow  Not over network  Rsync  Great for *NIX environments.  Almost guaranteed to be installed on  Linux, BSD, Mac OSX  Can be installed on Windows  Xcopy  Good for Windows environments  Faster than Phing Copy  Not over network  Version Control Checkout/Update  Syncs deleted files  User created files usually ignored Automated Deployment With Phing - Daniel Cousineau ZendCon '09 30
31. Pitfalls  User created files are a HUGE pitfall  /htdocs/images/upload  Sync programs usually will delete content your deployment machine doesn’t have  What if user upload are mixed in with files you want to manage?  Solutions  Keep user created content completely separated. The further up the file tree and consolodated the better.  Separate static file server, Amazon S3, or other style CDNs Automated Deployment With Phing - Daniel Cousineau ZendCon '09 31
32. Pitfalls  Cleanup Deleted Source Files  Usually only a problem when you have * include patterns  Also creates the previous user created file dillema Automated Deployment With Phing - Daniel Cousineau ZendCon '09 32
33. Executing External Tools  Nearly all file transfer tools will be external commands  For this we need the Exec task <exec command="cp file1 file2" /> Automated Deployment With Phing - Daniel Cousineau ZendCon '09 33
34. Rsync  So many different options you’re best off finding your own tutorials or reading MAN pages  rsync -aCvz -e 'ssh -i /path/to/ssh-key' ${project.basedir}/src user@domain: ${staging.deployment.directory}/ Automated Deployment With Phing - Daniel Cousineau ZendCon '09 34
35. Rsync Options Of Note  -a archive-mode  Recursive copy, preserve everything, etc.  -C cvs-exclude  Ignore .cvs, .svn, and other version control artifacts  -v verbose  Know EXACTLY what’s going on  -z compress  Compress information transmitted  -e alternative remote shell  Use a custom SSH command (namely use key-pair) ZendCon '09 Automated Deployment With Phing - Daniel Cousineau 35
36. Xcopy  Best when your live deployment process involves copying to a network share on Windows machines  xcopy ${project.basedir}${source.directory}\* ${staging.deployment.directory} /D /I /Y /S /E /Exclude:${project.basedir}\staging.deploy ment.exclude Automated Deployment With Phing - Daniel Cousineau ZendCon '09 36
37. Xcopy Options Of Note  /D  Update only those files that are different (timestamps)  /I  Creates target directories if they don’t exist  /Y  Do not prompt  /S  Copy all directories and sub-directories…  /E  …Even if the directories are empty  /Exclude  Exclude files based on glob patterns ZendCon '09 Automated Deployment With Phing - Daniel Cousineau 37
38. Xcopy Exclude File Example ZendCon '09 Automated Deployment With Phing - Daniel Cousineau 38 \tmp\ \uploads\
39. Configuration Management Automated Deployment With Phing - Daniel Cousineau ZendCon '09 39
40. Techniques  Check incoming domain  www.domain.com means load production configuration  localhost/domain.com means load development configuration  Check for an environment file  Contents of DOCUMENT_ROOT/.env determine configuration  Copy over during deployment Automated Deployment With Phing - Daniel Cousineau ZendCon '09 40
41. Check Incoming Domain  Pros  No chance for error  Cons  Detection is a weak link  Especially when development, staging, and live have different web servers (e.g. Apache to IIS) Automated Deployment With Phing - Daniel Cousineau ZendCon '09 41
42. Check for an Environment File  Pros  Consistent regardless of Server or OS changes  Easy to repair  Quick FTP or SSH  Easy Testing  Just swap file contents  Cons  Forget to copy changes? Automated Deployment With Phing - Daniel Cousineau ZendCon '09 42
43. Copy Over During Deployment  Pros  Images and CSS now manageable  Reduce redundancy  Simple to repurpose regular push command  Cons  Difficult to manage  Forget to copy changes? Automated Deployment With Phing - Daniel Cousineau ZendCon '09 43
44. Which Is Best?  Depends on your application architecture  Depends on your environment  You’ll know what’s best for your project  I use a combination of Environment File and Copy Over During Deployment Automated Deployment With Phing - Daniel Cousineau ZendCon '09 44
45. Database Migration Automated Deployment With Phing - Daniel Cousineau ZendCon '09 45
46. Database Deployment Philosophy  “Versions” often referred to as “Deltas”  Each Delta has an UP and a DOWN script  UP makes the changes  DOWN reverts the changes  Each change should be a non-destructive schema change  Unless of course you need data alteration Automated Deployment With Phing - Daniel Cousineau ZendCon '09 46
47. DBDeploy Philosophy  Each delta is 2 native SQL scripts  Separated by --//@undo  Database version is stored in a special table  Beware corruption! Automated Deployment With Phing - Daniel Cousineau ZendCon '09 47
48. DBDeploy Under The Hood  Connect to the database, read from changelog table, get the most current revision installed  Retrieve all delta files newer that the current revision  Revision numbers are based off of alphabetic sorting, name your files accordingly  Pull all the “UP” changes and concatenate them to a migration script  Up to YOU to run the migration script Automated Deployment With Phing - Daniel Cousineau ZendCon '09 48
49. Installing DBDeploy  Is Phing already installed? Great! So is DBDeploy…  Create the changelog table  Track the current version of your DB CREATE TABLE changelog ( change_number BIGINT NOT NULL, delta_set VARCHAR(10) NOT NULL, start_dt TIMESTAMP NOT NULL, complete_dt TIMESTAMP NULL, applied_by VARCHAR(100) NOT NULL, description VARCHAR(500) NOT NULL ); ALTER TABLE changelog ADD CONSTRAINT Pkchangelog PRIMARY KEY (change_number, delta_set); Automated Deployment With Phing - Daniel Cousineau ZendCon '09 49
50. Basic Delta: 01_init.sql --// CREATE TABLE IF NOT EXISTS `users` ( `id` int(10) unsigned NOT NULL auto_increment, `handle` varchar(25) NOT NULL default '', PRIMARY KEY (`id`), UNIQUE KEY `users_handle_index` (`handle`) ) ENGINE=InnoDB DEFAULT; --//@UNDO DROP TABLE IF EXISTS `users`; --// Automated Deployment With Phing - Daniel Cousineau ZendCon '09 50
51. Run Migration  First add a task to your Phing build file <target name="migrate"> </target> Automated Deployment With Phing - Daniel Cousineau ZendCon '09 51
52. Run Migration  Generate the SQL migration file <target name="migrate"> <dbdeploy url="mysql:host=${db.host};dbname=${db.name}" userid="${db.user}" password="${db.pass}" dir="${project.basedir}/db/deltas" outputfile="${project.basedir}/up.sql" undooutputfile="${project.basedir}/down.sql" /> </target> Automated Deployment With Phing - Daniel Cousineau ZendCon '09 52
53. Run Migration  Execute the SQL script <target name="migrate"> <dbdeploy url="mysql:host=${db.host};dbname=${db.name}" userid="${db.user}" password="${db.pass}" dir="${project.basedir}/db/deltas" outputfile="${project.basedir}/uup.sql" undooutputfile="${project.basedir}/down.sql" /> <exec command="/usr/bin/mysql -h${db.local.host} -u${db.local.user} -p${db.local.pass} ${db.local.name} < ${project.basedir}/up.sql" dir="${project.basedir}" checkreturn="true"> </target> Automated Deployment With Phing - Daniel Cousineau ZendCon '09 53
54. Pitfalls  Make sure you have a good CLI app for loading a SQL file  /usr/bin/mysql  mysql.exe  Build script travelling across operating systems?  Write your own Phing task to execute the migration output?  Was the changelog table created in the first place? Automated Deployment With Phing - Daniel Cousineau ZendCon '09 54
55. Doctrine Database Migrations  Integrated into the Doctrine CLI tool  ./doctrine migrate  Same philosophies  BUT, deltas are PHP objects that contain an UP and a DOWN method  Run using the Phing exec task Automated Deployment With Phing - Daniel Cousineau ZendCon '09 55
56. Other Database Migration Tools  http://www.liquibase.org/ Automated Deployment With Phing - Daniel Cousineau ZendCon '09 56
57. Closing Thoughts Automated Deployment With Phing - Daniel Cousineau ZendCon '09 57
58. Further Resources  The people around you  More people than you know automate their deployment  There are many, many different techniques  My techniques may suck for your particular setup  #phpc on freenode  Many of the speakers hang out there  Many smart people hang out there  I HANG OUT THERE! Automated Deployment With Phing - Daniel Cousineau ZendCon '09 58
59. Further Resources  Blogs  http://phpdeveloper.org  Documentation  http://phing.info  http://dbdeploy.com  Google  Experimentation  Pushing to a local directory  Pushing to a virtual machine Automated Deployment With Phing - Daniel Cousineau ZendCon '09 59
60. http://joind.in/937