CI Your JavaScripts!

These are the slides from my talk on DIY CI that I did for a couple of my personal (small scale) projects @ #kaunasjs

Transcript

1. CI  YOUR   JAVASCRIPTS! @joncys
3. ssss
13. IDEAL  WORKFLOW How would I like to structure my development process
14. PREPARE  PRODUCTION  SERVER Install utilities and tools, create directory structure And when ready push to master to trigger CI DEVELOP  LOCALLY RUN  CI  PROCESSES Run unit and e2e tests, transpile, minify and concat files DEPLOY  TO  PRODUCTION Copy files to remote server, restart applications
15. PREPARE   PRODUCTION   SERVER Prepare production server Develop locally Run CI Deploy to production
16. CREATE  STRUCTURE Deploy every individual component to its own subtree Prepare production server Develop locally Run CI Deploy to production
17. # create project directory structure cd ~ mkdir schnitzel mkdir www # results in # /home/deployment/schnitzel # /home/deployment/www
18. AUTHORIZE  CI  AGENT Set up SSH authorization for the deployment user Prepare production server Develop locally Run CI Deploy to production
20. # let the CI agent authenticate via ssh to prevent # hanging up on password prompts # open and paste in with an editor nano ~/.ssh/authorized_keys # or copy via command line cat id.pub | ssh deployment@12.345.67.890 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"
21. INSTALL  NVM For easier node version control Prepare production server Develop locally Run CI Deploy to production
22. • Version manager for nodejs • A set of functions and aliases NVM
23. # install curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.26.1/install.sh | bash # install a node version nvm install 0.12.7 # use it nvm use 0.12.7 # alias a version nvm alias stable 0.12.7 # lel, not true # use an alias nvm use stable
24. INSTALL  FOREVER To monitor and reboot inactive applications Prepare production server Develop locally Run CI Deploy to production
25. • Utility to monitor nodejs applications • Auto restarts apps if they fail FOREVER
26. # install forever as a global package npm install -g forever # run and monitor a script with forever forever start --sourceDir ~/schnitzel/latest --uid schnitzel --spinSleepTime 10000 --append index.js >> ~/schnitzel/forever.log 2>&1 # list all apps running forever list # stop an app forever stop schnitzel # restart an app forever restart schnitzel
27. # cat ~/schnitzel/reboot.sh #!/bin/sh if [ $(ps -e -o uid,cmd | grep $UID | grep node | grep -v grep | wc -l | tr -s "\n") -eq 0 ] then export NVM_DIR="/home/deployment/.nvm" [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvm forever start --sourceDir ~/schnitzel/latest --uid schnitzel --spinSleepTime 10000 -- append index.js >> ~/schnitzel/forever.log 2>&1 fi
28. # open crontab for editing crontab -e # run reboot.sh on every machine reboot @reboot ~/schnitzel/reboot.sh
29. INSTALL  NGINX To proxy node apps and serve static files Prepare production server Develop locally Run CI Deploy to production
30. • A web server • Not apache2 • Can serve static files • Can be used as a proxy NGINX
31. # install nginx sudo apt-get update sudo apt-get install nginx # configure nano /etc/nginx/conf.d/schnitzel.lt.conf # restart to reflect changes nginx -s restart
32. # serve static files server { listen 80; server_name www.schnitzel.lt; location / { root /home/schnitzel/www } } server { listen 80; server_name schnitzel.lt return 301 http://www.schnitzel.lt$request_uri; }
33. # proxy node apps server { listen 80; server_name api.schnitzel.lt location / { proxy_pass http://localhost:1337; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; } }
34. DEVELOP   LOCALLY Prepare production server Develop locally Run CI Deploy to production
35. HAVE  A  GIT  STRAT Deploy on every push to master Prepare production server Develop locally Run CI Deploy to production
36. USE  NPM  SCRIPTS To enable easy switching between CI strategies Prepare production server Develop locally Run CI Deploy to production
37. { "name": "schnitzel-api", "version": "0.4.20", "description": "API Application for schnitzelkartoffel!", "main": "index.js", "dependencies": { "hapi": "^8.8.1", "lowdb": "^0.10.2" }, "devDependencies": { "code": "^1.5.0", "flightplan": "^0.6.4", "lab": "^5.14.0" }, "scripts": { "test": "node_modules/.bin/lab", "package": "gulp" } }
38. # easily run scripts in CI # e2e tests npm run test # or just npm test # build npm run package
39. PRACTICE  TDD Have suites of e2e and unit tests Prepare production server Develop locally Run CI Deploy to production
40. RUN  CI   PROCESSES Prepare production server Develop locally Run CI processes Deploy to production
41. INITIALIZE  NODE Boot up correct version of nodejs using nvm Prepare production server Develop locally Run CI processes Deploy to production
42. # initialize nvm install 0.12.7 nvm use 0.12.7 # install project dependencies npm install # trigger build pipeline npm run package
43. TRANSPILE If using any superset of EcmaScript 5 Prepare production server Develop locally Run CI processes Deploy to production
44. TEST  UNITS To make sure components function in isolation Prepare production server Develop locally Run CI processes Deploy to production
45. BUILD  SOURCES Build up dependency tree, concatenate, minify Prepare production server Develop locally Run CI processes Deploy to production
46. RUN  E2E  TESTS To make sure the whole thing functions correctly Prepare production server Develop locally Run CI processes Deploy to production
47. USE  A  TASK  RUNNER To better facilitate the whole build / test process Prepare production server Develop locally Run CI processes Deploy to production
48. DEPLOY  TO   PRODUCTION Prepare production server Develop locally Run CI Deploy to production
49. COPY  FILES Transfer build artefacts to production server Prepare production server Develop locally Run CI Deploy to production
50. # sync with deleting any files that are not present rsync --delete -avz -e "ssh" ~/clone/ deployment@12.345.67.890:~/schnitzel
51. # sync with deleting any files that are not present # excluding .git, tmp, log directories rsync --delete --exclude '.git' --exclude 'tmp' --exclude 'log' -avz -e "ssh" ~/ clone/ deployment@12.345.67.890:~/api
52. RESTART  APPS Relaunch applications to pick up changes Prepare production server Develop locally Run CI Deploy to production
53. # since the forever app is running # we can reference it by its uid forever restart schnitzel
54. FLIGHTPLAN.JS A tool to run shell commands locally and remotely Prepare production server Develop locally Run CI Deploy to production
55. • Describes deployment process • Runs commands on local and remote machines FLIGHTPLAN.JS
56. var plan = require('flightplan'); var PROJECT_NAME = require('./package.json').name; plan.target('production', { host: '12.345.67.890', username: 'deploy', agent: process.env.SSH_AUTH_SOCK }); var tmpDir = PROJECT_NAME + '-' + new Date().getTime(); var projectHomeDir = '~/' + PROJECT_NAME + '/'; plan.local(function(local) { var filesToCopy = local.exec('git ls-files', {silent: true}); local.transfer(filesToCopy, '/tmp/' + tmpDir); }); plan.remote(function(remote) { remote.exec('cp -R /tmp/' + tmpDir + ' ' + projectHomeDir + tmpDir); remote.rm('-rf /tmp/' + tmpDir); remote.exec('npm --production --prefix ' + projectHomeDir + tmpDir + ' install ' + projectHomeDir + tmpDir); remote.exec('ln -snf ' + projectHomeDir + tmpDir + ' ' projectHomeDir + 'latest'); remote.exec('nvm use stable'); remote.exec('forever restart ' + PROJECT_NAME); });
57. plan.target('production', { host: '12.345.67.890', username: 'deploy', agent: process.env.SSH_AUTH_SOCK });
58. var tmpDir = PROJECT_NAME + '-' + new Date().getTime(); var projectHomeDir = '~/' + PROJECT_NAME + '/'; plan.local(function(local) { var filesToCopy = local.exec('git ls-files', {silent: true}); local.transfer(filesToCopy, '/tmp/' + tmpDir); });
59. var tmpDir = PROJECT_NAME + '-' + new Date().getTime(); var projectHomeDir = '~/' + PROJECT_NAME + '/'; plan.remote(function(remote) { remote.exec('cp -R /tmp/' + tmpDir + ' ' + projectHomeDir + tmpDir); remote.rm('-rf /tmp/' + tmpDir); remote.exec('npm --production -- prefix ' + projectHomeDir + tmpDir + ' install ' + projectHomeDir + tmpDir); remote.exec('ln -snf ' + projectHomeDir + tmpDir + ' ' projectHomeDir + 'latest') remote.exec('nvm use stable'); remote.exec('forever restart ' + PROJECT_NAME); });
60. CAVEATS forever: command not found
61. Installs global dependencies for specific node versions NVM
62. # loads up all globally installed dependencies nvm use stable
63. CAVEATS nvm: command not found
64. • Version manager for nodejs • A set of functions and aliases NVM
65. EXPORT  NVM For the CI agent that needs to run nvm functions
66. # display bash config nano .bashrc
67. # If not running interactively, don't do anything case $- in *i*) ;; *) return;; esac # ... stuff ... export NVM_DIR="/home/deploy/.nvm" [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvm
68. export NVM_DIR="/home/deploy/.nvm" [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvm # If not running interactively, don't do anything case $- in *i*) ;; *) return;; esac # ... stuff ...
70. End of slideshow, click to destroy computer.