CI Your JavaScripts!

CI Your JavaScripts!

These are the slides from my talk on DIY CI that I did for a couple of my personal (small scale) projects @ #kaunasjs

60b52fa64d237bf7a89ea24fd1029aef?s=128

Simonas Jončys

August 26, 2015
Tweet

Transcript

  1. CI  YOUR   JAVASCRIPTS! @joncys

  2. None
  3. ssss

  4. None
  5. None
  6. None
  7. None
  8. None
  9. None
  10. None
  11. None
  12. None
  13. IDEAL  WORKFLOW How would I like to structure my development

    process
  14. PREPARE  PRODUCTION  SERVER Install utilities and tools, create directory structure

    And when ready push to master to trigger CI DEVELOP  LOCALLY RUN  CI  PROCESSES Run unit and e2e tests, transpile, minify and concat files DEPLOY  TO  PRODUCTION Copy files to remote server, restart applications
  15. PREPARE   PRODUCTION   SERVER Prepare production server Develop locally

    Run CI Deploy to production
  16. CREATE  STRUCTURE Deploy every individual component to its own subtree

    Prepare production server Develop locally Run CI Deploy to production
  17. # create project directory structure cd ~ mkdir schnitzel mkdir

    www # results in # /home/deployment/schnitzel # /home/deployment/www
  18. AUTHORIZE  CI  AGENT Set up SSH authorization for the deployment

    user Prepare production server Develop locally Run CI Deploy to production
  19. None
  20. # let the CI agent authenticate via ssh to prevent

    # hanging up on password prompts # open and paste in with an editor nano ~/.ssh/authorized_keys # or copy via command line cat id.pub | ssh deployment@12.345.67.890 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"
  21. INSTALL  NVM For easier node version control Prepare production server

    Develop locally Run CI Deploy to production
  22. • Version manager for nodejs • A set of functions

    and aliases NVM
  23. # install curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.26.1/install.sh | bash # install a

    node version nvm install 0.12.7 # use it nvm use 0.12.7 # alias a version nvm alias stable 0.12.7 # lel, not true # use an alias nvm use stable
  24. INSTALL  FOREVER To monitor and reboot inactive applications Prepare production

    server Develop locally Run CI Deploy to production
  25. • Utility to monitor nodejs applications • Auto restarts apps

    if they fail FOREVER
  26. # install forever as a global package npm install -g

    forever # run and monitor a script with forever forever start --sourceDir ~/schnitzel/latest --uid schnitzel --spinSleepTime 10000 --append index.js >> ~/schnitzel/forever.log 2>&1 # list all apps running forever list # stop an app forever stop schnitzel # restart an app forever restart schnitzel
  27. # cat ~/schnitzel/reboot.sh #!/bin/sh if [ $(ps -e -o uid,cmd

    | grep $UID | grep node | grep -v grep | wc -l | tr -s "\n") -eq 0 ] then export NVM_DIR="/home/deployment/.nvm" [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvm forever start --sourceDir ~/schnitzel/latest --uid schnitzel --spinSleepTime 10000 -- append index.js >> ~/schnitzel/forever.log 2>&1 fi
  28. # open crontab for editing crontab -e # run reboot.sh

    on every machine reboot @reboot ~/schnitzel/reboot.sh
  29. INSTALL  NGINX To proxy node apps and serve static files

    Prepare production server Develop locally Run CI Deploy to production
  30. • A web server • Not apache2 • Can serve

    static files • Can be used as a proxy NGINX
  31. # install nginx sudo apt-get update sudo apt-get install nginx

    # configure nano /etc/nginx/conf.d/schnitzel.lt.conf # restart to reflect changes nginx -s restart
  32. # serve static files server { listen 80; server_name www.schnitzel.lt;

    location / { root /home/schnitzel/www } } server { listen 80; server_name schnitzel.lt return 301 http://www.schnitzel.lt$request_uri; }
  33. # proxy node apps server { listen 80; server_name api.schnitzel.lt

    location / { proxy_pass http://localhost:1337; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; } }
  34. DEVELOP   LOCALLY Prepare production server Develop locally Run CI

    Deploy to production
  35. HAVE  A  GIT  STRAT Deploy on every push to master

    Prepare production server Develop locally Run CI Deploy to production
  36. USE  NPM  SCRIPTS To enable easy switching between CI strategies

    Prepare production server Develop locally Run CI Deploy to production
  37. { "name": "schnitzel-api", "version": "0.4.20", "description": "API Application for schnitzelkartoffel!",

    "main": "index.js", "dependencies": { "hapi": "^8.8.1", "lowdb": "^0.10.2" }, "devDependencies": { "code": "^1.5.0", "flightplan": "^0.6.4", "lab": "^5.14.0" }, "scripts": { "test": "node_modules/.bin/lab", "package": "gulp" } }
  38. # easily run scripts in CI # e2e tests npm

    run test # or just npm test # build npm run package
  39. PRACTICE  TDD Have suites of e2e and unit tests Prepare

    production server Develop locally Run CI Deploy to production
  40. RUN  CI   PROCESSES Prepare production server Develop locally Run

    CI processes Deploy to production
  41. INITIALIZE  NODE Boot up correct version of nodejs using nvm

    Prepare production server Develop locally Run CI processes Deploy to production
  42. # initialize nvm install 0.12.7 nvm use 0.12.7 # install

    project dependencies npm install # trigger build pipeline npm run package
  43. TRANSPILE If using any superset of EcmaScript 5 Prepare production

    server Develop locally Run CI processes Deploy to production
  44. TEST  UNITS To make sure components function in isolation Prepare

    production server Develop locally Run CI processes Deploy to production
  45. BUILD  SOURCES Build up dependency tree, concatenate, minify Prepare production

    server Develop locally Run CI processes Deploy to production
  46. RUN  E2E  TESTS To make sure the whole thing functions

    correctly Prepare production server Develop locally Run CI processes Deploy to production
  47. USE  A  TASK  RUNNER To better facilitate the whole build

    / test process Prepare production server Develop locally Run CI processes Deploy to production
  48. DEPLOY  TO   PRODUCTION Prepare production server Develop locally Run

    CI Deploy to production
  49. COPY  FILES Transfer build artefacts to production server Prepare production

    server Develop locally Run CI Deploy to production
  50. # sync with deleting any files that are not present

    rsync --delete -avz -e "ssh" ~/clone/ deployment@12.345.67.890:~/schnitzel
  51. # sync with deleting any files that are not present

    # excluding .git, tmp, log directories rsync --delete --exclude '.git' --exclude 'tmp' --exclude 'log' -avz -e "ssh" ~/ clone/ deployment@12.345.67.890:~/api
  52. RESTART  APPS Relaunch applications to pick up changes Prepare production

    server Develop locally Run CI Deploy to production
  53. # since the forever app is running # we can

    reference it by its uid forever restart schnitzel
  54. FLIGHTPLAN.JS A tool to run shell commands locally and remotely

    Prepare production server Develop locally Run CI Deploy to production
  55. • Describes deployment process • Runs commands on local and

    remote machines FLIGHTPLAN.JS
  56. var plan = require('flightplan'); var PROJECT_NAME = require('./package.json').name; plan.target('production', {

    host: '12.345.67.890', username: 'deploy', agent: process.env.SSH_AUTH_SOCK }); var tmpDir = PROJECT_NAME + '-' + new Date().getTime(); var projectHomeDir = '~/' + PROJECT_NAME + '/'; plan.local(function(local) { var filesToCopy = local.exec('git ls-files', {silent: true}); local.transfer(filesToCopy, '/tmp/' + tmpDir); }); plan.remote(function(remote) { remote.exec('cp -R /tmp/' + tmpDir + ' ' + projectHomeDir + tmpDir); remote.rm('-rf /tmp/' + tmpDir); remote.exec('npm --production --prefix ' + projectHomeDir + tmpDir + ' install ' + projectHomeDir + tmpDir); remote.exec('ln -snf ' + projectHomeDir + tmpDir + ' ' projectHomeDir + 'latest'); remote.exec('nvm use stable'); remote.exec('forever restart ' + PROJECT_NAME); });
  57. plan.target('production', { host: '12.345.67.890', username: 'deploy', agent: process.env.SSH_AUTH_SOCK });

  58. var tmpDir = PROJECT_NAME + '-' + new Date().getTime(); var

    projectHomeDir = '~/' + PROJECT_NAME + '/'; plan.local(function(local) { var filesToCopy = local.exec('git ls-files', {silent: true}); local.transfer(filesToCopy, '/tmp/' + tmpDir); });
  59. var tmpDir = PROJECT_NAME + '-' + new Date().getTime(); var

    projectHomeDir = '~/' + PROJECT_NAME + '/'; plan.remote(function(remote) { remote.exec('cp -R /tmp/' + tmpDir + ' ' + projectHomeDir + tmpDir); remote.rm('-rf /tmp/' + tmpDir); remote.exec('npm --production -- prefix ' + projectHomeDir + tmpDir + ' install ' + projectHomeDir + tmpDir); remote.exec('ln -snf ' + projectHomeDir + tmpDir + ' ' projectHomeDir + 'latest') remote.exec('nvm use stable'); remote.exec('forever restart ' + PROJECT_NAME); });
  60. CAVEATS forever: command not found

  61. Installs global dependencies for specific node versions NVM

  62. # loads up all globally installed dependencies nvm use stable

  63. CAVEATS nvm: command not found

  64. • Version manager for nodejs • A set of functions

    and aliases NVM
  65. EXPORT  NVM For the CI agent that needs to run

    nvm functions
  66. # display bash config nano .bashrc

  67. # If not running interactively, don't do anything case $-

    in *i*) ;; *) return;; esac # ... stuff ... export NVM_DIR="/home/deploy/.nvm" [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvm
  68. export NVM_DIR="/home/deploy/.nvm" [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" #

    This loads nvm # If not running interactively, don't do anything case $- in *i*) ;; *) return;; esac # ... stuff ...
  69. None
  70. End of slideshow, click to destroy computer.