Jenkins All the Things

Continuous Integration, Delivery, and Deployment are now viewed as a precondition to successful software development much like version control, build tools,... But often the question remains: "How?!"
This talk gives an overview how we build and deploy all the things with Jenkins:
* Continuous Deployment of our (static) website to a webserver and schemas to Amazon S3.
* Continuous Deployment of our Java applications to the development infrastructure on AWS.
* Continuous Delivery to the production infrastructure with a manual promotion step.

Additionally, we discuss what the preconditions and common problems are as well as best practices we have come to appreciate — like how to secure your credentials.

Transcript

6. Electronic Data Interchange EDI
7. ViennaDB Papers We Love Vienna
8. Conferences Booster Bergen • CodeMotion Rome • ConFESS Vienna • Linuxwochen Vienna • PHP Tour Luxembourg • GeeCON Krakow • JavaDay Minsk • JEEConf Kiev • nosql matters Dublin • Dev Talks Bucharest • Devoxx Poland Krakow • distributed matters Berlin • code.talks Hamburg • JavaDay Charkiv • WebCamp Zagreb • SilverStripe Europe London • Devoxx Antwerp • Devoxx Marocco Casablanca • BSides Vienna • IT-Tage Frankfurt
9. Towards Continuous ✨
10. ! Version control
11. ! Dependency management
12. ! Automated tests
13. ! Automated builds
14. ! Automated deploys
16. Integration Build + test
17. Delivery Run in production-like
18. Deployment Run in production
19. You build it, you run it. — Werner Vogels
21. Kohsuke Kawaguchi Sun Microsystems Oracle CloudBees
24. Eierlegende Wollmilch- sau
25. "egg-laying wool-milk- sow"
29. Schema to S3
31. Execute shell find . -type f -exec sed -i '/XMLSpy/d' {} \; if [[ ! -d s3cmd/ ]] then wget https://github.com/s3tools/s3cmd/releases/download/v1.5.2/s3cmd-1.5.2.tar.gz tar -xzvf s3cmd-1.5.2.tar.gz mv s3cmd-1.5.2/ s3cmd/ fi virtualenv --distribute DEV DEV/bin/pip install python-dateutil source DEV/bin/activate s3cmd/s3cmd del --config=/private/<myuser>/s3cfg --verbose --recursive s3://schemas.ecosio.com/erpel-1p0/ s3cmd/s3cmd put --config=/private/<myuser>/s3cfg --verbose --recursive --acl-public external/* s3://schemas.ecosio.com/erpel-1p0/
34. Website to DigitalOcean
35. http://jbake.org
36. https://middlemanapp.com
37. Execute shell curl -s -o use-ruby https://repository-cloudbees.forge.cloudbees.com/distributions/ci-addons/ruby/use-ruby RUBY_VERSION=1.9.3-p448 . ./use-ruby curl -s -o use-node https://repository-cloudbees.forge.cloudbees.com/distributions/ci-addons/node/use-node NODE_VERSION=0.11.8 . ./use-node gem install --conservative --no-ri --no-rdoc bundler middleman cd $WORKSPACE bundle install bundle exec middleman build --clean --verbose
40. Check URL #!/bin/bash URLS=( "http://ecosio.com" "https://ecosio.com" "http://ecosio.com/de" ) echo -e "\x1B[0mPositive tests" for URL in "${URLS[@]}" do if curl --output /dev/null --silent --head --fail "$URL" then echo -e "\x1B[0;32mURL exists: $URL" else echo -e "\x1B[0;31mURL does not exist: $URL" STATUS=1 fi done echo -e "\x1B[0mExit code $STATUS" exit $STATUS
42. Java apps to AWS
45. Execute shell rm -fR $JAVA_HOME cp -Lr /opt/jdk/openjdk7.latest/ $JAVA_HOME cp -f /private/<myuser>/jce7/* $JAVA_HOME/jre/lib/security/ FILE=app.version rm -f "$FILE" echo ${BUILD_ID} > "$FILE" echo `git rev-parse HEAD` >> "$FILE" echo ${POM_VERSION} >> "$FILE" echo ${BUILD_NUMBER} >> "$FILE" echo "master" >> "$FILE"
46. Stability Automated builds Manual blessing Weekend deployments
47. Code master release Pull Requests
48. Tools Java + Maven ↓ GitHub ↓ Jenkins ↓ S3
    EC2
49. Environments local, vagrant, test development, stage, production
50. Properties db.host=127.0.0.1 db.database=dev
51. First try
52. Blessing ! Promote the build on Jenkins $ mvn package -Dspring.profiles.active=development Copy the artifact to S3 Fetch it during the deployment
54. What You Bless Is not What You Package Changed snapshot dependencies Always packages the latest build MissingProjectException
55. in the cloud, no one can hear you scream — @sadserver, https://twitter.com/sadserver/ status/641960756678889472
56. If you fail Try again
57. Unified Build Package during the build Copy to S3 during the promotion
58. Spring Profiles Load the right properties file Define the environment on the instance
62. Execute shell cp -f ${BUILD_TAG}/target/*.zip ${BUILD_TAG}/target/ftp-latest-app.zip if [[ ! -d s3cmd/ ]] then wget https://github.com/s3tools/s3cmd/releases/download/v1.5.2/s3cmd-1.5.2.tar.gz tar -xzvf s3cmd-1.5.2.tar.gz mv s3cmd-1.5.2/ s3cmd/ fi virtualenv --distribute DEV DEV/bin/pip install python-dateutil source DEV/bin/activate sudo /opt/jenkins/sbin/mount-webdav https://repository-<myuser>.forge.cloudbees.com/private <myuser> s3cmd/s3cmd put --config=/private/<myuser>/s3cfg ${BUILD_TAG}/target/*.zip s3://com.ecosio.artifacts/prod/ftp/
68. Secrets?
69. Encrypt all the properties http://ejohn.org/blog/keeping-passwords-in-source-control/
71. .gitignore env/*/secret.properties
72. #!/bin/sh FILE=$1 FILENAME=$(basename "$FILE") DIRECTORY=$(dirname "$FILE") EXTENSION="${FILENAME##*.}" NAME="${FILENAME%.*}" if [[ "$EXTENSION" != "aes256" ]] then echo "Encrypting $FILENAME and removing the plaintext file" openssl aes-256-cbc -e -a -in $DIRECTORY/$FILENAME -out $DIRECTORY/${FILENAME}.aes256 rm $DIRECTORY/$FILENAME else then echo "Decrypting $FILENAME" openssl aes-256-cbc -d -a -in $DIRECTORY/$FILENAME -out $DIRECTORY/$NAME fi
73. Separate code and config Test: Could you opensource your app right now?
74. Pull Requests
80. Be polite Retest this please
82. Conclusion
86. Developers ❤ Jenkins
87. Operations ❤ Jenkins
88. Business ❤ Jenkins
90. ش ك ر ا Questions? @xeraa