Deployment Strategies at Codecademy

nycgolang meetup

Transcript

2. Deploying Go At Codecademy
3. Engineer @ Codecademy dan@codecademy http://danordille.com Dan Ordille
4. About Codecademy •Small engineering team (8) •Limited existing Go expertise •No dedicated ops team
5. How We Use Go •Microservice exposing WebSocket API used to interact with Learning Environment (VMs) •Service Deployed to 1K VMs at current peak usage •VMs thrown away once a user’s session becomes idle
6. Terminal Provide PTY sessions for user Code Editor API for file operations (open, write, create, ..) Web Server Adapters Rails, Static File Server Learning Environment
7. Deployment Should Be •Fast •Easy •Testable •Secure
8. First Attempt GIT Checkout + Build •Checkout Code From GIT •go build •go run
9. Fast GIT checkout takes seconds, Go Build takes more seconds. Overall a poor experience. Easy Deployments are relatively simple. Testable Can checkout an arbitrary SHA by modifying an attribute in CHEF. Secure Requires deploy keys and src to exist on VM Failure
10. Second Attempt Binaries on S3 •Build Binary, trigger on CI build •Upload Binary to S3 •Fetch Binary from S3
11. Fast As fast as network Easy Only publishing a binary so configuration must come from secondary source. Run into issues when code has dependency on new configuration. Testable See above. Secure VM only needs access to binary Failure
12. Packages on S3 •Build Packages triggered on CI build •Upload Package to S3 •Tag Package •Fetch Package on VM only if modified
13. Build Compiles and uses ldflags to set SHA Archive Creates a tar.gz file containing binaries, configs for all environments, and any other non binary dependencies. Build Package
14. Build GIT SHA into Binary go build -ldflags “-X main.gitversion `git rev-parse HEAD`”
15. Upload To S3 Every SHA that is built by CI server is uploaded to S3. Store branch as metadata in s3. aws s3api put-object --bucket builds --key releases/godex.tar.gz.${CIRCLE_SHA1} --body bin/godex.tar.gz --metadata branch=${CIRCLE_BRANCH}
16. Copies package from builds bucket into releases bucket. List Tags $ ./contrib/deploy builds GIT SHA GIT BRANCH BUILD DATE 37f4333be1a1b2e02884f0575a477ce07274aae2 master 2015-02-18 15:26:19 -0500 af6e77e1329cf9a06c68240922b29926bb78b10e statsd_timers 2015-02-18 13:20:16 -0500 b0c3cfcb6a6e7da0de0cb3072c0f4f10cbb6c130 master 2015-02-16 12:03:37 -0500 4b1b24d62db022f4d110115d413c73cc1d405d8d mitm_logging 2015-02-16 11:46:01 -0500 1b38976548a1d3a197c58efe5aefec047703463f mitm_logging 2015-02-16 11:26:36 -0500 $ ./contrib/deploy tags TAG GIT SHA PUBLISH DATE staging 698652afe6619dbc42842af8b709563c6783df12 2015-02-15 23:15:03 -0500 production c5b60fa59e22d88987d697371d7afe7613e0224c 2015-02-13 19:31:17 -0500 $ ./contrib/deploy publish --sha 37f4333be1a1b2e02884f0575a477ce07274aae2 --tag production Displays Recent Builds List Builds Copies package from builds bucket into releases bucket. Tag Build
17. Compare SHAs Get currently deployed SHA Fetch new package godexTarMD5=$(md5sum godex.tar.gz | awk '{print $1;}’) head=$(aws s3api head-object --bucket codecademy-godex-builds --key tags/<%= @environment %>.tar.gz) if echo head | grep $godexTarMD5 > /dev/null; then echo "Godex At Latest" exit 0 fi aws s3 cp s3://codecademy-godex-builds/tags/<%= @environment %>.tar.gz ./godex.tar.gz && tar -xvzf ./godex.tar.gz Get SHA of currently tagged package Extract package
18. Questions?