Continuous Delivery in COOKPAD

Slide 1

Slide 1 text

Continuous Delivery in COOKPAD RubyWorld Conference 2013

Slide 2

Slide 2 text

Naoto Takai Manager, Tech Department

Slide 3

Slide 3 text

20M UU/month 1.5M recipes (October 31 2013)

Slide 4

Slide 4 text

Ruby 2.0 Rails 3.2 (October 31 2013)

Slide 5

Slide 5 text

✓ 1,082 models ✓ 318 controllers ✓ 3,304 view templates ✓ 2,127 lines of routes.rb ✓ 3,946 assets (October 31 2013)

Slide 6

Slide 6 text

14,432 examples (October 31 2013)

Slide 7

Slide 7 text

about 10 minutes (October 31 2013)

Slide 8

Slide 8 text

How often do you deploy?

Slide 9

Slide 9 text

11+ deploys/day (October 31 2013)

Slide 10

Slide 10 text

Build-Measure-Learn Build Measure Learn product data idea

Slide 11

Slide 11 text

How we do that?

Slide 12

Slide 12 text

Continuous Delivery “Continuous Delivery is a software development discipline where you build software in such a way that the software can be released to production at any time.” http://martinfowler.com/bliki/ContinuousDelivery.html

Slide 13

Slide 13 text

Continuous Delivery (cont.) “You achieve continuous delivery by continuously integrating the software done by the development team, building executables, and running automated tests on those executables to detect problems.” http://martinfowler.com/bliki/ContinuousDelivery.html

Slide 14

Slide 14 text

✓ Development ✓ Source code review ✓ Continuous integration ✓ Production test ✓ Production Source Code Review Continuous Integration Production Test Developement Production GitHub Git Repository merge pull req pull tag deploy deploy Deployment Pipeline

Slide 15

Slide 15 text

Development Environment Developer Machine Redis Shared Development MySQL EC2 Ruby on Rails memcached Remote Spec Workers remote_spec worker remote_spec worker GitHub Enterprise LAN Tokyo Tyrant Development App Server push rspec deploy HipChat

Slide 16

Slide 16 text

How many people use git?

Slide 17

Slide 17 text

 GitHub Flow ✓ Anything in the master branch is deployable ✓ To work on something new, create a descriptively named branch oﬀ of master (ie: new-oauth2-scopes) ✓ When you need feedback or help, or you think the branch is ready for merging, open a pull request ✓ After someone else has reviewed and signed oﬀ on the feature, you can merge it into master http://scottchacon.com/2011/08/31/github- ow.html

Slide 18

Slide 18 text

 GitHub Flow feature branch feature branch master branch pull request & merge

Slide 19

Slide 19 text

Feature Toggle “The basic idea is to have a con guration le that de nes a bunch of toggles for various features you have pending. The running application then uses these toggles in order to decide whether or not to show the new feature.” http://martinfowler.com/bliki/FeatureToggle.html

Slide 20

Slide 20 text

Feature Toggle Target? Original feature New feature User request Yes No

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

Number of Pull Requests 0 100 200 300 400 500 600 700 800 2012-04 2012-05 2012-06 2012-07 2012-08 2012-09 2012-10 2012-11 2012-12 2013-01 2013-02 2013-03 2013-04 2013-05 2013-06 2013-07 2013-08 2013-09 2013-10 0 10 20 30 40 50 60 Developers Pull Requests

Slide 28

Slide 28 text

Source Code Flow LAN EC2 CI Server Git Repository Production Test HipChat tag pull notify deploy Development DB schema Remote Spec distribute Git Repository clone pusher service hook pull push Developer merge

Slide 29

Slide 29 text

post '/update' do payload = JSON.parse(params[:payload]) project = payload['repository']['name'] if system("pgrep -f \"sh -c git --git-dir=#{REPOSITORY_ROOT}/#{project}\"") next "Found another process for #{project}\n" end cmd = "git --git-dir=#{REPOSITORY_ROOT}/#{project} remote prune origin " + "&& git --git-dir=#{REPOSITORY_ROOT}/#{project} fetch -q " + "&& git --git-dir=#{REPOSITORY_ROOT}/#{project} push -q --mirror git-repo" pid = Process.spawn(cmd) Process.detach(pid) "Spawn following command: #{cmd}\n" end Clone Pusher

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

HipChat Noti cation

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

CI Servers CI Master CI Slave CI Slave CI Slave CI Slave CI Slave Remote Spec Worker Remote Spec Worker Remote Spec Worker Remote Spec Worker Remote Spec Worker Remote Spec Worker Remote Spec Worker Remote Spec Worker CI Slave

Slide 34

Slide 34 text

RSpec Examples Build Time

Slide 35

Slide 35 text

HipChat Noti cation

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Production Test ✓ Same server con guration as production ✓ Server speci cations ✓ Application server (unicorn) ✓ Production database ✓ Manual testing

Slide 38

Slide 38 text

Deployment ✓ Latest CI passed revision ✓ Weekday 9:30 am - 5:00 pm (3:00pm on Friday)

Slide 39

Slide 39 text

$ bundle exec cap production deploy

Slide 40

Slide 40 text

Application Server Deploy Server deploy CI Server HipChat Blog & Wiki Git Repository Developer Machine SSH notify post pull assets LAN EC2 Deployment Server

Slide 41

Slide 41 text

HipChat Noti cation Cry Bot ✓ “A deploy starts.” https://bitbucket.org/winebarrel/zakkuri

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

Rollback and Deploy Lock ✓ bundle exec cap production deploy:rollback ✓ bundle exec cap production deploy:lock

Slide 45

Slide 45 text

11+ deploys/day

Slide 46

Slide 46 text

How we do that?

Slide 47

Slide 47 text

The answer is “nothing special”: ✓ Build a deployment pipeline ✓ Automate a process as possible

Slide 48

Slide 48 text

Continuous delivery makes: ✓ the deployment safer. ✓ nding the bug easier. ✓ faster hypothesis testing.