From 'Legacy' to 'Edge' 2014 Edition

by SHIBATA Hiroshi

Slide 1

Slide 1 text

! From ‘Legacy' to ‘Edge' 2014 edition ! Hiroshi SHIBATA @hsbt RubyConfTaiwan 2014

Slide 2

Slide 2 text

Thank you! At ﬁrst, Thank you so much for accept to my proposal. I love taiwan and taipei.

Slide 3

Slide 3 text

SHIBATA Hiroshi(@hsbt) My name is Hiroshi SHIBATA, Please call me hiroshi or shibata. my twitter and github handle name is hsbt.

Slide 4

Slide 4 text

Ruby Committer I’m ruby committer. I prevent that ruby break rails and your application. sometimes I fix ruby, fix rails, fix my code.

Slide 5

Slide 5 text

ruby-lang.org And I’m maintaining ruby-lang.org. I support ruby commiter and make easy to get rubies all over the world like CDN. ! Our site is hosted by github and heroku. If you ﬁnd something to improve and documentation errors in www.ruby-lang.org, you can ﬁx and send a pull request. we can merge it.

Slide 6

Slide 6 text

Rails Girls in Japan I’m one of organizers in RailsGirls in Japan.

Slide 7

Slide 7 text

asakusa.rb And I’m member of asakusa.rb. asakusa.rb is the most active meetup group in japan. we are talking about ruby and writing code every Tuesday.

Slide 8

Slide 8 text

I’m working for GMO pepabo in japan. we provide server hosting, blog service, shopping mall service etc. we have a lot of web services.

Slide 9

Slide 9 text

ϛυϧ΢ΣΞͷΞοϓάϨʔυ ࣾ಺։ൃج൫ͷߏங ౷ܭج൫ͷߏங +409ͷӡ༻੔උ HJUIVC ηΩϡϦςΟ؂ࠪ ίʔυϨϏϡʔ ςετج൫ͷߏங ΞΫηεղੳج൫ͷߏங ։ൃϓϩηε ৽ਓڭҭ ݪߘࣥච ΧϯϑΝϨϯεൃද 044։ൃ αʔόʔߏ੒؅ཧͷ࡮৽ ϦʔϯελʔτΞοϓ +FOLJOT 3VCZ3BJMT 3%#.4 ٕज़ج൫νʔϜ I’m at technical platform team on company. example for framework upgrade and opensource development, building to analysis platform etc.

Slide 10

Slide 10 text

http://30d.jp We are running photo sharing service named 30days album. Service concept of the service is private photo sharing.

Slide 11

Slide 11 text

from 2008/4 30days launched in April, two thousand eight.

Slide 12

Slide 12 text

380,000 users 230,000,000 photos we have three hundred eighteen thousand users, and two hundred thirteen million photos.

Slide 13

Slide 13 text

Our Rails app 46 models 5000 lines in controllers 400 lines in routes.rb 1:1.3 code to test ratio our rails is traditional rails. probably our rails is same as your rails.

Slide 14

Slide 14 text

system architecture but our system architecture is too complex system. I’m going to describe system architecture of 30days.

Slide 15

Slide 15 text

application server these are application servers. we separate to customer transaction database from application database.

Slide 16

Slide 16 text

storage server these are storage servers. we don’t use cloud platform. these are build on on-premises platform.

Slide 17

Slide 17 text

job server job servers handles to image manipulation workers. we have a lot of job pattern like rotate and resize.

Slide 18

Slide 18 text

transaction server we can access only office network to customer transaction server.

Slide 19

Slide 19 text

storage is over 450 TB our storage has over four hundred ﬁfteen tera bytes of photos.

Slide 20

Slide 20 text

database size is over 250 GB our database has over 250GB of data. It use MySQL 5.5 with barracuda plugin.

Slide 21

Slide 21 text

Development Culture I’ll introduce the our team background and culture.

Slide 22

Slide 22 text

github workﬂow We use workﬂow like a github’s one. master branch is always to allow deployment and all-green test states. all of code is reviewed by others.

Slide 23

Slide 23 text

IRC github issue costomer’s contact nagios we are using IRC(Internet relay chat). we integrate all of alert and monitoring messages to it.

Slide 24

Slide 24 text

DevOps we are using modern operation tools. puppet and newrelic, munin, nagios and more.

Slide 25

Slide 25 text

4 deploy/day we are deploying one to four times per everyday.

Slide 26

Slide 26 text

I delete unused code. this ﬁgure is our team graph of github. deleting code is bigger than writing code. my main work is code vanishment.

Slide 27

Slide 27 text

How to migrate edge rails from legacy rails? It’s Today’s main theme, I’ll introduce the migration strategy.

Slide 28

Slide 28 text

Ruby 1.8.6 Rails 2.0.2 30days had been running under ruby 1.8.6 and rails 2.0.2 when I joined. I was very surprised. because latest rails was 3.2 at that time.

Slide 29

Slide 29 text

Ruby 1.8.6 Rails 2.0.2 Ruby 2.1.1 Rails 4.1.0 new! I decided to upgrade ruby and rails. and started thinking aboud migration strategy.

Slide 30

Slide 30 text

Why? Why we need to migrate and upgrade Ruby and Rails?

Slide 31

Slide 31 text

the answer is this tweet. he said “Ruby 2.1.1 is too fast. We can make faster our web application only upgrading language version, It seems joke to me.”

Slide 32

Slide 32 text

Ruby on Rails It’s true. Rails is Ruby code. We need to focus to Ruby.

Slide 33

Slide 33 text

Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 this matrix shows compatibility between ruby and rails.

Slide 34

Slide 34 text

Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 Slow Fast In Generally, Upgrading Ruby give more faster experience for us, but Upgrading Rails give more slower experience for us.

Slide 35

Slide 35 text

Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 Red line shows big wall of Upgrading Rails, and Blue line shows big wall of Upgrading Ruby for us

Slide 36

Slide 36 text

Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 Our strategy is red arrows routes.

Slide 37

Slide 37 text

Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 At ﬁrst we jumped Rails 2.3 and Ruby 1.8.7.

Slide 38

Slide 38 text

Rails 2.0 Rails 2.3 I’ll describing our branch strategy. we created Rails 2.3 branch such as above ﬁgure. master branch is rails 2.0 line.

Slide 39

Slide 39 text

Rails 2.0 Rails 2.3 hotﬁx A migration A we made hot ﬁx branch from Rails 2.0 branch. and made a migration branch from Rails 2.3, migration branch contains to rewrite code adopting more rails way, etc.

Slide 40

Slide 40 text

Rails 2.0 Rails 2.3 hotfix A migration A hotfix branch is finished, we merge it to master branch and backport to rails 2.3 branch.

Slide 41

Slide 41 text

Rails 2.0 Rails 2.3 hotfix A migration A we finished migration branch, I merged it to rails 2.3 branch. I merged whole hotfixes and migrations to rails 2.3 branch.

Slide 42

Slide 42 text

It’s real branch status like rainbow line.

Slide 43

Slide 43 text

Zero Downtime Deploy one of the most offensive operation is “zero downtime deploy”

Slide 44

Slide 44 text

cookpad/kage well, I’ll introduce kage.

Slide 45

Slide 45 text

Kage (kah-geh) is an HTTP shadow proxy server that sits between clients and your server(s) to enable "shadow requests". kage is an http shadow proxy server that sits between clients and your server to enable “shadow requests”. kage is developed by cookpad. cookpad is best recipe sharing service in japan.

Slide 46

Slide 46 text

the basic of kage real request this ﬁgure is basic usage of kage. we put kage at the back of nginx. while kage proxies a request to production, it duplicate the request and pass though it to staging environment. we can test big changes with real request now. kage can build powerful test suites.

Slide 47

Slide 47 text

but sometimes kage caused trouble to our system.our service can receive big pictures. perhaps it caused this problem I guess ;)

Slide 48

Slide 48 text

Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 Next plan is very hard time for us, Upgrading Rails 3

Slide 49

Slide 49 text

rails 2.3 with bundler I ﬁnished to integrate to rails 2.3 and bundler. we can use a lot of gems. example for new relic, delayed_job, rack, and more

Slide 50

Slide 50 text

source 'https://rubygems.org'! ! gem 'rails', '~> 2.3.18'! gem 'rake', '~> 0.9.2'! gem 'rdoc'! gem 'rake-confirm'! ! gem 'mysql'! gem "mysql_retry_lost_connection"! gem 'acts_as_paranoid', :github => 'paperboy-30days/acts_as_paranoid'! gem 'passenger', '~> 3.0'! gem 'memcache-client', :require => 'memcache'! gem 'system_timer'! gem 'yajl-ruby', :require => 'yajl'! gem 'will_paginate', '~> 2.3'! gem 'mail'! ! gem 'sass'! gem 'compass-rails'! ! gem 'osaipo_client', :git => '[email protected]:paperboy-all/ osaipo_client.git', :branch => 'legacy'! gem 'jugem_client', :git => '[email protected]:paperboy-all/ jugem_client.git'! gem 'ppb_footer', :github => 'paperboy-all/ppb_footer'! We extracted some plugins, acts_as_paranoid was put vendor/plugin folder, now we moved on our github organization. and we moved our external service client in library folder to organization folder.

Slide 51

Slide 51 text

require "./config/environment"! ! use Rails::Rack::LogTailer! use Rails::Rack::Static! run ActionController::Dispatcher.new rack with rails 2.3 we are using rack with rails 2.3, it can be easily applied to rails 2.3. Because we need to migrate rack in the future.

Slide 52

Slide 52 text

30days-front can’t use hyphen can’t use number can’t convert ! application.rb... but, I have one of big problem. our repository name can’t be converted to rails 3 style namespace. because our repository use hyphen and integer.

Slide 53

Slide 53 text

Reduce customer frustration. I did works only upgrading. We need to write a new code for customer requirements and business goal.

Slide 54

Slide 54 text

app1 app2 backgroundrb to dj I replaced old job worker named backgroundrb. backgroundrb is on-memory system, so it’s cleared when memory refresh. It means all job is deleted. We and our user got frustration on this situation. I replaced all of job system to persistent workers like delayed_job.

Slide 55

Slide 55 text

this is Pull Request of Upgrading branch. commits are over than two hundred commits. ﬁle changed are over the three hundred sixteen ﬁve.

Slide 56

Slide 56 text

Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 After Upgrading Rails 3, I started to upgrade Ruby runtime.

Slide 57

Slide 57 text

Use trunk everyday. Basically, I’m Ruby commiter. I use trunk and head of all branches builds.

Slide 58

Slide 58 text

It’s easy to ﬁnd to defects Ruby, Rubygems, bundler. I report our tracking system. and ﬁxed it quickly.

Slide 59

Slide 59 text

this is my rails ﬁxes. I aimed to targeted version of Ruby is 2.1. I ﬁxed all of interruptions for our Application with Ruby 2.1.

Slide 60

Slide 60 text

1.8 to 2.0 How to upgrade Ruby 1.8 to 2.0.

Slide 61

Slide 61 text

my friends named mrkn. he talked that how to upgrade Ruby 1.9.3. In this talk contains a lot of technics in the wild. Please read it later.

Slide 62

Slide 62 text

2.0 to 2.1 2.0 to 2.1 has no trouble. it cares backward compatible.

Slide 63

Slide 63 text

CPU Usage(%) 0 25 50 75 100 49 69 2.0 2.1 This is benchmark for new relic average score. CPU Usage is reduced twenty percents on Ruby 2.1

Slide 64

Slide 64 text

Resp. time(ms) 0 50 100 150 200 250 189 221 2.0 2.1 It’s Response Time comparison. Ruby 2.1 reduces twenty to thirty percents of response time.

Slide 65

Slide 65 text

memory usage(giga byte) 0 1 2 3 4 3.8 3.2 2.0 2.1 But memory usage are increased 10-20% by Ruby 2.1. If you use PaaS service like heroku, you need to evaluate this situation before upgrading Ruby 2.1

Slide 66

Slide 66 text

https://www.ruby-lang.org/en/news/2014/03/10/regression-of-hash-reject-in-ruby-2-1-1/ I noticed you, Ruby 2.1.1 have regression feature. It’s difference behaviour of reject method on Inheritance Hash. Please read our announcement. This regression is already ﬁxed Rails 4.0 and 4.1, If you use Rails 3.2, You need to applied workaround patch.

Slide 67

Slide 67 text

Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 After Upgrade Ruby 2.1, I started Upgrading Rails 4.0 line.

Slide 68

Slide 68 text

Details of Upgrading Rails 4.0, they are written by this book. this book is free e-book. I recommended this book.

Slide 69

Slide 69 text

Use modern architecture I replaced modern architecher to upgrade Rails 4.

Slide 70

Slide 70 text

app1 app2 Our System used Kyoto Tycoon. Kyoto Tycoon is memcached compatible session storage, it have dual master function. but session client named dalli not support Kyoto tycoon on Rails 4.

Slide 71

Slide 71 text

kyoto tycoon is slower than mysql… And biggest problem is this. Kyoto Tycoon is slower than mysql. It’s caused by dual master mechanism

Slide 72

Slide 72 text

I replaced pure memcached. I solved all problem for upgrading Rails 4

Slide 73

Slide 73 text

Resp. time(ms) 0 60 120 180 240 300 287 189 3.2 4.0 This is benchmark of Rails4. rails 4 is slower than rails 3. it increased 100 micro seconds of response time.

Slide 74

Slide 74 text

Ruby 1.9.3 2.0.0 2.1 2.2 Rails 3.0 3.2 ? 4.0 ? 4.1 ? this is upgraded version of Ruby and Rails combination matrix. I dropped Ruby 1.8 and Rails 2.3.

Slide 75

Slide 75 text

Ruby 1.9.3 2.0.0 2.1 2.2 Rails 3.0 3.2 ? 4.0 ? 4.1 ? New!! I upgraded Rails 4.1 last week!

Slide 76

Slide 76 text

I recommend to official upgrade guides

Slide 77

Slide 77 text

1. Flash structure changes 2. Mutator methods called on Relation And I notice big incompatible things on Rails 4.1. One Flash structure changes. Rails changed key class of ﬂash to String from Symbol. Two. Mutator methods called on Relation. before Rails 4.1 allows mutator methods like map exclamation on Relation objects. but Rails 4.1 raised this methods.

Slide 78

Slide 78 text

And changed namespace of Minitest. If you use ActionSupport::Testcase and test helper libraries like webmock. Probably Minitest broke these helpers. this is my Pull request of web mock, I already ﬁxed it.

Slide 79

Slide 79 text

Resp. time(ms) 0 60 120 180 240 300 266 287 4.0 4.1 It’s benchmark of Rails 4.0 and 4.1. Rails 4.1 is faster than 4.0. but it’s slow than Rails 3.0 yet.

Slide 80

Slide 80 text

Ruby 1.9.3 2.0.0 2.1 2.2 Rails 3.0 3.2 ? 4.0 ? 4.1 ? I started to evaluate Rails 4.1 with Ruby 2.2. Ruby 2.2 will be releasing December 2014 for us. I work hard to solve incompatible changes before releases.

Slide 81

Slide 81 text

class Bar! def bar(foo = foo())! foo! end! ! def buzz(foo = foo)! foo! end! ! def foo! :buzz! end! end! ! p Bar.new.bar! p Bar.new.buzz This is example of incompatible changes on Ruby 2.2. class Bar have foo equal foo method arguments. bar method called foo methods, buzz method doesn’t called foo methods.

Slide 82

Slide 82 text

% ruby -v r45272.rb! ruby 2.2.0dev (2014-04-13 trunk 45580) [x86_64-darwin13]! :buzz! nil! ! % ruby -v r45272.rb! ruby 2.1.2p80 (2014-03-01 revision 45231) [x86_64-darwin13.0]! :buzz! :buzz ruby 2.2. returns nil. it isn’t assignment same name of default value.

Slide 83

Slide 83 text

This changes break Rails. but it’s already ﬁxed Rails 4.1 by Aaron Patterson. this patch is backported Rails 4.0. but didn’t ﬁx 3.2 branch. If you still use Rails 3.2, You can’t use Ruby 2.2.

Slide 84

Slide 84 text

conclusion Conclusion.

Slide 85

Slide 85 text

2. fix gem 4. fix Ruby 3. fix Rails 1. fix your code Fix your code. fix using gems. fix Rails. fix Ruby. You should get more controllable things. It’s expand your life!