From 'Legacy' to 'Edge' 2014 Edition

Transcript

1. ! From ‘Legacy' to ‘Edge' 2014 edition ! Hiroshi SHIBATA @hsbt RubyConfTaiwan 2014
2. Thank you! At first, Thank you so much for accept to my proposal. I love taiwan and taipei.
3. SHIBATA Hiroshi(@hsbt) My name is Hiroshi SHIBATA, Please call me hiroshi or shibata. my twitter and github handle name is hsbt.
4. Ruby Committer I’m ruby committer. I prevent that ruby break rails and your application. sometimes I fix ruby, fix rails, fix my code.
5. ruby-lang.org And I’m maintaining ruby-lang.org. I support ruby commiter and make easy to get rubies all over the world like CDN. ! Our site is hosted by github and heroku. If you find something to improve and documentation errors in www.ruby-lang.org, you can fix and send a pull request. we can merge it.
6. Rails Girls in Japan I’m one of organizers in RailsGirls in Japan.
7. asakusa.rb And I’m member of asakusa.rb. asakusa.rb is the most active meetup group in japan. we are talking about ruby and writing code every Tuesday.
8. I’m working for GMO pepabo in japan. we provide server hosting, blog service, shopping mall service etc. we have a lot of web services.
9. ϛυϧ΢ΣΞͷΞοϓάϨʔυ ࣾ಺։ൃج൫ͷߏங ౷ܭج൫ͷߏங +409ͷӡ༻੔උ HJUIVC ηΩϡϦςΟ؂ࠪ ίʔυϨϏϡʔ ςετج൫ͷߏங ΞΫηεղੳج൫ͷߏங ։ൃϓϩηε ৽ਓڭҭ ݪߘࣥච ΧϯϑΝϨϯεൃද 044։ൃ αʔόʔߏ੒؅ཧͷ࡮৽ ϦʔϯελʔτΞοϓ +FOLJOT 3VCZ3BJMT 3%#.4 ٕज़ج൫νʔϜ I’m at technical platform team on company. example for framework upgrade and opensource development, building to analysis platform etc.
10. http://30d.jp We are running photo sharing service named 30days album. Service concept of the service is private photo sharing.
11. from 2008/4 30days launched in April, two thousand eight.
12. 380,000 users 230,000,000 photos we have three hundred eighteen thousand users, and two hundred thirteen million photos.
13. Our Rails app 46 models 5000 lines in controllers 400 lines in routes.rb 1:1.3 code to test ratio our rails is traditional rails. probably our rails is same as your rails.
14. system architecture but our system architecture is too complex system. I’m going to describe system architecture of 30days.
15. application server these are application servers. we separate to customer transaction database from application database.
16. storage server these are storage servers. we don’t use cloud platform. these are build on on-premises platform.
17. job server job servers handles to image manipulation workers. we have a lot of job pattern like rotate and resize.
18. transaction server we can access only office network to customer transaction server.
19. storage is over 450 TB our storage has over four hundred fifteen tera bytes of photos.
20. database size is over 250 GB our database has over 250GB of data. It use MySQL 5.5 with barracuda plugin.
21. Development Culture I’ll introduce the our team background and culture.
22. github workflow We use workflow like a github’s one. master branch is always to allow deployment and all-green test states. all of code is reviewed by others.
23. IRC github issue costomer’s contact nagios we are using IRC(Internet relay chat). we integrate all of alert and monitoring messages to it.
24. DevOps we are using modern operation tools. puppet and newrelic, munin, nagios and more.
25. 4 deploy/day we are deploying one to four times per everyday.
26. I delete unused code. this figure is our team graph of github. deleting code is bigger than writing code. my main work is code vanishment.
27. How to migrate edge rails from legacy rails? It’s Today’s main theme, I’ll introduce the migration strategy.
28. Ruby 1.8.6 Rails 2.0.2 30days had been running under ruby 1.8.6 and rails 2.0.2 when I joined. I was very surprised. because latest rails was 3.2 at that time.
29. Ruby 1.8.6 Rails 2.0.2 Ruby 2.1.1 Rails 4.1.0 new! I decided to upgrade ruby and rails. and started thinking aboud migration strategy.
30. Why? Why we need to migrate and upgrade Ruby and Rails?
31. the answer is this tweet. he said “Ruby 2.1.1 is too fast. We can make faster our web application only upgrading language version, It seems joke to me.”
32. Ruby on Rails It’s true. Rails is Ruby code. We need to focus to Ruby.
33. Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 this matrix shows compatibility between ruby and rails.
34. Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 Slow Fast In Generally, Upgrading Ruby give more faster experience for us, but Upgrading Rails give more slower experience for us.
35. Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 Red line shows big wall of Upgrading Rails, and Blue line shows big wall of Upgrading Ruby for us
36. Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 Our strategy is red arrows routes.
37. Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 At first we jumped Rails 2.3 and Ruby 1.8.7.
38. Rails 2.0 Rails 2.3 I’ll describing our branch strategy. we created Rails 2.3 branch such as above figure. master branch is rails 2.0 line.
39. Rails 2.0 Rails 2.3 hotfix A migration A we made hot fix branch from Rails 2.0 branch. and made a migration branch from Rails 2.3, migration branch contains to rewrite code adopting more rails way, etc.
40. Rails 2.0 Rails 2.3 hotfix A migration A hotfix branch is finished, we merge it to master branch and backport to rails 2.3 branch.
41. Rails 2.0 Rails 2.3 hotfix A migration A we finished migration branch, I merged it to rails 2.3 branch. I merged whole hotfixes and migrations to rails 2.3 branch.
42. It’s real branch status like rainbow line.
43. Zero Downtime Deploy one of the most offensive operation is “zero downtime deploy”
44. cookpad/kage well, I’ll introduce kage.
45. Kage (kah-geh) is an HTTP shadow proxy server that sits between clients and your server(s) to enable "shadow requests". kage is an http shadow proxy server that sits between clients and your server to enable “shadow requests”. kage is developed by cookpad. cookpad is best recipe sharing service in japan.
46. the basic of kage real request this figure is basic usage of kage. we put kage at the back of nginx. while kage proxies a request to production, it duplicate the request and pass though it to staging environment. we can test big changes with real request now. kage can build powerful test suites.
47. but sometimes kage caused trouble to our system.our service can receive big pictures. perhaps it caused this problem I guess ;)
48. Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 Next plan is very hard time for us, Upgrading Rails 3
49. rails 2.3 with bundler I finished to integrate to rails 2.3 and bundler. we can use a lot of gems. example for new relic, delayed_job, rack, and more
50. source 'https://rubygems.org'! ! gem 'rails', '~> 2.3.18'! gem 'rake', '~> 0.9.2'! gem 'rdoc'! gem 'rake-confirm'! ! gem 'mysql'! gem "mysql_retry_lost_connection"! gem 'acts_as_paranoid', :github => 'paperboy-30days/acts_as_paranoid'! gem 'passenger', '~> 3.0'! gem 'memcache-client', :require => 'memcache'! gem 'system_timer'! gem 'yajl-ruby', :require => 'yajl'! gem 'will_paginate', '~> 2.3'! gem 'mail'! ! gem 'sass'! gem 'compass-rails'! ! gem 'osaipo_client', :git => 'git@github.com:paperboy-all/ osaipo_client.git', :branch => 'legacy'! gem 'jugem_client', :git => 'git@github.com:paperboy-all/ jugem_client.git'! gem 'ppb_footer', :github => 'paperboy-all/ppb_footer'! We extracted some plugins, acts_as_paranoid was put vendor/plugin folder, now we moved on our github organization. and we moved our external service client in library folder to organization folder.
51. require "./config/environment"! ! use Rails::Rack::LogTailer! use Rails::Rack::Static! run ActionController::Dispatcher.new rack with rails 2.3 we are using rack with rails 2.3, it can be easily applied to rails 2.3. Because we need to migrate rack in the future.
52. 30days-front can’t use hyphen can’t use number can’t convert ! application.rb... but, I have one of big problem. our repository name can’t be converted to rails 3 style namespace. because our repository use hyphen and integer.
53. Reduce customer frustration. I did works only upgrading. We need to write a new code for customer requirements and business goal.
54. app1 app2 backgroundrb to dj I replaced old job worker named backgroundrb. backgroundrb is on-memory system, so it’s cleared when memory refresh. It means all job is deleted. We and our user got frustration on this situation. I replaced all of job system to persistent workers like delayed_job.
55. this is Pull Request of Upgrading branch. commits are over than two hundred commits. file changed are over the three hundred sixteen five.
56. Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 After Upgrading Rails 3, I started to upgrade Ruby runtime.
57. Use trunk everyday. Basically, I’m Ruby commiter. I use trunk and head of all branches builds.
58. It’s easy to find to defects Ruby, Rubygems, bundler. I report our tracking system. and fixed it quickly.
59. this is my rails fixes. I aimed to targeted version of Ruby is 2.1. I fixed all of interruptions for our Application with Ruby 2.1.
60. 1.8 to 2.0 How to upgrade Ruby 1.8 to 2.0.
61. my friends named mrkn. he talked that how to upgrade Ruby 1.9.3. In this talk contains a lot of technics in the wild. Please read it later.
62. 2.0 to 2.1 2.0 to 2.1 has no trouble. it cares backward compatible.
63. CPU Usage(%) 0 25 50 75 100 49 69 2.0 2.1 This is benchmark for new relic average score. CPU Usage is reduced twenty percents on Ruby 2.1
64. Resp. time(ms) 0 50 100 150 200 250 189 221 2.0 2.1 It’s Response Time comparison. Ruby 2.1 reduces twenty to thirty percents of response time.
65. memory usage(giga byte) 0 1 2 3 4 3.8 3.2 2.0 2.1 But memory usage are increased 10-20% by Ruby 2.1. If you use PaaS service like heroku, you need to evaluate this situation before upgrading Ruby 2.1
66. https://www.ruby-lang.org/en/news/2014/03/10/regression-of-hash-reject-in-ruby-2-1-1/ I noticed you, Ruby 2.1.1 have regression feature. It’s difference behaviour of reject method on Inheritance Hash. Please read our announcement. This regression is already fixed Rails 4.0 and 4.1, If you use Rails 3.2, You need to applied workaround patch.
67. Ruby 1.8.6 1.8.7 1.9.3 2.0.0 2.1 Rails 2.0/2.1 2.3 3.0 3.2 4.0 After Upgrade Ruby 2.1, I started Upgrading Rails 4.0 line.
68. Details of Upgrading Rails 4.0, they are written by this book. this book is free e-book. I recommended this book.
69. Use modern architecture I replaced modern architecher to upgrade Rails 4.
70. app1 app2 Our System used Kyoto Tycoon. Kyoto Tycoon is memcached compatible session storage, it have dual master function. but session client named dalli not support Kyoto tycoon on Rails 4.
71. kyoto tycoon is slower than mysql… And biggest problem is this. Kyoto Tycoon is slower than mysql. It’s caused by dual master mechanism
72. I replaced pure memcached. I solved all problem for upgrading Rails 4
73. Resp. time(ms) 0 60 120 180 240 300 287 189 3.2 4.0 This is benchmark of Rails4. rails 4 is slower than rails 3. it increased 100 micro seconds of response time.
74. Ruby 1.9.3 2.0.0 2.1 2.2 Rails 3.0 3.2 ? 4.0 ? 4.1 ? this is upgraded version of Ruby and Rails combination matrix. I dropped Ruby 1.8 and Rails 2.3.
75. Ruby 1.9.3 2.0.0 2.1 2.2 Rails 3.0 3.2 ? 4.0 ? 4.1 ? New!! I upgraded Rails 4.1 last week!
76. I recommend to official upgrade guides
77. 1. Flash structure changes 2. Mutator methods called on Relation And I notice big incompatible things on Rails 4.1. One Flash structure changes. Rails changed key class of flash to String from Symbol. Two. Mutator methods called on Relation. before Rails 4.1 allows mutator methods like map exclamation on Relation objects. but Rails 4.1 raised this methods.
78. And changed namespace of Minitest. If you use ActionSupport::Testcase and test helper libraries like webmock. Probably Minitest broke these helpers. this is my Pull request of web mock, I already fixed it.
79. Resp. time(ms) 0 60 120 180 240 300 266 287 4.0 4.1 It’s benchmark of Rails 4.0 and 4.1. Rails 4.1 is faster than 4.0. but it’s slow than Rails 3.0 yet.
80. Ruby 1.9.3 2.0.0 2.1 2.2 Rails 3.0 3.2 ? 4.0 ? 4.1 ? I started to evaluate Rails 4.1 with Ruby 2.2. Ruby 2.2 will be releasing December 2014 for us. I work hard to solve incompatible changes before releases.
81. class Bar! def bar(foo = foo())! foo! end! ! def buzz(foo = foo)! foo! end! ! def foo! :buzz! end! end! ! p Bar.new.bar! p Bar.new.buzz This is example of incompatible changes on Ruby 2.2. class Bar have foo equal foo method arguments. bar method called foo methods, buzz method doesn’t called foo methods.
82. % ruby -v r45272.rb! ruby 2.2.0dev (2014-04-13 trunk 45580) [x86_64-darwin13]! :buzz! nil! ! % ruby -v r45272.rb! ruby 2.1.2p80 (2014-03-01 revision 45231) [x86_64-darwin13.0]! :buzz! :buzz ruby 2.2. returns nil. it isn’t assignment same name of default value.
83. This changes break Rails. but it’s already fixed Rails 4.1 by Aaron Patterson. this patch is backported Rails 4.0. but didn’t fix 3.2 branch. If you still use Rails 3.2, You can’t use Ruby 2.2.
84. conclusion Conclusion.
85. 2. fix gem 4. fix Ruby 3. fix Rails 1. fix your code Fix your code. fix using gems. fix Rails. fix Ruby. You should get more controllable things. It’s expand your life!