Evolution of Rails application architecture: 14 years in production

Evolution of Rails application architecture: 14 years in production

The story about the architecture of famous and biggest Rails projects such as Shopify, Basecamp and Cookpad.

Ee8543e8f5554aaeabb6d140fa07b919?s=128

Nikolay Sverchkov

October 07, 2018
Tweet

Transcript

  1. None
  2. None
  3. None
  4. evilmartians.com

  5. WHAT AWAITS US NOW?

  6. CLOSING KEYNOTE

  7. WE LOVE CATS!

  8. TATARSTAN

  9. None
  10. THE 2017/2018 CHAMPIONS

  11. 15 DAKAR RALLY WINS

  12. 34TH BY POPULATION CITY IN RUSSIA

  13. 34TH BY POPULATION CITY IN RUSSIA THE THIRD AMONG THE

    CITIES THAT ARE NOT THE CENTRES OF THE SUBJECTS OF FEDERATIONS
  14. None
  15. None
  16. None
  17. None
  18. https://ru.wikipedia.org/wiki/Эволюция_(значения)

  19. https://ru.wikipedia.org/wiki/Эволюция_(значения)

  20. EVOLUTION OF RAILS APPLICATION ARCHITECTURE: 14 YEARS IN PRODUCTION

  21. THE PAST

  22. Ruby on Rails to Basecamp 21 Mar 2004 by David

    Heinemeier Hansson
  23. Ruby on Rails to Basecamp 21 Mar 2004 by David

    Heinemeier Hansson
  24. Ruby on Rails to Basecamp 21 Mar 2004 by David

    Heinemeier Hansson
  25. Ruby on Rails to Basecamp 21 Mar 2004 by David

    Heinemeier Hansson
  26. 2008 2009 2010 MAX ACCELERATION

  27. POPULAR GEMS Devise СanCan Will paginate Paperclip Capistrano Friendly Id

    SimleForm 2 Jan 2010 14 Dec 2009 2 Mar 2008 19 Mar 2008 24 Dec 2006 (2009) 26 Aug 2008 6 Feb 2010
  28. None
  29. 2008. PORTLAND https://conferences.oreilly.com/rails2008/public/schedule/full

  30. 2009. LAS VEGAS https://conferences.oreilly.com/rails2009/public/schedule/full

  31. https://www.youtube.com/watch?v=YX3iRjKj7C0

  32. 2013 2014 MAX SPEED

  33. 2013. PORTLAND

  34. 2014. CHICAGO Ultra Light and Maintainable Rails Wizards REFLECTION ON

    CODE
  35. 2014. CHICAGO REFLECTION ON CODE Domain Driven Design and Hexagonal

    Architecture with Rails
  36. 2014. CHICAGO Where did the OO go? Views should be

    objects too! REFLECTION ON CODE
  37. 2014. CHICAGO Make an Event of It REFLECTION ON CODE

  38. 2014. CHICAGO •Ultra Light and Maintainable Rails Wiz… •Domain Driven

    Design and Hexagonal … •Where did the OO go? Views should … •Modeling on the Right Side of the Brain •Make an Event of It •Service Oriented Authenication •Concerns, Decorators, Presenters… •Reading Code Good •Rails as an SOA Client •Refactoring Towards Component… •Authorization in a Service-Oriented … •What is REST? Why is it Part of the … •Service Extraction at Groupon Scale REFLECTION ON CODE
  39. 2014. CHICAGO PERFORMANCE ?

  40. 2014. CHICAGO PERFORMANCE Improve Performance Quick and Cheap: Optimize Memory

    and Upgrade to Ruby 2.1
  41. PERFORMANCE REFLECTION ON CODE 15 1 :

  42. BACK TO THE FUTURE 2018 … And this is our

    future?
  43. 2018. PITTSBURGH

  44. 2018. PITTSBURGH

  45. TODAY

  46. None
  47. None
  48. None
  49. None
  50. Kir Shatrov Rails, Ruby and Kubernetes. Site Reliability Engineer at

    @Shopify. @evilmartians alumni.
  51. None
  52. -------------------------------------------------------------------------------- Language files blank comment code -------------------------------------------------------------------------------- Ruby 13995 271122

    19124 1290867 YAML 1791 8962 1166 559395 JSON 332 77 0 303475 Sass 484 39191 4968 185032 ERB 2584 7208 83 111490 JavaScript 1065 22635 5956 109023 CoffeeScript 667 12130 1003 45734 SQL 4 14 0 28300 TypeScript 455 3794 190 23939 liquid 64 4391 28 21809 …… …… -------------------------------------------------------------------------------- SUM: 21742 376631 38507 2705978 -------------------------------------------------------------------------------- Shopify 30/05/2018 306k commits SUM: 2.7M LOC
  53. WHY U STILL MONOLITHIC?

  54. MONOLITHIC MICROSERVICES >

  55. А ЧЕГО МЫ ДОСТИГНЕМ ЕСЛИ СЕЙЧАС ОСТАНОВИТЬСЯ И НАЧАТЬ РАСПИЛИВАТЬ

    МОНОЛИТ?
  56. КОМПАНИИ ОТКАЗЫВАЮТСЯ ОТ МОНОЛИТА НА РЕЛЬСАХ ПОТОМУ ЧТО РЕЛЬСЫ НЕ

    ПРЕДЛАГАЮТ ХОРОШИХ СПОСОБОВ ОРГАНИЗАЦИИ КОДА НА БОЛЬШОЙ КОДОВОЙ БАЗЕ
  57. У НАС НАКОПИЛСЯ ОЧЕНЬ КРУТОЙ ТУЛИНГ ДЛЯ МОНОЛИТА. К ПРИМЕРУ,

    ТЕСТЫ В 300 ПОТОКОВ ЗА 10 МИНУТ
  58. None
  59. None
  60. > 400

  61. https://stackshare.io/shopify/e-commerce-at-scale-inside-shopifys-tech-stack

  62. SHOW ME YOUR CODE! ROBERT MARTIN

  63. COMPONENT BASED ARCHITECTURE ├── Gemfile ├── Gemfile.lock ├── README.md ├──

    components │ ├── component_a │ │ ├── │ │ ├── │ │ ├── │ │ ├── │ │ └── │ ├── component_b │ │ ├── │ │ ├── │ │ ├── │ │ ├── │ │ └── │ └── component_c │ │ ├── │ │ ├── │ │ ├── │ │ ├── │ │ └── ├── lib ├── test └── etc...
  64. COMPONENT BASED ARCHITECTURE ├── Gemfile ├── Gemfile.lock ├── README.md ├──

    components │ ├── component_a │ │ ├── README.md │ │ ├── app │ │ ├── component.yml │ │ ├── lib │ │ └── test │ ├── component_b │ │ ├── README.md │ │ ├── app │ │ ├── component.yml │ │ ├── lib │ │ └── test │ └── component_c │ │ ├── README.md │ │ ├── app │ │ ├── component.yml │ │ ├── lib │ │ └── test ├── lib ├── test └── etc...
  65. ENTRY POINTS LAYER

  66. НАДО ПОНИМАТЬ, ЧТО ЛЕГАСИ МНОГО И ЕЩЕ НЕ ВСЕ ПЕРЕЕХАЛИ

    НА ENTRY POINTS. МЫ СЧИТАЕМ НЕКИЙ SCORE КАЖДОГО КОМПОНЕНТА И ОЦЕНИВАЕМ КАКИЕ ИЗ НИХ БОЛЕЕ ЛЕГАСИ ЧЕМ ОСТАЛЬНЫЕ
  67. ЕСТЬ NATIVE GEM КОТОРЫЙ ИСПОЛЬЗУЕТ TRACEPOINTS ЧТОБЫ КОНТРОЛИРОВАТЬ ВЫЗОВЫ ВНЕ

    КОМПОНЕНТОВ
  68. ПРОВОДЯТСЯ ВНУТРЕННИЕ МИТАПЫ ДЛЯ ОЦЕНКИ КОНЦЕПЦИИ, СБОРА FEEDBACK’A И ПРЕЗЕНТАЦИИ

    СЛЕДУЮЩИХ ШАГОВ
  69. None
  70. Tom Ward Developer @basecamp, substandard twitter biography writer

  71. THE FIRST BASECAMP WAS STARTED BACK IN 2004, BASECAMP 2

    WAS STARTED IN 2011, AND BC3 IN 2014
  72. 3 VERSIONS OF BASECAMP

  73. 3 VERSIONS OF BASECAMP

  74. 3 VERSIONS OF BASECAMP

  75. STARTING A WHOLE NEW CODEBASE ALLOWS TO USE NEW, BETTER

    WAYS TO STRUCTURE OUR CODE, BASED ON THINGS WE’VE LEARNT BUILDING PREVIOUS ITERATIONS
  76. WE ALSO CREATE AN ENTIRELY NEW ARCHITECTURE, DATABASE MODEL, AND

    MORE WITH EACH ITERATION, AGAIN BASED ON WHAT WE’VE LEARNT
  77. THE PROS FOR OUR CUSTOMERS ARE THAT THEY CAN CONTINUE

    TO USE APPS THEY RELY ON UNTIL ‘THE END OF THE INTERNET’; THEY NEVER HAVE TO UPGRADE TO A NEW VERSION IF THE EXISTING VERSION WORKS FOR THEM
  78. [R]EVOLUTION OF RAILS APPLICATION ARCHITECTURE: 14 YEARS IN PRODUCTION

  79. THE MAJESTIC MONOLITH https://signalvnoise.com/the-majestic-monolith-29166d022228

  80. WE DO HAVE AN INDEPENDENT SECURITY/ AUTHORISATION APP SHARED ACROSS

    ALL OUR PRODUCTS, AS WELL AS A BILLING AND ACCOUNT MANAGEMENT APP
  81. WHEN WILL THE TIME COME FOR THE 4 VERSION?

  82. SOME REASONS MIGHT BE IF THE CODEBASE FEELS IT’S NO

    LONGER FUN TO WORK ON, IF THERE ARE NEW TECHNOLOGIES WE WANT TO COMPLETELY BASE A NEW VERSION AROUND
  83. DO YOU REALLY USE ONLY MVC AND PATTERNS BASED ON

    RAILS (E.G. SERIALIZERS)? WITHOUT ANY OTHER DESIGN PATTERNS LIKE SERVICE OBJECT, PRESENTER, FORM OBJECT AND ETC.
  84. MARTIN FOWLER DO YOU REALLY USE ONLY MVC AND PATTERNS

    BASED ON RAILS (E.G. SERIALIZERS)? WITHOUT ANY OTHER DESIGN PATTERNS LIKE SERVICE OBJECT, PRESENTER, FORM OBJECT AND ETC.
  85. WE TRY TO USE LIGHTWEIGHT CONTROLLERS, SIMPLE MODELS, AND BASIC

    PATTERNS. BUT IT’S NOT POSSIBLE TO IMPLEMENT EVERYTHING WE’D LIKE TO IN THIS WAY
  86. WE ALSO HAVE PRESENTERS, SOME CO- ORDINATING MODELS, AND EVEN

    A FEW TRANSACTION- LIKE OBJECTS FOR COMPLEX THINGS LIKE MOVES AND COPIES
  87. BC3

  88. BC3

  89. None
  90. None
  91. None
  92. HELLO DARKNESS, MY OLD FRIEND NO ONE ANSWERED ME

  93. None
  94. http://tenderlovemaking.com/railsconf2010.pdf Tasty Burgers Aaron Patterson [RELAX MODE ON/]

  95. http://tenderlovemaking.com/railsconf2010.pdf Aaron Patterson Tasty Burgers [RELAX MODE ON/]

  96. None
  97. None
  98. None
  99. None
  100. None
  101. None
  102. None
  103. None
  104. None
  105. None
  106. None
  107. None
  108. None
  109. None
  110. None
  111. None
  112. None
  113. None
  114. None
  115. None
  116. None
  117. None
  118. None
  119. None
  120. None
  121. [/RELAX MODE OFF]

  122. None
  123. Matz (the creator of Ruby), Koichi and Mame (full-time Ruby

    committers at Cookpad) Cookpad Ruby Hack Challenge 2018
  124. AT COOKPAD WE HAVE TWO INDEPENDENT ENGINEERING TEAMS: COOKPAD JAPAN

    AND COOKPAD GLOBAL (WHERE I WORK)
  125. Sebastian Sogamoso Software engineer @Cookpad_Dev. Organizer @RubyConfCo

  126. IN COOKPAD GLOBAL WE HAVE A MONOLITH AND SEVERAL OTHER

    APPLICATIONS THAT COMMUNICATE BETWEEN, BUT I WOULDN’T CALL THEM MICRO SERVICES
  127. WE HAVE CREATED A FEW APPS AS SERVICES THAT WE

    HAVE MERGE INTO OUR MONOLITH AFTER WE’VE SEEN THAT IT INTRODUCED MORE PROBLEMS THAN THE ONES IT FIXED
  128. WE’RE STARTING TO BUILD A BUNCH OF DISTRIBUTED SERVICES RELYING

    ON EVENT SOURCING BECAUSE IT MAKES OUR LIVES EASIER
  129. WE’RE STARTING TO BUILD A BUNCH OF DISTRIBUTED SERVICES RELYING

    ON EVENT SOURCING BECAUSE IT MAKES OUR LIVES EASIER Events? Great!
  130. WE DO CARE A LOT ABOUT OUR ARCHITECTURE, WE HAVE

    BEEN WORKING ON IMPROVING IT
  131. None
  132. Takamichi Yoshikawa Cat allergy [Ex] Software engineer COOKPAD Inc. Sunao

    Hogelog Software engineer di クックパッド
  133. https://speakerdeck.com/adorechic/how-cookpad-shifts-to-microservices

  134. 2011-2012 WE HAVE SOME-INCREDIBLE-HUGE- MONOLITHIC-APPS THAT HAVE SHARED CODES AND

    DBS.
  135. WE USED SYMLINKS AT FIRST. WE EXTRACTED "SHARED" CODES TO

    A DIRECTORY, AND ALL RAILS APPLICATIONS REFERS IT VIA SYMLINK
  136. https://speakerdeck.com/adorechic/how-cookpad-shifts-to-microservices

  137. None
  138. GOT A PROBLEM WITH SHARED CODE - DIFFICULT TO DO

    CHANGES, DIFFICULT TO COORDINATE COMMANDS, DIFFICULT TO TRACK DEPENDENCIES AND APPS HAD MONKEY PATCHING OF SHARED LOGIC
  139. PANTRY

  140. MICROSERVICES API UNCONTROLLED AND NOT UNIFIED. SOLUTION: OWN EXTENSION FOR

    RAILS APP FOR MAKING API - GARAGE!
  141. MICROSERVICES API UNCONTROLLED AND NOT UNIFIED. SOLUTION: OWN EXTENSION FOR

    RAILS APP FOR MAKING API - GARAGE!
  142. https://github.com/cookpad/garage

  143. MANY DIFFERENT MICROSERVICES ARE PRONE TO NETWORK PROBLEMS. SOLUTION: OWN

    GEM EXPEDITOR!
  144. https://github.com/cookpad/expeditor

  145. -------------------------------------------------------------------------------- Language files blank comment code -------------------------------------------------------------------------------- Ruby 1778 34618

    4380 171259 Sass 817 17876 449 110313 Haml 3273 10047 649 83502 YAML 37 450 313 33705 CoffeeScript 296 2246 329 11993 JavaScript 95 832 409 5560 ERB 225 814 51 5293 C 1 86 10 635 Bourne Shell 6 78 77 289 Bourne Again Shell 10 34 0 157 builder 2 4 0 86 Markdown 3 25 0 46 Perl 2 12 0 38 Ruby HTML 1 0 0 12 CSS 1 0 0 1 -------------------------------------------------------------------------------- SUM: 6547 67122 6667 422889 -------------------------------------------------------------------------------- COOKPAD 27/09/2018 THE MOST HUGE APP SUM: 422K LOC
  146. https://blog.envoyproxy.io/service-mesh-and-cookpad-ba4d5d915dbd

  147. https://blog.envoyproxy.io/service-mesh-and-cookpad-ba4d5d915dbd + go-control-plane.

  148. SUMM4RY

  149. RAILS IS AN YOUNG ADULT

  150. RAILS IS LEGO

  151. THE BEST OSS IS BORN FROM PRODUCTION

  152. TAKE CARE OF ARCHITECTURE

  153. BC3 COMPETITION 13

  154. I'M PRETTY SURE IT'S A CAT

  155. cultofmartians.com

  156. @ssnickolay Nikolay Sverchkov @ssnickolay THANK YOU! @evilmartians evilmartians.com/blog