Evolution of Rails application architecture: 14 years in production

by Nikolay Sverchkov

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

evilmartians.com

Slide 5

Slide 5 text

WHAT AWAITS US NOW?

Slide 6

Slide 6 text

CLOSING KEYNOTE

Slide 7

Slide 7 text

WE LOVE CATS!

Slide 8

Slide 8 text

TATARSTAN

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

THE 2017/2018 CHAMPIONS

Slide 11

Slide 11 text

15 DAKAR RALLY WINS

Slide 12

Slide 12 text

34TH BY POPULATION CITY IN RUSSIA

Slide 13

Slide 13 text

34TH BY POPULATION CITY IN RUSSIA THE THIRD AMONG THE CITIES THAT ARE NOT THE CENTRES OF THE SUBJECTS OF FEDERATIONS

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

https://ru.wikipedia.org/wiki/Эволюция_(значения)

Slide 19

Slide 19 text

https://ru.wikipedia.org/wiki/Эволюция_(значения)

Slide 20

Slide 20 text

EVOLUTION OF RAILS APPLICATION ARCHITECTURE: 14 YEARS IN PRODUCTION

Slide 21

Slide 21 text

THE PAST

Slide 22

Slide 22 text

Ruby on Rails to Basecamp 21 Mar 2004 by David Heinemeier Hansson

Slide 23

Slide 23 text

Ruby on Rails to Basecamp 21 Mar 2004 by David Heinemeier Hansson

Slide 24

Slide 24 text

Ruby on Rails to Basecamp 21 Mar 2004 by David Heinemeier Hansson

Slide 25

Slide 25 text

Ruby on Rails to Basecamp 21 Mar 2004 by David Heinemeier Hansson

Slide 26

Slide 26 text

2008 2009 2010 MAX ACCELERATION

Slide 27

Slide 27 text

POPULAR GEMS Devise СanCan Will paginate Paperclip Capistrano Friendly Id SimleForm 2 Jan 2010 14 Dec 2009 2 Mar 2008 19 Mar 2008 24 Dec 2006 (2009) 26 Aug 2008 6 Feb 2010

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

2008. PORTLAND https://conferences.oreilly.com/rails2008/public/schedule/full

Slide 30

Slide 30 text

2009. LAS VEGAS https://conferences.oreilly.com/rails2009/public/schedule/full

Slide 31

Slide 31 text

https://www.youtube.com/watch?v=YX3iRjKj7C0

Slide 32

Slide 32 text

2013 2014 MAX SPEED

Slide 33

Slide 33 text

2013. PORTLAND

Slide 34

Slide 34 text

2014. CHICAGO Ultra Light and Maintainable Rails Wizards REFLECTION ON CODE

Slide 35

Slide 35 text

2014. CHICAGO REFLECTION ON CODE Domain Driven Design and Hexagonal Architecture with Rails

Slide 36

Slide 36 text

2014. CHICAGO Where did the OO go? Views should be objects too! REFLECTION ON CODE

Slide 37

Slide 37 text

2014. CHICAGO Make an Event of It REFLECTION ON CODE

Slide 38

Slide 38 text

2014. CHICAGO •Ultra Light and Maintainable Rails Wiz… •Domain Driven Design and Hexagonal … •Where did the OO go? Views should … •Modeling on the Right Side of the Brain •Make an Event of It •Service Oriented Authenication •Concerns, Decorators, Presenters… •Reading Code Good •Rails as an SOA Client •Refactoring Towards Component… •Authorization in a Service-Oriented … •What is REST? Why is it Part of the … •Service Extraction at Groupon Scale REFLECTION ON CODE

Slide 39

Slide 39 text

2014. CHICAGO PERFORMANCE ?

Slide 40

Slide 40 text

2014. CHICAGO PERFORMANCE Improve Performance Quick and Cheap: Optimize Memory and Upgrade to Ruby 2.1

Slide 41

Slide 41 text

PERFORMANCE REFLECTION ON CODE 15 1 :

Slide 42

Slide 42 text

BACK TO THE FUTURE 2018 … And this is our future?

Slide 43

Slide 43 text

2018. PITTSBURGH

Slide 44

Slide 44 text

2018. PITTSBURGH

Slide 45

Slide 45 text

TODAY

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

Kir Shatrov Rails, Ruby and Kubernetes. Site Reliability Engineer at @Shopify. @evilmartians alumni.

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

-------------------------------------------------------------------------------- Language files blank comment code -------------------------------------------------------------------------------- Ruby 13995 271122 19124 1290867 YAML 1791 8962 1166 559395 JSON 332 77 0 303475 Sass 484 39191 4968 185032 ERB 2584 7208 83 111490 JavaScript 1065 22635 5956 109023 CoffeeScript 667 12130 1003 45734 SQL 4 14 0 28300 TypeScript 455 3794 190 23939 liquid 64 4391 28 21809 …… …… -------------------------------------------------------------------------------- SUM: 21742 376631 38507 2705978 -------------------------------------------------------------------------------- Shopify 30/05/2018 306k commits SUM: 2.7M LOC

Slide 53

Slide 53 text

WHY U STILL MONOLITHIC?

Slide 54

Slide 54 text

MONOLITHIC MICROSERVICES >

Slide 55

Slide 55 text

А ЧЕГО МЫ ДОСТИГНЕМ ЕСЛИ СЕЙЧАС ОСТАНОВИТЬСЯ И НАЧАТЬ РАСПИЛИВАТЬ МОНОЛИТ?

Slide 56

Slide 56 text

КОМПАНИИ ОТКАЗЫВАЮТСЯ ОТ МОНОЛИТА НА РЕЛЬСАХ ПОТОМУ ЧТО РЕЛЬСЫ НЕ ПРЕДЛАГАЮТ ХОРОШИХ СПОСОБОВ ОРГАНИЗАЦИИ КОДА НА БОЛЬШОЙ КОДОВОЙ БАЗЕ

Slide 57

Slide 57 text

У НАС НАКОПИЛСЯ ОЧЕНЬ КРУТОЙ ТУЛИНГ ДЛЯ МОНОЛИТА. К ПРИМЕРУ, ТЕСТЫ В 300 ПОТОКОВ ЗА 10 МИНУТ

Slide 58

Slide 58 text

No content

Slide 59

Slide 59 text

No content

Slide 60

Slide 60 text

> 400

Slide 61

Slide 61 text

https://stackshare.io/shopify/e-commerce-at-scale-inside-shopifys-tech-stack

Slide 62

Slide 62 text

SHOW ME YOUR CODE! ROBERT MARTIN

Slide 63

Slide 63 text

COMPONENT BASED ARCHITECTURE ├── Gemfile ├── Gemfile.lock ├── README.md ├── components │ ├── component_a │ │ ├── │ │ ├── │ │ ├── │ │ ├── │ │ └── │ ├── component_b │ │ ├── │ │ ├── │ │ ├── │ │ ├── │ │ └── │ └── component_c │ │ ├── │ │ ├── │ │ ├── │ │ ├── │ │ └── ├── lib ├── test └── etc...

Slide 64

Slide 64 text

COMPONENT BASED ARCHITECTURE ├── Gemfile ├── Gemfile.lock ├── README.md ├── components │ ├── component_a │ │ ├── README.md │ │ ├── app │ │ ├── component.yml │ │ ├── lib │ │ └── test │ ├── component_b │ │ ├── README.md │ │ ├── app │ │ ├── component.yml │ │ ├── lib │ │ └── test │ └── component_c │ │ ├── README.md │ │ ├── app │ │ ├── component.yml │ │ ├── lib │ │ └── test ├── lib ├── test └── etc...

Slide 65

Slide 65 text

ENTRY POINTS LAYER

Slide 66

Slide 66 text

НАДО ПОНИМАТЬ, ЧТО ЛЕГАСИ МНОГО И ЕЩЕ НЕ ВСЕ ПЕРЕЕХАЛИ НА ENTRY POINTS. МЫ СЧИТАЕМ НЕКИЙ SCORE КАЖДОГО КОМПОНЕНТА И ОЦЕНИВАЕМ КАКИЕ ИЗ НИХ БОЛЕЕ ЛЕГАСИ ЧЕМ ОСТАЛЬНЫЕ

Slide 67

Slide 67 text

ЕСТЬ NATIVE GEM КОТОРЫЙ ИСПОЛЬЗУЕТ TRACEPOINTS ЧТОБЫ КОНТРОЛИРОВАТЬ ВЫЗОВЫ ВНЕ КОМПОНЕНТОВ

Slide 68

Slide 68 text

ПРОВОДЯТСЯ ВНУТРЕННИЕ МИТАПЫ ДЛЯ ОЦЕНКИ КОНЦЕПЦИИ, СБОРА FEEDBACK’A И ПРЕЗЕНТАЦИИ СЛЕДУЮЩИХ ШАГОВ

Slide 69

Slide 69 text

No content

Slide 70

Slide 70 text

Tom Ward Developer @basecamp, substandard twitter biography writer

Slide 71

Slide 71 text

THE FIRST BASECAMP WAS STARTED BACK IN 2004, BASECAMP 2 WAS STARTED IN 2011, AND BC3 IN 2014

Slide 72

Slide 72 text

3 VERSIONS OF BASECAMP

Slide 73

Slide 73 text

3 VERSIONS OF BASECAMP

Slide 74

Slide 74 text

3 VERSIONS OF BASECAMP

Slide 75

Slide 75 text

STARTING A WHOLE NEW CODEBASE ALLOWS TO USE NEW, BETTER WAYS TO STRUCTURE OUR CODE, BASED ON THINGS WE’VE LEARNT BUILDING PREVIOUS ITERATIONS

Slide 76

Slide 76 text

WE ALSO CREATE AN ENTIRELY NEW ARCHITECTURE, DATABASE MODEL, AND MORE WITH EACH ITERATION, AGAIN BASED ON WHAT WE’VE LEARNT

Slide 77

Slide 77 text

THE PROS FOR OUR CUSTOMERS ARE THAT THEY CAN CONTINUE TO USE APPS THEY RELY ON UNTIL ‘THE END OF THE INTERNET’; THEY NEVER HAVE TO UPGRADE TO A NEW VERSION IF THE EXISTING VERSION WORKS FOR THEM

Slide 78

Slide 78 text

[R]EVOLUTION OF RAILS APPLICATION ARCHITECTURE: 14 YEARS IN PRODUCTION

Slide 79

Slide 79 text

THE MAJESTIC MONOLITH https://signalvnoise.com/the-majestic-monolith-29166d022228

Slide 80

Slide 80 text

WE DO HAVE AN INDEPENDENT SECURITY/ AUTHORISATION APP SHARED ACROSS ALL OUR PRODUCTS, AS WELL AS A BILLING AND ACCOUNT MANAGEMENT APP

Slide 81

Slide 81 text

WHEN WILL THE TIME COME FOR THE 4 VERSION?

Slide 82

Slide 82 text

SOME REASONS MIGHT BE IF THE CODEBASE FEELS IT’S NO LONGER FUN TO WORK ON, IF THERE ARE NEW TECHNOLOGIES WE WANT TO COMPLETELY BASE A NEW VERSION AROUND

Slide 83

Slide 83 text

DO YOU REALLY USE ONLY MVC AND PATTERNS BASED ON RAILS (E.G. SERIALIZERS)? WITHOUT ANY OTHER DESIGN PATTERNS LIKE SERVICE OBJECT, PRESENTER, FORM OBJECT AND ETC.

Slide 84

Slide 84 text

MARTIN FOWLER DO YOU REALLY USE ONLY MVC AND PATTERNS BASED ON RAILS (E.G. SERIALIZERS)? WITHOUT ANY OTHER DESIGN PATTERNS LIKE SERVICE OBJECT, PRESENTER, FORM OBJECT AND ETC.

Slide 85

Slide 85 text

WE TRY TO USE LIGHTWEIGHT CONTROLLERS, SIMPLE MODELS, AND BASIC PATTERNS. BUT IT’S NOT POSSIBLE TO IMPLEMENT EVERYTHING WE’D LIKE TO IN THIS WAY

Slide 86

Slide 86 text

WE ALSO HAVE PRESENTERS, SOME CO- ORDINATING MODELS, AND EVEN A FEW TRANSACTION- LIKE OBJECTS FOR COMPLEX THINGS LIKE MOVES AND COPIES

Slide 87

Slide 87 text

BC3

Slide 88

Slide 88 text

BC3

Slide 89

Slide 89 text

No content

Slide 90

Slide 90 text

No content

Slide 91

Slide 91 text

No content

Slide 92

Slide 92 text

HELLO DARKNESS, MY OLD FRIEND NO ONE ANSWERED ME

Slide 93

Slide 93 text

No content

Slide 94

Slide 94 text

http://tenderlovemaking.com/railsconf2010.pdf Tasty Burgers Aaron Patterson [RELAX MODE ON/]

Slide 95

Slide 95 text

http://tenderlovemaking.com/railsconf2010.pdf Aaron Patterson Tasty Burgers [RELAX MODE ON/]

Slide 96

Slide 96 text

No content

Slide 97

Slide 97 text

No content

Slide 98

Slide 98 text

No content

Slide 99

Slide 99 text

No content

Slide 100

Slide 100 text

No content

Slide 101

Slide 101 text

No content

Slide 102

Slide 102 text

No content

Slide 103

Slide 103 text

No content

Slide 104

Slide 104 text

No content

Slide 105

Slide 105 text

No content

Slide 106

Slide 106 text

No content

Slide 107

Slide 107 text

No content

Slide 108

Slide 108 text

No content

Slide 109

Slide 109 text

No content

Slide 110

Slide 110 text

No content

Slide 111

Slide 111 text

No content

Slide 112

Slide 112 text

No content

Slide 113

Slide 113 text

No content

Slide 114

Slide 114 text

No content

Slide 115

Slide 115 text

No content

Slide 116

Slide 116 text

No content

Slide 117

Slide 117 text

No content

Slide 118

Slide 118 text

No content

Slide 119

Slide 119 text

No content

Slide 120

Slide 120 text

No content

Slide 121

Slide 121 text

[/RELAX MODE OFF]

Slide 122

Slide 122 text

No content

Slide 123

Slide 123 text

Matz (the creator of Ruby), Koichi and Mame (full-time Ruby committers at Cookpad) Cookpad Ruby Hack Challenge 2018

Slide 124

Slide 124 text

AT COOKPAD WE HAVE TWO INDEPENDENT ENGINEERING TEAMS: COOKPAD JAPAN AND COOKPAD GLOBAL (WHERE I WORK)

Slide 125

Slide 125 text

Sebastian Sogamoso Software engineer @Cookpad_Dev. Organizer @RubyConfCo

Slide 126

Slide 126 text

IN COOKPAD GLOBAL WE HAVE A MONOLITH AND SEVERAL OTHER APPLICATIONS THAT COMMUNICATE BETWEEN, BUT I WOULDN’T CALL THEM MICRO SERVICES

Slide 127

Slide 127 text

WE HAVE CREATED A FEW APPS AS SERVICES THAT WE HAVE MERGE INTO OUR MONOLITH AFTER WE’VE SEEN THAT IT INTRODUCED MORE PROBLEMS THAN THE ONES IT FIXED

Slide 128

Slide 128 text

WE’RE STARTING TO BUILD A BUNCH OF DISTRIBUTED SERVICES RELYING ON EVENT SOURCING BECAUSE IT MAKES OUR LIVES EASIER

Slide 129

Slide 129 text

WE’RE STARTING TO BUILD A BUNCH OF DISTRIBUTED SERVICES RELYING ON EVENT SOURCING BECAUSE IT MAKES OUR LIVES EASIER Events? Great!

Slide 130

Slide 130 text

WE DO CARE A LOT ABOUT OUR ARCHITECTURE, WE HAVE BEEN WORKING ON IMPROVING IT

Slide 131

Slide 131 text

No content

Slide 132

Slide 132 text

Takamichi Yoshikawa Cat allergy [Ex] Software engineer COOKPAD Inc. Sunao Hogelog Software engineer di クックパッド

Slide 133

Slide 133 text

https://speakerdeck.com/adorechic/how-cookpad-shifts-to-microservices

Slide 134

Slide 134 text

2011-2012 WE HAVE SOME-INCREDIBLE-HUGE- MONOLITHIC-APPS THAT HAVE SHARED CODES AND DBS.

Slide 135

Slide 135 text

WE USED SYMLINKS AT FIRST. WE EXTRACTED "SHARED" CODES TO A DIRECTORY, AND ALL RAILS APPLICATIONS REFERS IT VIA SYMLINK

Slide 136

Slide 136 text

https://speakerdeck.com/adorechic/how-cookpad-shifts-to-microservices

Slide 137

Slide 137 text

No content

Slide 138

Slide 138 text

GOT A PROBLEM WITH SHARED CODE - DIFFICULT TO DO CHANGES, DIFFICULT TO COORDINATE COMMANDS, DIFFICULT TO TRACK DEPENDENCIES AND APPS HAD MONKEY PATCHING OF SHARED LOGIC

Slide 139

Slide 139 text

PANTRY

Slide 140

Slide 140 text

MICROSERVICES API UNCONTROLLED AND NOT UNIFIED. SOLUTION: OWN EXTENSION FOR RAILS APP FOR MAKING API - GARAGE!

Slide 141

Slide 141 text

MICROSERVICES API UNCONTROLLED AND NOT UNIFIED. SOLUTION: OWN EXTENSION FOR RAILS APP FOR MAKING API - GARAGE!

Slide 142

Slide 142 text

https://github.com/cookpad/garage

Slide 143

Slide 143 text

MANY DIFFERENT MICROSERVICES ARE PRONE TO NETWORK PROBLEMS. SOLUTION: OWN GEM EXPEDITOR!

Slide 144

Slide 144 text

https://github.com/cookpad/expeditor

Slide 145

Slide 145 text

-------------------------------------------------------------------------------- Language files blank comment code -------------------------------------------------------------------------------- Ruby 1778 34618 4380 171259 Sass 817 17876 449 110313 Haml 3273 10047 649 83502 YAML 37 450 313 33705 CoffeeScript 296 2246 329 11993 JavaScript 95 832 409 5560 ERB 225 814 51 5293 C 1 86 10 635 Bourne Shell 6 78 77 289 Bourne Again Shell 10 34 0 157 builder 2 4 0 86 Markdown 3 25 0 46 Perl 2 12 0 38 Ruby HTML 1 0 0 12 CSS 1 0 0 1 -------------------------------------------------------------------------------- SUM: 6547 67122 6667 422889 -------------------------------------------------------------------------------- COOKPAD 27/09/2018 THE MOST HUGE APP SUM: 422K LOC

Slide 146

Slide 146 text

https://blog.envoyproxy.io/service-mesh-and-cookpad-ba4d5d915dbd

Slide 147

Slide 147 text

https://blog.envoyproxy.io/service-mesh-and-cookpad-ba4d5d915dbd + go-control-plane.

Slide 148

Slide 148 text

SUMM4RY

Slide 149

Slide 149 text

RAILS IS AN YOUNG ADULT

Slide 150

Slide 150 text

RAILS IS LEGO

Slide 151

Slide 151 text

THE BEST OSS IS BORN FROM PRODUCTION

Slide 152

Slide 152 text

TAKE CARE OF ARCHITECTURE

Slide 153

Slide 153 text

BC3 COMPETITION 13

Slide 154

Slide 154 text

I'M PRETTY SURE IT'S A CAT

Slide 155

Slide 155 text

cultofmartians.com

Slide 156

Slide 156 text

@ssnickolay Nikolay Sverchkov @ssnickolay THANK YOU! @evilmartians evilmartians.com/blog