Architecture: The Next Generation - Railsconf 2017

Slide 1

Slide 1 text

HI. @HIIMTAYLORJONES

Slide 2

Slide 2 text

ARCHITECTURE THE NEXT GENERATION THIS TALK IS CALLED: @HIIMTAYLORJONES

Slide 3

Slide 3 text

HI. MY NAME IS TAYLOR JONES. @HIIMTAYLORJONES YOU CAN FIND ME ON THE INTERNET UNDER @HIIMTAYLORJONES

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

AN INTRODUCTION SO WHAT DO I ACTUALLY DO? @HIIMTAYLORJONES I Write Things on the Internet @codeship hiimtaylorjones.com @hiimtaylorjones

Slide 7

Slide 7 text

STAR TREK @HIIMTAYLORJONES THIS TALK ISN'T REALLY ABOUT OR WHICH ENTERPRISE SHIP IS THE BEST

Slide 8

Slide 8 text

SORRY @HIIMTAYLORJONES

Slide 9

Slide 9 text

AN INTRODUCTION SO WHO DO YOU ACTUALLY WORK FOR? @HIIMTAYLORJONES

Slide 10

Slide 10 text

AN INTRODUCTION WHAT EXACTLY DOES IZEA DO? @HIIMTAYLORJONES Content Marketing Social Media Analytics General Data Analytics Enterprise Marketing Systems

Slide 11

Slide 11 text

IZEA HAS A GROWING MOUNTAIN OF DATA TO SORT THROUGH @HIIMTAYLORJONES HONESTLY, WE ALL DO.

Slide 12

Slide 12 text

IZEA’S BUSINESS DEPENDS ON A LOT OF EXTERNAL APIS AND SUCH @HIIMTAYLORJONES

Slide 13

Slide 13 text

AN INTRODUCTION WHAT TOOLS DOES IZEA USE? @HIIMTAYLORJONES A Whole Lot of Rails A Whole Lot of EmberJS A Good Bit of Java A Decent Amount of Python

Slide 14

Slide 14 text

THE BALANCE OF LANGUAGES IN IZEA’S STACK IS ALWAYS CHANGING @HIIMTAYLORJONES

Slide 15

Slide 15 text

RAILS ARCHITECTURE @HIIMTAYLORJONES THIS TALK IS MOSTLY ABOUT WHERE ITS BEEN AND WHERE ITS GOING

Slide 16

Slide 16 text

TECHNICAL DEBT. @HIIMTAYLORJONES THIS TALK IS ALSO ABOUT AND HOW ARCHITECTURE CAN CREATE OR RELIEVE IT

Slide 17

Slide 17 text

IZEA HAS TECHNICAL DEBT @HIIMTAYLORJONES

Slide 18

Slide 18 text

HAS TECHNICAL DEBT @HIIMTAYLORJONES YOUR COMPANY

Slide 19

Slide 19 text

ARCHITECTURE IS THE ROOT OF A LOT OF OUR TECHNICAL DEBT @HIIMTAYLORJONES I BELIEVE

Slide 20

Slide 20 text

HOW EXACTLY DOES ARCHITECTURE CREATE TECHNICAL DEBT? @HIIMTAYLORJONES WE OFTEN BECOME SO FOCUSED ON BUILDING STUFF, THAT WE FORGET TO CLEAN UP OUR MESS. FIXING TECHNICAL DEBT IS OFTEN VIEWED AS A SECONDARY FOCUS. AFTER ALL, FEATURES MAKE MONEY

Slide 21

Slide 21 text

HAVE THINGS ALWAYS BEEN THIS WAY? @HIIMTAYLORJONES DOES MODERN ARCHITECTURE HAVE MORE DEBT? DID WE LOSE SOMETHING ALONG THE WAY?

Slide 22

Slide 22 text

THE PAST LETS LOOK BACK FOR A MOMENT AT MAYBE WE CAN FIND SOMETHING WITHIN IT

Slide 23

Slide 23 text

WHAT WAS EARLY WEB DEVELOPMENT LIKE? @HIIMTAYLORJONES Applications Were Hard To Deploy Code Organization Was Inconsistent License-Driven Development Environments were hard to Duplicate

Slide 24

Slide 24 text

BUT THEN A NEW CONTENDER APPEARED WHAT COULD IT BE?

Slide 25

Slide 25 text

CONVENTION OVER CONFIGURATION @HIIMTAYLORJONES What does it mean? Rails employs this idea of Why does it matter?

Slide 26

Slide 26 text

RAILS TOOK THE WEB DEVELOPMENT PROCESS AND REFACTORED IT. @HIIMTAYLORJONES

Slide 27

Slide 27 text

REFACTORS AREN’T AUTOMATICALLY GOOD BY NATURE @HIIMTAYLORJONES

Slide 28

Slide 28 text

WHAT KIND OF REFACTOR IS RAILS? @HIIMTAYLORJONES SO,

Slide 29

Slide 29 text

HOW HAS RAILS EVOLVED? This is a kitten. !

Slide 30

Slide 30 text

SOME PERSPECTIVE - THE DEVELOPMENT OF RAILS THE ORIGIN OF RAILS @HIIMTAYLORJONES Based on an extraction of existing product. Its creation was rooted in a real-world use case. Eventually found its home as an open-source project.

Slide 31

Slide 31 text

SOME PERSPECTIVE - THE DEVELOPMENT OF RAILS MERB @HIIMTAYLORJONES MERB was a reaction to the problems in Rails “Clean Room” implementation Speed Scalability Modularity API Tools

Slide 32

Slide 32 text

SOME PERSPECTIVE - THE DEVELOPMENT OF RAILS MERB + RAILS @HIIMTAYLORJONES Rails and Merb decided to work together. This was pitched as a good thing for Ruby on the Web? In short, I believe it worked out really well. Some ideas didn’t quite survive the merger.

Slide 33

Slide 33 text

POST- MERGER RAILS @HIIMTAYLORJONES

Slide 34

Slide 34 text

ARCHITECTURE BEGINS TO BECOME SLOW, MESSY, AND A PAIN TO MAINTAIN AFTER TIME, @HIIMTAYLORJONES RECOGNIZING CODE SMELLS, BAD PATTERNS, AND OTHER HARMFUL THINGS ISN’T EASY.

Slide 35

Slide 35 text

WHAT KILLED SMALLTALK WAS THAT IT WAS EASY TO MAKE A MESS Robert C. Martin (Quoting Ward Cunningham) - RailsConf 2010 SOME PERSPECTIVE - HOW RAILS HAS EVOLVED @HIIMTAYLORJONES

Slide 36

Slide 36 text

SOME PERSPECTIVE - HOW HAS RAILS EVOLVED HOW TO MAKE A MESS IN RAILS ▸ Lack of distribution between Model, View, and Controller layer ▸ Misunderstanding Rails Queries - The N+1 Problem ▸ Misunderstanding Ruby’s API ▸ Common code smells @HIIMTAYLORJONES

Slide 37

Slide 37 text

IT BEGAN TO ADDRESS THE ISSUES THAT PLAGUED THE COMMUNITY THE MOST AS RAILS GREW, @HIIMTAYLORJONES THIS INVOLVED MAKING THE FRAMEWORK MORE MODULAR, EFFICIENT, AND SCALABLE

Slide 38

Slide 38 text

IS RAILS AS USEFUL AS IT ONCE WAS? @HIIMTAYLORJONES

Slide 39

Slide 39 text

IS RAILS AS RELEVANT AS IT ONCE WAS? @HIIMTAYLORJONES

Slide 40

Slide 40 text

IS RAILS FINISHED? @HIIMTAYLORJONES

Slide 41

Slide 41 text

SHOULD WE ALL JUST GO HOME AND START USING SOMETHING ELSE? @HIIMTAYLORJONES

Slide 42

Slide 42 text

@HIIMTAYLORJONES

Slide 43

Slide 43 text

AND THE CASE FOR SWITCHING AWAY FROM RAILS TWITTER

Slide 44

Slide 44 text

TWITTER SUDDENLY BECAME A REALLY BIG DEAL. @HIIMTAYLORJONES

Slide 45

Slide 45 text

LESSONS FROM THE PAST - TWITTER THE BIRTH OF TWITTER ▸ When the product was initially created, it was written in Rails. ▸ Twitter, for a moment, became this poster child for Rails in popular culture. ▸ Then, rumors started to emerge that their Engineering team was starting to shift away from their Rails stack. @HIIMTAYLORJONES

Slide 46

Slide 46 text

WHY DID TWITTER START MOVING AWAY FROM RAILS? @HIIMTAYLORJONES

Slide 47

Slide 47 text

LESSONS FROM THE PAST - TWITTER A PROBLEM OF REINVENTING SEARCH ▸ When Twitter was rewriting their search service, they found some frustrations with their tech stack, Specifically Rails. ▸ Instead of trying to better leverage Rails / Ruby for the task, they decided to build it from the ground up with Scala. ▸ Twitter has a lot of data to store and comb through ▸ The easiest way to escape tech debt is rewriting from scratch. ▸ Twitter gradually dismantled their Rails instance and wrote their own iterations of those services. @HIIMTAYLORJONES

Slide 48

Slide 48 text

IN SHORT, THEY BROKE DOWN SOMETHING BIGGER INTO UNIQUE SERVICES THAT WORK TOGETHER @HIIMTAYLORJONES THIS SOUNDS FAMILIAR

Slide 49

Slide 49 text

MICROSERVICES @HIIMTAYLORJONES ALL ABOARD THE HYPE TRAIN

Slide 50

Slide 50 text

HOW HAVE MICROSERVICES BENEFITED OTHER COMPANIES? @HIIMTAYLORJONES WHAT’S THE BIG DEAL?

Slide 51

Slide 51 text

ALL-IN ON MICROSERVICES @HIIMTAYLORJONES

Slide 52

Slide 52 text

NETFLIX IS A REALLY PREVALENT SUCCESS STORY AND ADVOCATE FOR MICROSERVICES @HIIMTAYLORJONES

Slide 53

Slide 53 text

@HIIMTAYLORJONES LESSONS FROM THE PAST - NETFLIX ▸ Netflix took an early bet on Microservices. Its worked out pretty well for them so far. ▸ Frees developers to create new features in whatever language they want. NETFLIX HAS REALLY GOOD ARCHITECTURE HOW DO YOU TEST THIS KIND OF STUFF?

Slide 54

Slide 54 text

NETFLIX’S SIMIAN ARMY

Slide 55

Slide 55 text

NETFLIX CREATED NEW TESTING STANDARDS IN ORDER TO PROPERLY EXERCISE THEIR SUITE OF MICRO SERVICES @HIIMTAYLORJONES

Slide 56

Slide 56 text

THE SIMIAN ARMY CREATES INCREDIBLY HIGH STAKES FOR DEVELOPERS @HIIMTAYLORJONES

Slide 57

Slide 57 text

WHEN WE CHANGE ARCHITECTURES OUR PROCESS HAS TO CHANGE @HIIMTAYLORJONES THE TAKEAWAY:

Slide 58

Slide 58 text

WE TRADE THE COMFORT OF UNDERSTANDING FOR THE PROMISE OF HAPPIER DEVELOPMENT @HIIMTAYLORJONES

Slide 59

Slide 59 text

EVERY ARCHITECTURE IS A BRAVE NEW WORLD! @HIIMTAYLORJONES

Slide 60

Slide 60 text

HOW DO THESE SUCCESS STORIES AND LESSONS RELATE TO RAILS ARCHITECTURE? @HIIMTAYLORJONES

Slide 61

Slide 61 text

DESIGNING ARCHITECTURE WITH RAILS LET’S TALK ABOUT RAILSCONF EDITION

Slide 62

Slide 62 text

MONOLITHIC DESIGN @HIIMTAYLORJONES

Slide 63

Slide 63 text

MONOLITHIC DESIGN WHAT EXACTLY IS A MONOLITH? @HIIMTAYLORJONES Views Middleware Models Database

Slide 64

Slide 64 text

MONOLITHIC DESIGN WHAT’S THE DEAL WITH MONOLITHS ▸ Rails historically has skewed towards a Monolithic tendency. ▸ This tendency has served the framework well ▸ It has also been the subject of a lot of hate mail ▸ “The Majestic Monolith” @HIIMTAYLORJONES

Slide 65

Slide 65 text

MONOLITHS ARE ROOTED IN UNIFORMITY @HIIMTAYLORJONES

Slide 66

Slide 66 text

MONOLITHS ARE ROOTED IN VERY TIGHT COUPLING @HIIMTAYLORJONES

Slide 67

Slide 67 text

IZEA AND MONOLITHIC ARCHITECTURE @HIIMTAYLORJONES

Slide 68

Slide 68 text

MONOLITHIC DESIGN HOW IZEA HANDLES MONOLITHIC APPLICATIONS ▸ We learned the hard way about how jobs should be balanced and queued properly ▸ We also realized the need for a greater means of queued processing. ▸ To accomplish this, we started leveraging things like AWS Lambda’s alongside other smaller services ▸ In short, we started to dabble in Microservices @HIIMTAYLORJONES

Slide 69

Slide 69 text

MICROSERVICE DESIGN @HIIMTAYLORJONES MICORSERVICE MICORSERVICE MICORSERVICE MICORSERVICE MICORSERVICE MICORSERVICE MICORSERVICE MICORSERVICE MICORSERVICE MICORSERVICE MICORSERVICE MICORSERVICE MICORSERVICE

Slide 70

Slide 70 text

MICROSERVICE DESIGN REMEMBER MONOLITHS? @HIIMTAYLORJONES Views Middleware Models Database

Slide 71

Slide 71 text

MICROSERVICE DESIGN THESE ARE MICROSERVICES @HIIMTAYLORJONES Views Models Database Middleware

Slide 72

Slide 72 text

MICROSERVICE DESIGN NATURALLY LOOKS CLEANER @HIIMTAYLORJONES

Slide 73

Slide 73 text

MICROSERVICE DESIGN HOW DOES MICROSERVICE DESIGN WITH RAILS WORK? ▸ The most popular implementation is Rails as an API ▸ The API utility of Rails is largely due to the rise of front- end frameworks and popularity of Mobile Applications ▸ There are certainly use cases for extracting the view layer and middleware for other cases. @HIIMTAYLORJONES

Slide 74

Slide 74 text

MICROSERVICE DESIGN HOW IZEA HANDLES MICROSERVICE APPLICATIONS ▸ AWS is a powerful ally for us ▸ We create services that help us delegate heavier system tasks ▸ We leverage diverse, but stable languages for this ▸ Java ▸ Python ▸ SQL @HIIMTAYLORJONES

Slide 75

Slide 75 text

MICROSERVICE DESIGN HOW IZEA HANDLES MICROSERVICE APPLICATIONS ▸ Stable languages are incredibly important ▸ Ultimately, you want to be able to hire developers that can easily maintain and improve your applications. ▸ Newer languages eventually need to be proven by someone. However, taking smaller risks is important. @HIIMTAYLORJONES

Slide 76

Slide 76 text

MICROSERVICES ARE ROOTED IN MODULARITY @HIIMTAYLORJONES

Slide 77

Slide 77 text

MICROSERVICES ARE ROOTED IN LOOSE COUPLING @HIIMTAYLORJONES

Slide 78

Slide 78 text

MICROSERVICE ECOSYSTEMS @HIIMTAYLORJONES

Slide 79

Slide 79 text

MICROSERVICE ECOSYSTEM DESIGN REMEMBER MONOLITHS? @HIIMTAYLORJONES Views Middleware Models Database

Slide 80

Slide 80 text

MICROSERVICE ECOSYSTEM DESIGN REMEMBER MICROSERVICES? @HIIMTAYLORJONES Views Models Database Middleware

Slide 81

Slide 81 text

MICROSERVICE ECOSYSTEM DESIGN THIS IS A MICROSERVICE ECOSYSTEM @HIIMTAYLORJONES Views Middleware Models Database

Slide 82

Slide 82 text

MICROSERVICE ECOSYSTEMS ARE ROOTED IN MODULARITY @HIIMTAYLORJONES

Slide 83

Slide 83 text

MICROSERVICE ECOSYSTEMS ARE ROOTED IN TIGHTER COUPLING @HIIMTAYLORJONES

Slide 84

Slide 84 text

MOST OF OUR APPLICATIONS EXIST WITHIN THIS SPACE @HIIMTAYLORJONES

Slide 85

Slide 85 text

@HIIMTAYLORJONES

Slide 86

Slide 86 text

MICROSERVICE ECOSYSTEMS - DOCKER DOCKER IS A MICROSERVICE ECOSYSTEM ▸ You can create smaller docker containers of any configuration of your choosing. ▸ For best performance, you dockerize each part of your ecosystem and then wrap that within a Docker container. ▸ You then have a means to orchestrate your entire ecosystem and a common language for all the components to talk with. @HIIMTAYLORJONES

Slide 87

Slide 87 text

@HIIMTAYLORJONES

Slide 88

Slide 88 text

MICROSERVICE ECOSYSTEMS - AMAZON WEB SERVICES AWS IS A MICROSERVICE ECOSYSTEM ▸ You have all of these services built in different languages and stacks. ▸ Depending on how a user’s service is set up, certain elements of those services are coordinated by AWS. ▸ AWS is an ecosystem. Their services are Microservices. @HIIMTAYLORJONES

Slide 89

Slide 89 text

@HIIMTAYLORJONES

Slide 90

Slide 90 text

CAN RAILS FACILITATE A SYSTEM OF MICOSERVICES? @HIIMTAYLORJONES

Slide 91

Slide 91 text

IN MANY WAYS, RAILS IS A MICROSERVICE ECOSYSTEM. @HIIMTAYLORJONES

Slide 92

Slide 92 text

OK. I SEE WHAT YOU’RE SAYING BUT WE REALLY WANT TO SWITCH ARCHITECTURE BECAUSE REASONS @HIIMTAYLORJONES

Slide 93

Slide 93 text

ONE ARCHITECTURE DOES NOT FIT ALL RAILS APPS @HIIMTAYLORJONES AFTER ALL,

Slide 94

Slide 94 text

LETS SAY YOU WANT OUT OF YOUR CURRENT ARCHITECTURE? @HIIMTAYLORJONES HOW SHOULD WE APPROACH A SITUATION LIKE THIS?

Slide 95

Slide 95 text

RAILS EDITION HOW TO SWITCH ARCHITECTURE WITHOUT CRUSHING YOUR HOPES AND DREAMS

Slide 96

Slide 96 text

KILL THE HYPE. @HIIMTAYLORJONES First,

Slide 97

Slide 97 text

MIGRATING TO A NEW ARCHITECTURE - KILL THE HYPE KILL THE HYPE ▸ We’re prone to exalt shiny new things. ▸ Innovation and new ideas are really exciting! ▸ There’s a difference between implementing something in a side project and converting an entire production- app to a new architecture. ▸ Younger != Better @HIIMTAYLORJONES

Slide 98

Slide 98 text

RESET YOUR BIAS. THINGS HAVE CHANGED. @HIIMTAYLORJONES

Slide 99

Slide 99 text

MIGRATING TO A NEW ARCHITECTURE - KILL THE HYPE KILL THE HYPE ▸ What you’re doing is working just fine. ▸ Every method has a balance that’s required of developers. ▸ Think about what kind of architecture best supports your team size and skillsets. @HIIMTAYLORJONES

Slide 100

Slide 100 text

FIND YOUR PACING. @HIIMTAYLORJONES secondly,

Slide 101

Slide 101 text

MIGRATING TO A NEW ARCHITECTURE - FIND YOUR PACING FIND YOUR PACING ▸ We talk a lot about the concept of DevOps ▸ DevOps matters a lot in your architecture ▸ Your development process matters even more ▸ The cadence by which your team goes, determines what kind of architectures you can best implement. @HIIMTAYLORJONES

Slide 102

Slide 102 text

KNOWLEDGE IS POWER @HIIMTAYLORJONES finally,

Slide 103

Slide 103 text

THERE IS AN INEQUALITY OF KNOWLEDGE IN PROGRAMMING @HIIMTAYLORJONES

Slide 104

Slide 104 text

KNOWLEDGE IS POWER - SHARING THE WEALTH THERE’S AN INEQUALITY IN SOFTWARE DESIGN KNOWLEDGE ▸ We often only let the “Professionals” discuss, design, and improve architecture. ▸ What happens when those people retire, quit, move on? ▸ How do we train new developers in the ways of software design? @HIIMTAYLORJONES

Slide 105

Slide 105 text

WE ALL COME FROM DIFFERENT PLACES AND BACKGROUNDS. @HIIMTAYLORJONES

Slide 106

Slide 106 text

CODING BOOTCAMPS @HIIMTAYLORJONES FORMAL EDUCATION SELF-TAUGHT

Slide 107

Slide 107 text

EMBRACE YOUR HISTORY. OWN IT. IMPLEMENT IT IN YOUR WORK. @HIIMTAYLORJONES

Slide 108

Slide 108 text

DON’T LET ANYONE SHAME YOU ABOUT YOUR PAST @HIIMTAYLORJONES

Slide 109

Slide 109 text

HOW SHOULD I IMPLEMENT THIS KIND OF STUFF IN A TEAM ENVIRONMENT? @HIIMTAYLORJONES

Slide 110

Slide 110 text

KNOWLEDGE IS POWER - SHARING THE WEALTH IMPLEMENTING IT IN A TEAM SETTING 1.Design with yourself or a small core team 2.Bring the proposed design to a small, but diverse group of Engineers. 3.Go back and make edits with your core team. 4.Bring the edited design to your whole engineering team and explain it. Ask for questions, feedback, and clarification. 5.Workshop it once more. 6.Go back to the whole Engineering team for a final presentation. @HIIMTAYLORJONES

Slide 111

Slide 111 text

CORE Workshop Whole Engineering Team 4, 6 1, 3, 5 2 Charts and Stuff

Slide 112

Slide 112 text

KNOWLEDGE IS POWER - SHARING THE WEALTH THERE’S AN INEQUALITY IN SOFTWARE DESIGN KNOWLEDGE ▸ Effective Software teams teach each other ▸ Effective Software Developers have the desire to learn. ▸ Some teams only have a select few of these attributes. ▸ This disconnect is creating developers who are uncomfortable outside their own company zone. @HIIMTAYLORJONES

Slide 113

Slide 113 text

STORY OF MY LIFE EDITION ALRIGHT, LETS WRAP IT UP DUDE. YOU’VE BEEN TALKING FOR LIKE FOREVER

Slide 114

Slide 114 text

ARCHITECTURE IS A HOUSE @HIIMTAYLORJONES DEBT PILES UP IN DIFFERENT PLACES DEPENDING ON WHAT KIND OF ARCHITECTURE WE UTILIZE

Slide 115

Slide 115 text

No content

Slide 116

Slide 116 text

No content

Slide 117

Slide 117 text

ARCHITECTURE IS AN ESCAPE HATCH @HIIMTAYLORJONES SOMETIMES, ITS NECESSARY OR EASIER TO ESCAPE DEBT BY SWITCHING TO A NEW “HOUSE”

Slide 118

Slide 118 text

ARCHITECTURE IS A HOUSE @HIIMTAYLORJONES ULTIMATELY, WE NEED TO LEARN HOW TO PROPERLY CLEAN AND MAINTAIN OUR HOUSE. MOVING IS EXPENSIVE.

Slide 119

Slide 119 text

CONFRONT TECHNICAL DEBT @HIIMTAYLORJONES FIND A PLACE IN YOUR PROCESS THAT ALLOWS FOR ADDRESSING DEBT WHILE PUSHING FORWARD FEATURES.

Slide 120

Slide 120 text

CONFRONT TECHNICAL DEBT @HIIMTAYLORJONES IT MIGHT TAKE LONGER BUT IT’LL ULTIMATELY DELIVER CLEANER & MORE STABLE FEATURES TO YOUR CUSTOMERS

Slide 121

Slide 121 text

CONFRONT TECHNICAL DEBT @HIIMTAYLORJONES ARCHITECTURE SHOULD FEEL COMFORTABLE. ITS AIM IS ULTIMATELY DEVELOPER HAPPINESS.