How to Kill a Giant

Drupal 7 has been there so long, and it offered us unprecedented flexibility to build some pretty huge projects -- special kind of beasts that have nothing to do with the original "Drupal-as-a-CMS" paradigm. Think about Drupal sites that behave as OAuth2 servers; or sell content access, or prorate prices and bill them monthly; or news portals that are also exposing the API's for popular mobile apps; or membership databases of really big organizations. Sometimes, these projects would be so big and so crucial to our businesses that there would simply be too much disruption if they went offline.

But Drupal 7 is getting closer to its end of life. We have good migration scenarios for most of the use cases. However, the above giants are marginal cases which cannot simply be migrated to Drupal 8. And even if they could — that may result in building a new giant which would have to be killed once again at Drupal 8's end of life. Sometimes, a transition to something service-oriented may make more sense.

This session is going to suggest and describe a strangler application pattern in a Drupal 7 project. Martin Fowler describes the strangler application as “…a new system around the edges of the old, letting it grow slowly over several years until the old system is strangled”. In our PHP/Drupal realm, that means:

1. decoupling the frontend of a Drupal 7 project;
2. placing a middleware API proxy between the frontend and Drupal;
3. finally replacing individual parts of the old system with the newly developed ones

This approach promises zero downtime and makes no assumptions on what will be the resulting tech stack. The decoupled frontend can be a GatsbyJS project, or a React app, or even a static HTML served from the middleware. And the middleware itself can be a NodeJS/Express app, or a Golang API proxy, or something third. The backend can be replaced with a Drupal 8 monolith, or multiple Drupal 8 projects, or with a full-blown microservice architecture.

The sweetest part of the pattern is that even if you make an error in the architecture of your backend, which will require serious refactoring, you will be able to correct it without the downtime.

Transcript

1. How to Kill a Giant Branislav Bujisic
2. Drupal / NodeJS / Recurring billing Product Engineer at Platform.sh Branislav Bujišić
5. Bad Accent Included
6. You have an idea for a startup • SaaS, PaaS or WaaS to disrupt the market • You are a PHP programmer, but you know the related stuff (JS, HTML, you did some Java and C while studying...) • Free choice of the tech stack • Shortest possible time-to-market
7. Technology considerations Language? Framework? The architecture?
8. Your choices are so 2019! Because you are building a greenfield project
9. The reality of a 
 brownfield project Somebody else made the architectural decisions in 2011
10. The cutting-edge of 2011 • Drupal 7
 Rapid development. Great API's and entity system FTW. Mature community bound to sustain the product with the contributed code. • Monolithic architecture
 The less moving parts the better • Tightly coupled frontend
 Interactive XHTML frontend with AJAX (AJAJ/AJAH)
11. ...8 years later, the business succeeded • Lots of accumulated knowledge and custom code and consequential idiosyncracies. • People who grew as a programmers on a single piece of code and consequential empathetic relationship of developers towards the code. • Large customer base, with a steady growth.
12. Two problems: An accumulated technical debt and a Drupal 7 end-of-life
13. PSA-2019-02-25 Drupal 7 will reach end-of-life in November of 2021
14. Technical debt • Frequent abuse of the hook system and all the alterations of the state • Mutually dependent code • Mishmash of config, content and code that defines a single functionalities
15. The consequences
 of the accumulated technical debt • Frequent regressions • Velocity decreasing • Slow and painful onboarding
17. The design stamina hypothesis "Is it worth the effort to design software well?" -- Martin Fowler, 2007
19. It's time to replace the giant
20. The Drupal Way 1. Build the new version of the software (Drupal 8) 2. Write the migration scripts 3. Close the old site for the mainenance 4. Run the migrations 5. Update the DNS to the new site
21. The Problem of Giants • Designing from the ground up may be too complicated • Big risk in just redeveloping the system from the scratch • The data is often too complex to be simply migrated away • The customer base is so big that you cannot just shut the service down for maintenance
22. The Problem of Humans • We are bad at planning • We are bad at ensuring enough funds for the big overhauls • We make errors
23. Johannes Seitz, Architecting the uncertain - Getting started with Agile Software Architecture
 https://printhelloworld.de/posts/iteration-zero-architecture/
24. The Strangler Application Pattern
25. –Martin Fowler “[...] gradually create a new system around the edges of the old, letting it grow slowly over several years until the old system is strangled.” "StranglerFigApplication", Martin Fowler, 2004
 https://martinfowler.com/bliki/StranglerFigApplication.html
27. 3 steps of the strangler application: 3 steps of the strangler application: • Transform, create a parallel new site • Coexist, leave the existing site where it is for a while. Redirect from the existing to the new site as a new feature was built • Eliminate, remove the old functionality from the existing site as traffic is redirected away "Apply the Strangler Application pattern to microservices applications",
 Kyle Brown, IBM developer portal, 2017
 https://developer.ibm.com/articles/cl-strangler-application-pattern-microservices-apps-trs
28. Even more practical description of the steps • First, add a proxy, which sits between the legacy application and the user. Initially, this proxy doesn’t do anything but pass all traffic, unmodified, to the application. • Then, add new service (with its own database(s) and other supporting infrastructure) and link it to the proxy. Implement the first new page in this service. Then allow the proxy to serve traffic to that page. • Add more pages, more functionality and potentially more services. Open up the proxy to the new pages and services. Repeat until all required functionality is handled by the new stack. • The monolith no longer serves traffic and can be switched off. "The Strangler pattern in practice", Michiel Rook, 2016
 https://www.michielrook.nl/2016/11/strangler-pattern-practice
29. The client The monolith
30. The proxy The client The monolith
31. The proxy The client The monolith A service
32. The Proxy RewriteEngine On # Serve feature from new service for internal network RewriteCond expr "%{HTTP:X-FORWARDED-FOR} -ipmatch '192.168.0.1/24'" RewriteRule ^/feature/(.*)$ ${NEW_SERVICE_URL}/$1 [P,L] # Proxy everything else to legacy application RewriteRule ^/(.*) ${LEGACY_URL}/$1 [P] ProxyPassReverse / ${LEGACY_URL}/ The proxy The client /(.*) /feature/(.*)
33. The proxy The client The monolith A service A service
34. The proxy The client The monolith A service A service A service
35. When will this work, and when not? • This pattern works really well with web- or api- based monoliths. It’s much less applicable to desktop or mobile apps. • This pattern requires a standardized URL structure. • You really need a consistent UI. • Do you really want to replace one monolith with another?
36. The Advantages • Zero downtime • Reduced risk of failures due to frequent releases • Lesser cognitive load for the dev team • Ability to engage more mutually independent teams • Ability to choose the best tech stack for the each of the components
37. A Whole New World of Possibilities
38. Data Driven Refactoring • Do we really know how are the clients using our (legacy) app? • How easy it is to collect the usage data with Drupal? • How to prioritize the refactor? Where are the bottlenecks?
39. Data Driven Refactoring The proxy The client The monolith ELK?
40. Decoupled frontend • Solves the problem of inconsistent or outdated UI • Allows the frontend engineering team to grow independently • The users see the benefits immediately • As a free benefit: your users get the new feature -- an API to your SaaS
41. Decoupled Frontend The proxy The client The monolith RESTful API restful module
 services module
 swagger-php
42. Decoupled Frontend The proxy The client The monolith RESTful API OpenAPI 3 documentation OA3
43. Decoupled Frontend The proxy The client The monolith RESTful API OA3 React, VUE, Angular, etc. Decoupled frontend
44. Decoupled Frontend The proxy The client RESTful API OA3 Decoupled frontend RESTful API RESTful API
45. The Proxy as a Middleware • What if you want to consolidate the API's you are consuming and exposing to your clients? • Or you simply want to write the cutting edge frontend while you know for a fact that the legacy code cannot sustain it (e.g. GraphQL and Drupal 6/7)? • Or you want to avoid the oauth2 authentication on every single microservice • Or the URLs on your old app were not consistent The proxy GraphQL REST SOAP API Decoupled frontend Auth Server
46. The Proxy as a Middleware • Stateless NodeJS app • Runs the user authentication • Rewrites and routes the graphQL requests • rewrites the responses to consistent graphQL • single caching control layer • scales easily • the middleware fully decouples the frontend from the backend services The middleware GraphQL REST SOAP API Decoupled frontend Auth Server GraphQL
47. The Best Tool for the Job™ "With a system composed of multiple, collaborating services, we can decide to use different technologies inside each one. This allows us to pick the right tool for each job, rather than having to select a more standardized, one-size-fits-all approach that often ends up being the lowest common denominator."
 
 Sam Newman, "Building Microservices"
 2014

48. The Best Tool for the Job™ The proxy Decoupled frontend Authentication Big Data Billing Content Mgmt
49. Practical example:
 Platform.sh
50. User Drupal 7 monolith EU region US region You're good It's in the US region Show me my project Here it is Who's this guy? He's good Where's this project? ... Authorize
51. User Drupal 7 monolith EU region US region Authorize You're good It's in the US region Show me my project Here it is Who's this guy? He's good Where's this project? ... AngularJS UI AngularJS UI Drupal theme Auth bottleneck abstract it away
52. User Drupal 7 monolith EU region US reg Authorize You're good Show me my project Here it is Who's this guy? He's good API proxy It's in the US region Where's the project? Show me my project Here it is
53. User Drupal 7 monolith EU region US reg Authorize You're good Show me my project Here it is Who's this guy? He's good API proxy It's in the US region Where's the project? Show me my project Here it is headless headless headless decoupled frontend
54. API proxy Decoupled frontend Drupal 7
 monolith EU region US region ... Oauth2
 server Billing service Project discovery service Drupal Commerce Symfony Go Python Python Python Go Super efficient Cache is the only state Easy to horizontally scale Geographic load balancing
55. Agony included
56. You will hurt if you don't have a big enough team
57. You will hurt if you
 diversify your tech too much
58. You will hurt if you
 make a distributed monolyth
59. You will hurt if you
 don't monitor your services
60. You will hurt
 witout a good devops
61. You will hurt
 without good CI/CD
62. You will hurt
63. because microservices
 are complicated
64. because microservices
 are risky
65. minimize the risk
66. Thank you
67. Questions? 
 Comments?
 Rotten tomatoes? Branislav Bujisic
 branislav@platform.sh @bbujisic