Building Resilient Microservices APIs with REST and API Gateway

Building a microservices based architecture from an existing monolithic application is a non-trivial task with a lot of challenges. You will likely end up in with a completely different system in terms of architecture, components and more importantly the API surface you’re exposing; and that means breaking changes for your clients. Is there a better way? We will deconstruct a monolithic application and refactor it into two microservices using a naive approach, taming all the problems that will rise, and discover how RESTful architecture combined with an API Gateway are the way to go. Then we’ll fully make the switch from a monolithic application to microservices based version (and viceversa) without breaking the clients at all.

Transcript

1. Building Resilient Microservices APIs with REST and API Gateway Previously known as REST, for REAL! 1 Vincenzo Chianese, @d3dvincent
2. Who am I? 2 • vincenzo@express-gateway.io • https://vncz.js.org/ • https://twitter.com/D3DVincent • https://github.com/XVincentX Auth0 Ambassador
3. The Challenge
4. Express Gateway Monolith vs. Microservices 02/27/2018 4 Business requirements change rapidly AND continuously – so the need for shipping updated version of your app are increasing exponentially. A microservice app Continuous updating in parts. A monolithic app All or nothing — a big, giant thing that can’t fit through the shipping door. vs.
5. Express Gateway Microservices: 
 an ideal that’s been out there for a long time Decades of best practices in the modern web era Web-scale experiences (Amazon, Google, Netflix, etc) Cloud native technologies — Microservices as individual processes — Processes encapsulated in containers — Containers orchestrated 02/27/2018 5
6. Express Gateway Microservices and APIs The natural evolution of microservices are APIs to the “N” client - client traditionally was built by the company, but now can be built by anyone APIs = business leverage and amplification 02/27/2018 6
7. Express Gateway Uniform API 02/27/2018 7 { "id": 12340124, "name": "Vincenzo", "surname": "Chianese", "last_update": 69879841, "external_id": 1741 } { "Identifier": 12340124, "Name": "Vincenzo", "Surname": "Chianese", "LastUpdate": 69879841, "ExternalId": 1741 } Javascript client Go client
8. Architectural styles 8 Express Gateway 02/27/2018 Big central window A lot of towers Roof color Two towers on top
9. Express Gateway An Architectural Style, Not a Fixed Blueprint Microservices are the new norm – but also more of an architectural style There are too many ways to slice and dice Examples Problem 1: Services can be discreetly defined in many ways – Business capability – Domain and subdomains – Model Problem 2: Deployment units can vary in granularity – How many services per host? – What services per host? 02/27/2018 9
10. Microservices: a new set of problems to deal with 10 2. Propagation of server implementation details to the client 1. Duplication of code among services 3. A lot more we’re not going to look into now Express Gateway 02/27/2018
11. Express Gateway API Gateway Pattern Problem 1: Disparate Microservices Solution: Centralized Middleware Public APIs • Authentication • Security • Traffic Control • Ops • Logging • Transformations • Etc API Gateway 02/27/2018 11 Client Only for internal usage Only for specific partners AWS Lambda, etc
12. Express Gateway API Gateway Pattern (cont’d) Problem 2: Different Granularity Required Solution: Orchestration • Optimised endpoints • Request collapsing • And more API Gateway Invoices DB schema Customers DB schema Orders DB schema 02/27/2018 12 Client
13. REST 13 Express Gateway 02/27/2018 Roy Fielding — dissertation published on 2000 A system that has the following constraints: 
 – Client/Server – Layered – Code on Demand – Stateless – Uniform interface Rationale on deployment of HTTP/1.1
14. REST definition Everything is a resource! 14 No. No! Express Gateway 02/27/2018
15. Uniform interface 15 Resource identification in requests – URI Resource manipulation through representations – HTTP Content Negotiation mechanism Self descriptive messages – Headers + payload contain all the data you need Hypermedia as the Engine of Application State – HTML is an Hypermedia format Express Gateway 02/27/2018
16. Hypermedia examples <span>Name: Vincenzo </span> <span>Surname: Chianese </span> <a type="invoice" href="https: //customers.apitest.lan/invoices/">Get Invoices </a> 16 <h1>Main menu </h1> <a href="https: //customers.apitest.lan/">List customers </a> <a href="https: //customers.apitest.lan/">Add new customer </a> <a href="https: //invoices.apitest.lan/">List invoices </a> <a href="https: //invoices.apitest.lan/">Add invoices </a> <h1>Customers </h1> <div> <div>Vincenzo Chianese <a href="https: //customers.apitest.lan/1" type="customerDetail"> </a> </div> <div>Customer 2 <a href="https: //customers.apitest.lan/2" type="customerDetail"> </a> </div> <div>Customer 3 <a href="https: //customers.apitest.lan/3" type="customerDetail"> </a> </div> </div> Express Gateway 02/27/2018
17. Real HTML example <header class="site-header"> <section class="wrapper"> <a href="/" class="logo"> <svg id="Layer_1" data-name="Layer 1" xmlns="http: // www.w3.org/2000/svg" xmlns:xlink="http: // www.w3.org/1999/xlink" viewBox="0 0 575 88"><defs><style>.eglogo-1{fill:#128595}.eglogo-2{fill:url(#linear-gradient)}.eglogo-3{fill:url(#linear-gradient-2)}.eglogo-4{fill:url(#linear- gradient-3)}.eglogo-5{fill:url(#linear-gradient-4)}.eglogo-6{fill:#ffc50a} </style><linearGradient id="linear-gradient" x1="33.87" y1="82.81" x2="219.98" y2="-138.99" gradientTransform="matrix(1 0 0 -1 0 90)" gradientUnits="userSpaceOnUse"><stop offset="0" stop-color="#116c7b"> </ stop><stop offset=".05" stop-color="#1e7989"> </stop><stop offset=".17" stop-color="#3d97ab"> </stop><stop offset=".3" stop-color="#55afc5"> </stop><stop offset=".43" stop-color="#66c0d8"> </stop><stop offset=".57" stop-color="#71cbe3"> </stop><stop offset=".72" stop-color="#74cee7"> </ stop><stop offset=".73" stop-color="#74cee7"> </stop><stop offset="1" stop-color="#74cee7"> </stop> </linearGradient><linearGradient id="linear-gradient-2" x1="31.11" y1="80.5" x2="217.22" y2="-141.3" xlink:href="#linear-gradient"> </linearGradient><linearGradient id="linear-gradient-3" x1="10.58" y1="63.27" x2="196.69" y2="-158.52" xlink:href="#linear-gradient"> </linearGradient><linearGradient id="linear-gradient-4" x1="18.88" y1="70.23" x2="204.99" y2="-151.56" xlink:href="#linear-gradient"> </linearGradient> </defs><title>Artboard 1 </title><path class="eglogo-1" d="M122 28c-9.11 0-15.73 7.45-15.73 17.73v.5c0 10.71 6.33 17.63 16.13 17.63 7.13 0 12.41-3.74 14.12-10l.15-.46h-2l-.14.29c-1.48 5.34-5.92 8.41-12.18 8.41-8.94 0-14.13-6-14.27-16.35h28.6V44.6c.05-5.36-1.46-9.74-4.37-12.68A14.17 14.17 0 0 0 122 28zm0 1.85c7.83 0 12.63 5.29 12.87 14.15h-26.64c.55-8.49 6.07-14.17 13.77-14.17zM166 28.57l.15-.19h-2.31L152.5 43.79l-11.54-15.41h-2.31l12.59 16.82-13.39 18.22h2.31l12.34-16.81 12.47 16.63v.18h2.48L153.76 45.2 166 28.57zM187.9 28c-6.85 0-12 4.7-14.27 9.53v-9.15h-2v47h2v-20.7a15.64 15.64 0 0 0 14.27 9.24c9.14 0 15.52-7.25 15.52-17.62v-.5c-.1-10.33-6.62-17.8-15.52-17.8zm13.58 17.7v.5c0 9.5-5.46 15.88-13.58 15.88-9.27 0-14.27-8.18-14.27-15.87v-.5a16 16 0 0 1 4.48-11.77 13.94 13.94 0 0 1 9.79-4.21c9.33 0 13.58 8.27 13.58 15.97zM210.83 35.35v-7h-2v35h2V43.9c0-3.15.83-13.48 11.57-13.87h. 33v-1.76h-.43a12.82 12.82 0 0 0-11.47 7.08zM239.2 28c-9.11 0-15.73 7.45-15.73 17.73v.5c0 10.71 6.33 17.63 16.13 17.63 7.13 0 12.4-3.74 14.12-10l.15-.46h-2l-.14.29c-1.48 5.34-5.92 8.41-12.18 8.41-8.94 0-14.13-6-14.27-16.35h28.6V44.6c.05-5.33-1.46-9.71-4.39-12.66A14.14 14.14 0 0 0 239.2 28zm-13.77 16c.55-8.47 6.06-14.15 13.77-14.15s12.63 5.26 12.87 14.15zM270.72 44.58c-8.22-1.49-10.6-3.19-10.6-7.58 0-4.23 3.81-7.08 9.48-7.08s9 2.42 9.88 7.18v.32h1.85V37c-.9-5.83-5-8.91-11.72-8.91s-11.44 3.66-11.44 8.91c0 6.17 4 8 12 9.42 7.78 1.38 9.8 3 9.8 8 0 4.77-4 8-9.87 8-3.75 0-10.21-1.18-11.37-9.08V53h-1.85v.44c.61 6.67 5.71 10.81 13.32 10.81 7 0 11.82-4 11.82-9.72-.02-6.74-3.8-8.64-11.3-9.95zM298.42 44.58c-8.22-1.49-10.6-3.19-10.6-7.58 0-4.23 3.81-7.08 9.47-7.08s9 2.42 9.88 7.18v.32H309V37c-.9-5.83-5-8.91-11.72-8.91s-11.4 3.66-11.4 8.91c0 6.17 4 8 12 9.42 7.78 1.38 9.8 3 9.8 8 0 4.77-4 8-9.87 8-3.75 0-10.21-1.18-11.37-9.08V53h-1.85v.44c.61 6.67 5.71 10.81 13.32 10.81 7 0 11.82-4 11.82-9.72-.01-6.74-3.81-8.64-11.31-9.95zM355.67 36.53A14.24 14.24 0 0 0 342.1 28c-9.4 0-16.22 7.12-16.22 16.92v.5c0 11.69 7.9 16.92 15.72 16.92a15.41 15.41 0 0 0 14.08-8.83v7.29c0 10.72-7.44 13-13.67 13-3.57 0-12-.86-13.47-8.78v-.33h-2v.44a11.48 11.48 0 0 0 4.22 7.15c2.79 2.22 6.59 3.35 11.3 3.35 10 0 15.53-5.23 15.53-14.72V28.38h-2zm0 8.37v.5c0 9.9-7.08 15.07-14.08 15.07-8.37 0-13.77-5.92-13.77-15.07v-.5c0-8.93 5.87-15.17 14.27-15.17 8.13 0 13.58 6.09 13.58 15.17zM378.2 28c-7.16 0-12.31 3.93-13.12 10v.42h1.85v-.3c1-5.27 5.11-8.3 11.28-8.3 6.83 0 9.88 3.26 9.88 10.57v3.27h-9.39c-10.1 0-15.23 3.54-15.23 10.53 0 6.06 4.68 9.83 12.23 9.83 7.34 0 11-3.83 12.38-5.82v5.32h2V40.4C390 32.16 386 28 378.2 28zm9.88 17.35v7.75c0 4.78-5.78 9-12.37 9s-10.28-2.83-10.28-8c0-6.07 4.09-8.77 13.28-8.77zM407 62.08c-3.83 0-5.78-2.31-5.78-6.87v-25h10.1v-1.73h-10.2v-7.8h-2v7.8H394v1.75h5.2v25c0 5.63 2.74 8.72 7.73 8.72a14.74 14.74 0 0 0 4.31-.64l.24-.07.27-.13v-1.88l-.46.15a14.06 14.06 0 0 1-4.29.7zM429.6 28c-9.11 0-15.72 7.45-15.72 17.73v.5c0 10.71 6.33 17.63 16.13 17.63 7.13 0 12.41-3.74 14.12-10l.15-.46h-1.95l-.14.29c-1.49 5.31-5.93 8.39-12.19 8.39-8.94 0-14.13-6-14.27-16.35h28.6V44.6c.05-5.33-1.46-9.71-4.39-12.66A14.15 14.15 0 0 0 429.6 28zm-13.77 16c.55-8.47 6.06-14.15 13.77-14.15s12.63 5.26 12.87 14.15zM480.5 60.49l-10.18-32.03-.03-.08h-1.48L458.5 60.49l-10.18-32.03-.03-.08h-2.06l11.04 34.74v.3h2.22l10.01-31.31 9.79 31.05.13.26h2.07l11.13-34.49.14-.55h-1.95L480.5 60.49zM509 28c-7.16 0-12.31 3.93-13.12 10v.42h1.85v-.3c1-5.27 5.11-8.3 11.28-8.3 6.83 0 9.88 3.26 9.88 10.57v3.27h-9.39c-10.1 0-15.22 3.54-15.22 10.53 0 6.06 4.68 9.83 12.22 9.83 7.34 0 11-3.83 12.38-5.82v5.32h2V40.4C520.83 32.16 516.85 28 509 28zm-2.5 34.1c-6.63 0-10.28-2.83-10.28-8 0-6.07 4.09-8.77 13.28-8.77h9.38v7.77c0 4.78-5.79 8.98-12.38 8.98zM551.52 28.38l-12.42 30-12.77-29.71v-.29h-2.12l13.85 32.42-5.97 14.35-.08.18h2.07l19.23-46.37.29-.58h-2.08z"> </path><path class="eglogo-2" d="M85.9 42.8l-1.8-3.1c-6-5.5-13.9-11.2-24.8-11.6-9.1-.3-14.3 3.2-21 7.6-.7.5-1.5 1-2.2 1.5a85.61 85.61 0 0 1 8 4.1c1.1.6 2.2 1.2 3.2 1.7.6.3 1.1.6 1.7.9 13.2-5.1 25.8-3.6 37.1-.5a4.27 4.27 0 0 1 .1 2.5c-10.5-3-22.3-4.6-34.4-.5 1.3.7 2.4 1.4 3.5 2.1 4.1 2.5 7 4.2 11.8 5.4 5.9 1.5 11.4-.3 17-2.9l1.8-3.2a3.63 3.63 0 0 0 0-4z"> </path><path class="eglogo-3" d="M33.8 36.2c1.2-.8 2.3-1.5 3.4-2.2 7-4.6 12.5-8.2 22.2-7.9 9.1.3 16.1 4 21.7 8.4l-11.5-20a4.08 4.08 0 0 0-3.6-2.1H33.4a4.2 4.2 0 0 0-3.6 2.1L18.1 35c.9-.2 1.8-.4 2.9-.6a26.65 26.65 0 0 1 12.8 1.8z"> </path><path class="eglogo-4" d="M21.2 36.4a26.78 26.78 0 0 0-4.7 1.2l-3 5.2a3.58 3.58 0 0 0-.4 1.1 46.36 46.36 0 0 0 18.2-6.1c.1-.1.2-.1.3-.2a23 23 0 0 0-10.4-1.2z"> </path><path class="eglogo-5" d="M66.7 54.9c-5.1-1.3-8.1-3.1-12.4-5.7-1.5-.9-3.1-1.8-5-2.9-.9.3-1.7.7-2.6 1-7.6 3-15.4 6.2-23.2 6.2a25.73 25.73 0 0 1-6.7-.9L15 49.5c.7.3 1.4.6 2 .8 9.4 2.7 19.3-1.3 28.8-5.1.2-.1.5-.2.7-.3-.1 0-.1-.1-.2-.1L43 43a96.88 96.88 0 0 0-9.1-4.6c-.6.3-1.1.7-1.7 1A48.15 48.15 0 0 1 13 45.8a3 3 0 0 0 .4 1L29.7 75a4.08 4.08 0 0 0 3.6 2.1h32.6a4.2 4.2 0 0 0 3.6-2.1l12.9-22a30.36 30.36 0 0 1-11 2.5 19.54 19.54 0 0 1-4.7-.6z"> </path><path class="eglogo-6" d="M86.1 43.4c-11.2-3.1-23.9-4.6-37.1.5 0 .1-.1.1-.2.1-. 7.3-1.4.6-2.2.9-.2.1-.5.2-.7.3-9.6 3.8-19.4 7.8-28.8 5.1-.6-.2-1.3-.5-2-.8l1.8 3.2a25.73 25.73 0 0 0 6.7.9c7.8 0 15.6-3.1 23.2-6.2.9-.3 1.7-.7 2.6-1 .1 0 .2-.1.3-.1.7-.3 1.5-.5 2.2-.8 12.2-4.1 23.9-2.5 34.4.5a5.65 5.65 0 0 0-.2-2.6z"> </path> </svg> </a> <nav class="menu"> <span class="search-button-wrapper nav-element"> <label class="search-icon" for="search-check-box"> </label> <input id="search-check-box" type="checkbox"> <span class="algolia-autocomplete" style="position: relative; display: inline-block; direction: ltr;"><input type="text" class="search-text-box ds-input" placeholder="Search…" autocomplete="off" spellcheck="false" role="combobox" aria-autocomplete="list" aria-expanded="false" aria-owns="algolia-autocomplete-listbox-0" dir="auto" style="position: relative; vertical-align: top;"><pre aria-hidden="true" style="position: absolute; visibility: hidden; white-space: pre; font-family: system-ui; font-size: 11px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; word-spacing: 0px; letter-spacing: normal; text-indent: 0px; text-rendering: auto; text-transform: none;"> </pre><span class="ds-dropdown-menu" role="listbox" id="algolia-autocomplete-listbox-0" style="position: absolute; top: 100%; z-index: 100; display: none; left: 0px; right: auto;"><div class="ds-dataset-1"> </div> </span> </span> </span> <span class="nav-element"><a href="/about">About </a> </span> <span class="nav-element"><a href="/getting-started">Getting Started </a> </span> <span class="nav-element"> <a href="/docs" class="nav-sub-menu-trigger">Documentation </a> </span> <span class="nav-element"><a href="/resources">Resources </a> </span> <span class="nav-element"><a href="/blog">Blog </a> </span> <span class="github-button-wrapper nav-element"><iframe allowtransparency="true" scrolling="no" frameborder="0" src="https: //buttons.github.io/buttons.html#href=https%3A%2F%2Fgithub.com%2FExpressGateway%2Fexpress-gateway&amp;aria-label=Star%20ExpressGateway%2Fexpress- gateway%20on%20GitHub&amp;data-text=Star" style="width: 51.5px; height: 20px; border: none;"> </iframe> </span> </nav> 17 Express Gateway 02/27/2018
18. Can we make it with JSON? 18 Multiple JSON and Hypermedia friendly formats HAL + JSON – Resources – Links Siren – Entities – Links – Actions RESTful JSON – Links – Resources Express Gateway 02/27/2018
19. RESTful JSON specs 1. JSON objects MAY include a url property to indicate a link to itself 19 2. JSON objects MAY append _url to properties to indicate related links Express Gateway 02/27/2018 { "_id": "59fdb32de1e514001212698a", "name": "Vincenzo", "surname": "Chianese", "invoices_url": "http: //invoices.apitest.lan:81/59fdb32de1e514001212698a/invoices", "url": "http: //customers.apitest.lan:81/59fdb32de1e514001212698a" }, Where I am Where can I go from here
20. Introducing Express Gateway
21. Express Gateway Express Gateway Why? No matter what style - Microservices and APIs need Super Glue Everyone knows Express Already Proven - countless people have rolled their own What? A dynamic Express outfitted with Middleware for the API Gateway use case When? July 2017 02/27/2018 21
22. Express Gateway Features and Benefits Centralized Declarative Configuration - Manage, Control and Visualize API Consumer Management (users, apps, credentials, tokens etc.) – Get What You Need Most Express Middleware Driven Plugins – Getting Started is Dead Simple Security – Peace of Mind Basic Authorization Key Authorization oAuth 2.0 CORS JSON Web Token Quality of Services –Your Way Dynamic Routing Simple Logger Proxy with Load Balancer (coming soon) Rate Limiter JavaScript Expression 02/27/2018 22
23. Express Gateway Overview 02/27/2018 23
24. Express Gateway Overview 02/27/2018 24
25. Express Gateway 7/27/17 25 Wrap Up
26. Express Gateway Get Involved! How is this different? A seamless and unified platform for developing, managing and orchestrating microservices and APIs insanely fast – managing is just the tip of the iceberg! Installing is Dead Simple 02/27/2018 26 https://express-gateway.io $ npm install express-gateway
27. Express Gateway Quick Plug - LunchBadger Build, Manage, Deploy your Microservices as APIs 
 through One Unified Experience: Express Gateway - Management Express (LoopBack) - Composition Docker and Kubernetes - Cloud Native Portable Runtime 02/27/2018 27
28. Express Gateway Join Us! 02/27/2018 28 Express Gateway www.express-gateway.io github.com/
 ExpressGateway/express-gateway @express_gateway facebook.com/ExpressJSGateway Lunchbadger www.lunchbadger.com @lunchbadger facebook.com/LunchBadger
29. Express Gateway Thank You! 02/27/2018 29 Questions?