Slide 1

Slide 1 text

Domain Service Aggregators A Structured Approach to Microservice Composition Caoilte O’Connor @caoilte_oconnor

Slide 2

Slide 2 text

. INTRO Based on a true story...

Slide 3

Slide 3 text

“ You stand on the shoulders of giants How do you build Microservices? http://www.thoughtworks.com/insights/blog/microservices-nutshell http://www.se-radio.net/2014/10/episode-213-james-lewis-on-microservices

Slide 4

Slide 4 text

“Microservices Iceberg” Background Image Copyright “Pere” http://bit.ly/1BPTITp

Slide 5

Slide 5 text

Background Image Copyright “Pere” http://bit.ly/1BPTITp “Microservices Iceberg”

Slide 6

Slide 6 text

Background Image Copyright “Pere” http://bit.ly/1BPTITp “Microservices Iceberg”

Slide 7

Slide 7 text

“Firm Foundations” Background Image Copyright Daniel Mayer http://bit.ly/1BXZHui

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

Evolving our Architecture Our core systems are also difficult to change and do not scale well but we are are not ready to replace them yet. It is difficult to understand / change our current television catchup services. We want to replace them. Isolate and protect the core systems. Build Microservices. Use REST.

Slide 10

Slide 10 text

1. DOMAIN SERVICES Building Blocks for a Microservices Architecture

Slide 11

Slide 11 text

Guidance Licencing Schedule Images Video Catalogue

Slide 12

Slide 12 text

Guidance Licencing Schedule Images Video Catalogue A set of Bounded Contexts Organisational Boundaries work best High Cohesion, Loose Coupling Think Resource Oriented

Slide 13

Slide 13 text

/catalogue/episodes/123456 { "synopsis" : "Maggie Smith, Hugh Bonneville and Elizabeth McGovern star in a glittering period drama following the lives of the aristocratic Crawley family on their estate, Downton.", “orderInSeries” : 2, “orderInProgramme” : 23 } { "restricted" : true, “message” : “Contains Strong Language” } /guidance/episodes/123456 Guidance Catalogue REST

Slide 14

Slide 14 text

/licencing/licences?episode=123456 { "_embedded": { "licences": [ { "platform": "samsung", "startDate": "2013-01-04T19:30:00Z", "endDate": "2013-01-11T19:30:00Z" } ] } } Licencing REST

Slide 15

Slide 15 text

External Caching on a Domain Legacy System Microservice Varnish Cache

Slide 16

Slide 16 text

External Caching on a Domain Legacy System Microservice Varnish Cache

Slide 17

Slide 17 text

Invalidating External Caching on a Domain Legacy System Microservice Varnish Cache

Slide 18

Slide 18 text

Invalidating External Caching on a Domain Legacy System Microservice Varnish Cache

Slide 19

Slide 19 text

The Netflix API Gateway Pattern http://techblog.netflix.com/2012/07/embracing-differences-inside-netflix.html https://github.com/Netflix/zuul/wiki

Slide 20

Slide 20 text

Vendor API Gateways - The ESBs of Tomorrow? Build "mash-ups" and composite services by orchestration or aggregation of existing APIs and web services. Edge API Services “ ” Customizable API composition and virtualization capabilities. “ ” API Gateway (formerly CA Layer 7 API Gateway)

Slide 21

Slide 21 text

Simple Made Easy http://www.infoq.com/presentations/Simple-Made-Easy Rich Hickey

Slide 22

Slide 22 text

Successes Bounded Contexts Resource Oriented Anti-Corruption Layers

Slide 23

Slide 23 text

Compromises Ignored Underlying Data Sources API Gateway can become a monolith

Slide 24

Slide 24 text

2. THE WRONG WAY of Scaling Aggregation

Slide 25

Slide 25 text

API Gateway Why is our API Gateway so complex? We’re not doing batch queries We’re using HTTP queries to simulate database inner joins

Slide 26

Slide 26 text

VOD Aggregation Service client specific integrations separated new VOD Service gets datastore Video On Demand Aggregation less HTTP calls exploit appropriate datastore query tools VOD Service populated by events

Slide 27

Slide 27 text

Publishing Updated Events Legacy System Microservice Varnish Cache SQS / AMQP

Slide 28

Slide 28 text

Consuming Updated Events SQS / AMQP Aggregation Service Domain Service Aggregation Service Datastore

Slide 29

Slide 29 text

Domain Service and Aggregator Service compared Domain Service Aggregation Service Consistent Eventually Consistent Modified by Human Commands Modified by Domain Update Events Long lived datastore Ephemeral datastore Like a single DB table Like a DB projected view

Slide 30

Slide 30 text

Aggregated Resource Model { "synopsis" : "Maggie Smith, Hugh Bonneville and Elizabeth McGovern star in a glittering period drama [...] { "restricted" : true, [...] { "_embedded": { "licences": [ { "platform": "samsung", "startDate" : [...] /vod/ios/episodes?channel=ITV& { "_embedded": { "productions": [ { "synopsis" : "Maggie Smith, Hugh Bonneville and Elizabeth McGovern star in a glittering period drama [...] ”, “restricted” : true, “startDate” : [...], VOD q=Downton&grouping=alphabetical

Slide 31

Slide 31 text

Successes Client Integration separated from Service Aggregation “Inner Join” HTTP Calls eliminated Query logic delegated to specialist datastore

Slide 32

Slide 32 text

Compromises High Coupling between Aggregator and Domain Services Aggregator must be updated if Domain Services Change

Slide 33

Slide 33 text

3. THE RIGHT WAY of Scaling Aggregation

Slide 34

Slide 34 text

Guidance Licencing Schedule Images Video Catalogue Availability

Slide 35

Slide 35 text

HATEOAS and HAL { "synopsis" : "Maggie Smith, Hugh Bonneville and Elizabeth McGovern star in a glittering period drama [...] { "restricted" : true, [...] { "_embedded": { "licences": [ { "platform": "samsung", "startDate" : [...] /vod/ios/episodes?channel=ITV& { "_embedded": { "productions": [ { “episodeId” : 123456, “available” : true, "_links": { "catalogue": { "href": "/catalogue/episodes/123456", }, "guidance": { "href": "/catalogue/guidance/123456", }, [...] Availability q=Downton&grouping=alphabetical

Slide 36

Slide 36 text

Availability Request Sequence

Slide 37

Slide 37 text

Successes Identified a new and highly cohesive domain Decoupled domain services from domain aggregator

Slide 38

Slide 38 text

Compromises We’ve pushed some duplicated logic back into API Gateways A challenge for Consumer Contract Testing

Slide 39

Slide 39 text

4. VERSIONING What aggregation can teach us about it

Slide 40

Slide 40 text

Versioning Pitfalls /catalogue/v1/episodes/123456 /vod/ios/v1/episodes?channel=ITV&q=Downton&grouping=alphabetical Availability { "_embedded": { "productions": [ { "_links": { "catalogue": { "href": "/catalogue/v1/episodes/123456", [...] Catalogue

Slide 41

Slide 41 text

Versioning Solutions Version the Host Don’t Version your APIs Require a Custom Version Header Require a Version in the Accept Header Media Type HTTP GET: https://v1.api.itv.com/catalogue/episodes/123456 Accept: application/hal+json HTTP GET: https://api.itv.com/catalogue/episodes/123456 Api-Version: 1 Accept: application/hal+json HTTP GET: https://api.itv.com/catalogue/episodes/123456 Accept: application/vnd.itv.ctv.production.v1+hal+json HTTP GET: https://api.itv.com/catalogue/episodes/123456 Accept: application/vnd.itv.ctv.production+hal+json;version=1 Require a Version Request Parameter HTTP GET: https://api.itv.com/catalogue/episodes/123456?version=1 Accept: application/hal+json

Slide 42

Slide 42 text

Content Negotiation Web browser makes a standard request Give it html HTTP GET: https://api.itv.com/catalogue/episodes/123456 Accept: text/html;q=0.9,image/webp,*/*;q=0.8 HTTP GET: https://api.itv.com/catalogue/episodes/123456 Content-Type: text/html; charset=utf-8

Slide 43

Slide 43 text

No Version? No JSON!

Slide 44

Slide 44 text

. IN SUMMARY

Slide 45

Slide 45 text

Let the system design emerge during development You may need to think more about how you compose your core Domains Mix your Architectural Styles Watch out for that monolith coming back! We have promoted many concepts from application architecture integration architecture

Slide 46

Slide 46 text

Thanks! Any questions? You can find me at: @caoilte_oconnor [email protected]

Slide 47

Slide 47 text

Credits Thanks to: ▷ Presentation template by SlidesCarnival ▷ Photography by Daniel Mayer, Leighton Dann and “Pere” ▷ Icons by Webalys and Pixabay