Camunda Meetup Vienna 2017: Microservices & Workflow

Microservices and Workflow? They can become a power couple. But don't force them together. And know their culture!-)

Transcript

1. Microservices and Workflow Martin Schimak • • Camunda Vienna Meetup • April 4th, 2017 With thoughts from http://flowing.io @berndruecker | @martinschimak
2. Microservices and Workflow? Well, first at all: do not force them... ! But then: expect surprises if you treat both of them well!
3. Microservices
4. Monolithic vs. Microservices Architecture https://martinfowler.com/articles/microservices.html
5. Characteristics of Microservices • Application as a Suite of Small Services • Each Service in its own Application Process • Communicating via e.g. HTTP/REST or Messaging • Independently Deployable and Scalable • Built around a specific Business Capability • Owned by a Cross-Functional Team • Aiming at a Maximum of Autonomy
6. Cross-functional teams around capabilities https://martinfowler.com/articles/microservices.html
7. Conway‘s Law & Team Autonomy • “Organizations which design systems […] are constrained to produce designs which are copies of the communication structures of these organizations.“ • Agile organisations with highly autonomous teams … do the same!
8. Microservices & Decentral Governance • No standardisation on single technology platforms • Instead „using the right tool for the job“ • In particular: decentral Data Management
9. Microservices & Decentral Data Management https://martinfowler.com/articles/microservices.html
10. Decentral Data Management via Events Inventory Service Shipment Service Customer1 Account Service Payment Service Microservices Events Address changed µS µS µS µS
11. Microservices are actually not SOA • SOA aimed for reusability - on the level of services • Microservices instead aim for replaceability and autonomy • Microservices store “duplicated“ data locally - in a controlled manner • Microservices may store data in the form needed for their service • Event collaboration enables nicely decoupled Microservices
12. Everybody Microservices? Buyer‘s beware. • Microservices fit to organisations with highly autonomous teams and high levels of dev/ops automation. • As desirable as this seems, organisational „success“ may depend on industry and still come in different shapes
13. Microservices and Workflow
14. Workflow ... a simple example. Fetch items Ship parcel Collect money Order:placed Order:delivered • Simple order in the happy scenario ! • Money is collected, e.g. via Credit Card • Items are fetched from inventory • Parcel is shipped
15. Business Capabilities? Inventory Shipment Shop Payment Business1 Capabilities Order:placed Money:collected Item(s):fetched Parcel shipped Business Outputs
16. Microservices! Inventory Service Shipment Service Checkout Service Payment Service Order:placed Money:collected Item(s):fetched Parcel shipped Microservices „Events“ µS µS µS µS
17. Good Practice for Microservices Autonomy Inventory Service Shipment Service Checkout Service Payment Service µS µS µS µS „Reach out to maximise the situations for which you need to change only one service to implement a business requirement!“
18. But where to put the overall „flow“? Inventory Service Shipment Service Checkout Service Payment Service µS µS µS µS ? ? ? ?
19. Into the pipe? Don‘t even dare. ! Inventory Service Shipment Service Checkout Service Payment Service µS µS µS µS ? „Smart:endpoints: and:dumb1pipes!” Martin:Fowler:
20. We could create an order service... Inventory Service Shipment Service Checkout Service Payment Service µS µS µS µS Order1Service µS An Order Service transforms events to commands. May still be asynchronous. May make use of workflow or not. Other services are listening to commands. Money:collected Item(s):fetched Parcel shipped Collect money! Fetch items! Order:placed
21. Known as Orchestration Principle -Build a centralised service managing the overall flow -You may use workflow in that service, or just hardcode the flow -Upside: You have very good management regarding the flow logic -Downside: Some features require change of central „flow service“ „A:few smart:god1services1tell: anemic CRUD:services:what:to:do!” Sam:Newman:(Author of „Building:Microservices“)
22. But then... do we need any central flow? Inventory Service Shipment Service Checkout Service Payment Service Order:placed Money:collected Item(s):fetched Parcel shipped µS µS µS µS
23. Known as Choreography Principle -Services decentrally subscribe on events published by other services. -„Upside“: „no central director“, each endpoint works out its own duty -„Downside: extra work to monitor and track the processes „Systems:tending toward choreoN graphy are more loosely coupled!” Sam:Newman:(Author of „Building:Microservices“)
24. 95% Agreed! -„Sender“ (publisher) does not need to know of „receiver“ (subscriber) -Event driven architectures encourage decentral data management! -Microservices Architectures should heavily tend to choreography.
25. But, event purism has coupling issues, too. Inventory Service Shipment Service Checkout Service Payment Service Order:placed Money:collected Item(s):fetched Parcel shipped µS µS µS µS
26. Let‘s look at a simple business requirement. „Our customer is king.:From now on,:we fetch the items before we collect the money!“ Fetch items Ship parcel Collect money Order:placed Order:delivered
27. Event subscriptions of three (!) services change Fetch Items Service Ship Parcel Service Checkout Service Collect Money Service Order:placed Money:collected Item(s):fetched Parcel shipped µS µS µS µS A:Red Flag !
28. An order service would be really nice here... Inventory Service Shipment Service Checkout Service Payment Service µS µS µS µS Order1Service µS
29. Let‘s look at another business requirement. „For VIP:customers,:we will: collect the money by invoice!“ Fetch items Ship parcel Collect money Order:placed Order:delivered
30. Let‘s look where to implement... Inventory Service Shipment Service Checkout Service Payment Service µS µS µS µS • The payment service would need a concept of a VIP customer to decide whether to collect directly or issue an invoice later. • But the inventory service would be affected, too. And it would need to know the VIP concept, too. That‘s not good. A:Red Flag !
31. Such requirements suggest an order service. Inventory Service Shipment Service Checkout Service Payment Service µS µS µS µS Order1Service µS Don‘t use events in a way which are semantical commands. Transforming some events to commands increases Microservices Autonomy.
32. Business Capabilities - revisited. Inventory Shipment Shop Payment Order:placed Money:collected Item(s):fetched Parcel shipped Order1 Fulfillment Order:delivered Support1Capabilities Core1Capability
33. What now? God plus anemic CRUD? Inventory Service Shipment Service Checkout Service Payment Service Order1Service
34. It‘s a question of autonomy and ownership. Inventory Service Shipment Service Checkout Service Payment Service µS µS µS µS Order1Service µS „Could potentially take long,:if an:item:is out:of stock. „Will:for all:practical cases always take longer than wanted.: !
35. Flow logic must be provided inside services Inventory Service Shipment Service Checkout Service Payment Service µS µS µS µS Order1Service µS „If a:credit card is rejected the customer has a:week to provide new details...
36. God services harm autonomy. Inventory Service Shipment Service Checkout Service Payment Service µS µS µS µS Order1Service µS A:red flag for god.:!
37. Move responsibility to where it belongs. Inventory Service Shipment Service Checkout Service Payment Service µS µS µS µS Order1Service µS Core1Process Support1Process The:payment service is a:black box1to the order service team
38. For BPMN folks it‘s well known: „Collaboration“ Payment Service Order1Service µS µS
39. Microservices and Workflow? • Many (Micro-)Services can potentially need long to deliver their service • It‘s a service team‘s decentral decision how this is implemented • Workflow is an option, if the requirements are tricky or change often • A separate service managing end-to-end flow logic may be necessary • But to avoid god services, each service takes care of its own flow logic • Microservice architectures use events to decentralise data management • For the 95% of cases event collaboration increases service decoupling • But instead of „event chains“ I prefer event-command transformation
40. Architectural Consequences
41. Several services (potentially) need „long“ Inventory Service Shipment Service Checkout Service Payment Service µS µS µS µS Order1Service µS Order:placed Money:collected Item(s):fetched Parcel shipped Order:delivered May:finish:fast,:but: may take long,:e.g.: due:to outdated payment details
42. Some services may choose a workflow engine Inventory Service Shipment Service Checkout Service Payment Service µS µS µS µS Order1Service µS Order:placed Money:collected Item(s):fetched Parcel shipped Order:delivered Workflow: engine HardN coded flows 3rdparty solution
43. Workflow must be lightweight and embeddable Payment Service µS Order1Service µS Money:collected Order:delivered Embedded: workflow Embedded: workflow
44. Workflow tooling must be composable Payment Service µS Order1Service µS Money:collected Task:completed Order:delivered Human1Task Service Monitoring Service µS µS Flow:canceled Independently deployable Solutions
45. Workflow monitoring must provide e2e views Payment Service Order1Service µS µS
46. Integrated tooling must show „foreign“ systems Payment Service µS Order1Service µS Money:collected Task:completed Order:delivered Human1Task Service Monitoring Service µS µS Inventory Service Item(s):fetched Inventory Service: with hardcoded flow logic Monitoring: Service: integrates with inventory events
47. Workflow monitoring must provide integration Inventory Service Order1Service µS µS Generated flow visualisation could track 3rd:party system events
48. Conclusions • Real Life Business Processes may be more or less centrally managed • Microservices Architectures ask for heavily decentral governance • Choreography and Event Collaboration are fundamental to achieve that • Sophisticated End-To-End flow logic may ask for a separate service • Such services transform events to commands offered by other services • Commands may increase autonomy when compared to event chains • Events should trigger additional logic the publisher does not care about • Workflow Tools must be embeddable, composable and integrate „e2e“
49. https://github.com/flowing Slides online at plexiti.com, feedback there or at twitter.com/martinschimak With thoughts from http://flowing.io @berndruecker | @martinschimak
50. Microservices and Workflow? Any questions open? !