Message Driven Micro-Architectures

Sports websites have changed massively over several short years. The richness of their offerings and availability of rich data means that our development techniques also should be evolving. Join me for the journey of how we can use message driven development patterns to deal with larger amounts of realtime data.

Transcript

1. Message Driven Micro-Architectures @mackstar
2. Rich McIntyre @mackstar
3. Rich McIntyre @mackstar » Software Engineering Manager
4. Rich McIntyre @mackstar » Software Engineering Manager » Polyglot Development Background
5. Rich McIntyre @mackstar » Software Engineering Manager » Polyglot Development Background » Lived in Japan for 15 years
6. Rich McIntyre @mackstar » Software Engineering Manager » Polyglot Development Background » Lived in Japan for 15 years » Living in Leeds
9. Sport
16. Application Frontend Backend DB
18. Frontend Backend DB Frontend Team Backend Team
19. Frontend Backend Frontend Gateway Product Service Recommendation Service Pricing Service Basket Service DB
20. GraphQL Product Customer Basket Quote Content Campaigns Offers Rules Order Subsmission Store Locations Propositions Affiliates Cache Session Offline Message Queue Message Service Eligibility Consul Vault Prereg Reports Portfolio
22. Data Bottlenecks
23. The BBC says it has worked out how to eliminate "streaming lag", which causes live TV to be delayed by several seconds when watched online.
24. Coupling
25. Product Service Pricing Service Basket Service
26. Product Service Pricing Service Basket Service Security
27. Product Service Pricing Service Basket Service Security Data Warehousing
28. Product Service Pricing Service Basket Service Security Data Warehousing Monitoring
29. Product Service Pricing Service Basket Service Security Data Warehousing Monitoring Analytics
30. Product Service Pricing Service Basket Service Security Data Warehousing Monitoring Analytics
31. Product Service Pricing Service Basket Service Security Data Warehousing Monitoring Analytics
32. Product Service Pricing Service Basket Service Security Data Warehousing Monitoring Analytics
33. Product Service Pricing Service Basket Service Security Data Warehousing Monitoring Analytics
34. Cost of change
35. Personalisation
36. Neural Networks and Streams
37. Events
38. Redux
39. Action Reducer Store View
40. CQRS Publish Event Command Handler Command Query
41. Event Sourcing Publish Event Command Handler Command Query Event Store
42. Queues
43. Messaging 'decouples processing from data producers, to bu!er unprocessed messages'
44. A message is:
45. Lightweight
46. Immutable
47. Categorised
48. Subscribe-able
49. Ignore-able
50. Distribute-able
51. Asyncronous (Non-Blocking)
52. Can communicate failure
53. Idempotent
54. Push (not pull)
56. Kafka is built to allow real-time stream processing :not just processing of a single message at a time. This allows working with data streams at a much higher level of abstraction.
57. Event Sourcing In Event Sourcing, the single source of truth is the Event Stream, not the individual service datastores. And, to the contrary of PubSub, events are never removed from the stream when they are read. This property alone, the immutability and persistence of the event stream, make so that each datastore could be wiped out, and it could just be rebuilt by starting to read events from the beginning of time.
58. Use cases
59. Use cases » Messaging
60. Use cases » Messaging » Website Activity Tracking
61. Use cases » Messaging » Website Activity Tracking » Metrics
62. Use cases » Messaging » Website Activity Tracking » Metrics » Log Aggregation
63. Use cases » Messaging » Website Activity Tracking » Metrics » Log Aggregation » Event Sourcing
64. Use cases » Messaging » Website Activity Tracking » Metrics » Log Aggregation » Event Sourcing » Commit Log
65. Use cases » Messaging » Website Activity Tracking » Metrics » Log Aggregation » Event Sourcing » Commit Log » Stream Processing
68. Streaming Platform IoT Mobile App Watch Alexa API
69. Event Thinking Streaming Platform Mobile App Web App Api's Security Hadoop Monitoring Recommendations
70. Product Service Pricing Service Basket Service Security Data Warehousing Monitoring Analytics
71. Micro-Frontends Frontend Backend Product Discovery Team DB Search Team Checkout Team Product Information Team
72. Micro-Front ends
73. Micro-Front ends » Be Technology Agnostic
74. Micro-Front ends » Be Technology Agnostic » Isolate Team Code
75. Micro-Front ends » Be Technology Agnostic » Isolate Team Code » Establish Team Prefixes
76. Micro-Front ends » Be Technology Agnostic » Isolate Team Code » Establish Team Prefixes » Favor Native Browser Features over Custom APIs
77. Micro-Front ends » Be Technology Agnostic » Isolate Team Code » Establish Team Prefixes » Favor Native Browser Features over Custom APIs » Build a Resilient Site
89. A message driven micro- architecture is:
90. Loosly coupled
91. Scale-able
92. Elastic
93. Reactive
94. Fault tolerant
95. Go and build some amazing stu!!
96. Thank You @mackstar