Whaaat: Life without databases? Stateful architectures in Azure

Oh. My. God! You cannot be serious to take away that beautiful, cozy and beloved database from my architecture!? Well, contrary to common believe there are scenarios and use cases where adding a database to the mix can be tedious, complex and slow. E.g. when we are striving for high scalability, high availability and low latency stateful architectures may come handy. In this session, Christian sheds some light on this architecture style and its possible implementation in Azure. Come and see Azure Service Fabric's stateful services & reliable actors in action as well as Azure Durable Functions for completely serverless approaches. Let's see and experience which useful patterns we can realize with stateful solutions and let's talk about the good, the bad and the ugly parts, as well. Free your mind! 

Transcript

1. Whaaat: Life without databases? Stateful architectures in Azure Christian Weyer @christianweyer CTO
2. 2 § Co-founder, co-owner and CTO at Thinktecture AG § Focus on § Mobile & web-based application architectures § Interoperability, cross-device § Pragmatic end-to-end solutions § Cloud-native architectures § Microsoft MVP for ASP.NET (Architecture) ASPInsider, AzureInsider § Microsoft Regional Director, Germany § Google GDE for Web Technologies § christian.weyer@thinktecture.com § ! @christianweyer Christian Weyer Stateful Architectures in Azure Whaaat: Life without databases?
3. 3 § State of Affairs § Definitions & Patterns § Stateful #1 – Stateful Services § Azure Service Fabric § Stateful #2 – Stateful Orchestrations § Azure Durable Functions § The Good, The Bad, The Ugly Whaaat: Life without databases? Stateful Architectures in Azure Topics
4. 4 ❤ Our beloved database! Stateful Architectures in Azure Whaaat: Life without databases?
5. 5 What are databases good for? State of Affairs Stateful Architectures in Azure Whaaat: Life without databases?
6. 6 § This is not about building highly optimized… § base Cloud services to offer to others § games § This is about building business applications § This infamous boring stuff, you know – in the broadest sense § Functional microservices § Integration solutions § SaaS solutions § For ISVs & enterprises Target Audience & Use Cases Stateful Architectures in Azure Whaaat: Life without databases?
7. 7 § There is no such thing as ‘stateless architectures’ - actually § We deal with data – we always have state, somewhere, sometime Whaaat: Life without databases? Stateful Architectures in Azure The Myth of Stateless Architectures
8. 8 § We store our state in databases § Different kinds of state § Transient state § Persistent state § Different shapes of stores § Relational § Documents § Key-Value Pairs § Graphs etc. § Persistent business state as the ‘real’ data § Ability to analyze, do reports et. al. Databases – what are we using them for? Stateful Architectures in Azure Whaaat: Life without databases?
9. 9 § Single Responsibility § Focused around business domain § Small § Autonomy § Exclusive State § Isolation § Well-defined technology-agnostic API § Mobility § Asynchronous Communication § Decoupled message passing § Really, always? Whaaat: Life without databases? Stateful Architectures in Azure Microservices
10. 10 § Two meanings of stateful 1. Keeping state data nearby 2. Enabling stateful orchestrations Stateful: what does it mean, in the first place? Stateful Architectures in Azure Whaaat: Life without databases? § #1 ➔ we are not saving our persistent state to a database § Lower latency & higher performance § Pin data to a server node § In memory or on disk § What about scalability? § What about reliability? § #2 ➔ we want to enable stateful orchestrations § Realizing complex scenarios § E.g. long running tasks § What about ease of implementation? § What about the ‚right‘ abstraction?
11. 11 § Higher scalability § Higher performance § Higher availability Whaaat: Life without databases? Stateful Architectures in Azure #1 Stateful Services http://americanexpress.io/eliminate-the-database-for-higher-availability/
12. 12 § Realize real-world scenarios § No explicit state handling, e.g. with a database § Just write code Whaaat: Life without databases? Stateful Architectures in Azure #2 Stateful Orchestrations
13. 13 Stateful Services Azure Service Fabric Stateful Architectures in Azure Whaaat: Life without databases?
14. 14 § Service Fabric (SF) is a low-level Infrastructure-as-Service-towards-Platform-as-a-Service offering for building microservices § Some say “WCF on steroids” ;-) § Mental model: “Cloud operating system – with deployment management” § Optimized for exactly those usage scenarios § Quite low level § Underlying platform for a lot of Microsoft (1st party) Azure services § Clustered, reliable, available § Has runtime features for stateless & stateful services Azure Service Fabric Stateful Architectures in Azure Whaaat: Life without databases?
15. 15 § SF offers native programming model for microservices § Reliable, available, scalable, consistent § Agnostic to communication mechanisms § Stateless Services § As you know & love them § Reliable Services § Using reliable collections § Reliable Collections & Queues § Based on .NET collections, but different § Replicated, async, transactional, can be persisted § Reliable Actors § Implement Virtual Actor pattern: isolated, independent unit of compute & state with single-threaded execution Whaaat: Life without databases? Stateful Architectures in Azure Azure Service Fabric – Reliable Services & Actors https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-reliable-actors-platform
16. 16 Stateful Orchestrations Azure Durable Functions Stateful Architectures in Azure Whaaat: Life without databases?
17. 17 § Durable Functions (DF) let developers abstract away management of state for complex, stateful orchestrations § In code, not markup § Via HTTP addressable statemachine § Mental model: “Serverless orchestrations” § Based on Durable Task Framework § Write long running persistent workflows in C# using async/await § https://github.com/Azure/durabletask § Some say “WF done better” ;-) Azure Durable Functions – Serverless, FTW! Stateful Architectures in Azure Whaaat: Life without databases?
18. 18 § Extension of Azure Functions § Manages state, checkpoints, and restarts for you § You have to play by the rules § Define stateful workflows in a new type of function called orchestrator function § Advantages of orchestrator functions: § Define workflows in code § Call other functions synchronously and asynchronously § Output from called functions can be saved to local variables § Automatically checkpoint progress whenever the function awaits § Local state is never lost if the process recycles or the underlying VM reboots § Orchestrators can call activity functions, other sub-orchestrations, and wait for an external event Azure Durable Functions Stateful Architectures in Azure Whaaat: Life without databases?
19. 19 Whaaat: Life without databases? Stateful Architectures in Azure Azure Durable Functions - Patterns § Function chaining § Fan-out / fan-in § Async HTTP APIs § Stateful singletons § Human interaction
20. 20 Beyond the hype The Good, The Bad, The Ugly Stateful Architectures in Azure Whaaat: Life without databases?
21. Whaaat: Life without databases? Stateful Architectures in Azure Some areas of interest – it “really depends” IaaS/PaaS vs. Serverless/PaaS Cross- platform Programming models Runtimes Scalability Migration Maturity Reporting Time-to- market Pricing Availability … 21
22. § IaaS/PaaS vs. Serverless/PaaS § Trust vs. control § The future is Serverless § Maturity § SF: Very mature, has history inside MS § DF: Still in preview; based on immature Functions § Migration of applications § SF: Very easy on base platform § DF: Not a use case § Cross-platform, cross-location § SF & DF: Containers for both § SF & DF: Dedicated runtimes § Programming models § SF: Object-oriented, RPC-ish § DF: Functions & events § Reporting § SF: Roll your own, beware of disruptions § DF: Choose whatever you are used to § Pricing § SF: Can be reeeaaally expensive § DF: Can be really hard to predict § Time-to-market § Some want enterprise software at the speed of startups § Serverless is the future Whaaat: Life without databases? Stateful Architectures in Azure Good, Bad, Ugly – my take 22
23. 23 § So, life without databases? § No J § Stateful architectures are not a fit-it-all approach § You always pay a price – choose yours wisely § Reliable Services & Actors in Service Fabric are very powerful § Stateful reliable services & reliable actors § But they are also very challenging: E.g. you move your database in-memory, but have to deal with partitioning, transactions etc. § Durable Functions in Azure Functions are very powerful § Durable orchestrations § But you buy into different semantics you may know from .NET programming § In the end: It all depends! Of course. Summary & Conclusion Stateful Architectures in Azure Whaaat: Life without databases?