Architecture 101 for Mobile Backend Developers on Azure Daron Yöndem http://daron.me @daronyondem

What services / platform to pick? • Virtual Machine • Kubernetes (AKS) • Docker / Containers (ACI) • App Service Plans • Cloud Services • Serverless

How to build it? • N-Tier • Web-Queue-Worker • Microservices • Command and Query Responsibility Segregation • Event-driven

Where to store? • Relational • Key/Value • Document • Graph • Column • Object • Time Series

N-Tier • Monolithic design prevents independent deployment of features. • Portability between cloud and on-premises, and between cloud platforms. • Easy to end up with a middle tier that just does CRUD operations on the database, adding extra latency without doing any useful work.

No content

N-Tier • Use VM Scale Sets for autoscaling. • You can extend the Azure virtual network to your on-premises network using a site-to-site virtual private network (VPN) or Azure ExpressRoute.

Web-Queue-Worker • When you want to use managed services, rather than infrastructure as a service (IaaS). • Relatively simple architecture that is easy to understand. • Easy to deploy and manage. • Clear separation of concerns. • The front end is decoupled from the worker using asynchronous messaging. • The front end and the worker can be scaled independently. • Without careful design, the front end and the worker can become large, monolithic components that are difficult to maintain and update.

No content

Microservices architecture style • Each service is a separate codebase, which can be managed by a small development team. • Services are responsible for persisting their own data or external state. This differs from the traditional model, where a separate data layer handles data persistence. • Services communicate with each other by using well-defined APIs. Internal implementation details of each service are hidden from other services. • Services don’t need to share the same tech stack, libraries or frameworks.

Using Azure Container Services

Using Azure Container Services

Command and Query Responsibility Segregation • Commands may be placed in a queue for asynchronous processing, rather than being processed synchronously. • Queries never modify the database. A query returns a DTO that does not encapsulate any domain knowledge. • When the read and write workloads are asymmetrical. • Apply CQRS only to those subsystems where • there is clear value in separating reads / writes.

Event-driven • Multiple subsystems must process the same events. • Complex event processing, such as pattern matching or aggregation over time windows. • Producers and consumers are decoupled. • Highly scalable and distributed. • Processing events in order or exactly once. Are you idempotent?

No content

Let’s talk data • Azure SQL Database (Relational) • Redis Cache / Cosmos DB (Key/Value) • Document and Graph (CosmosDB) • Column Family (HBase in HDInsight) • Search Engine Database (Azure Search) • Time Series Database (Time Series Insights) • Object Storage (Blobs)

RDBMS (Azure SQL) • Data is highly normalised. • Database schemas are required and enforced. • Many-to-many relationships between data entities in the database. • Constraints are defined in the schema and imposed on any data in the database. • Data requires high integrity. Indexes and relationships need to be maintained accurately.

RDBMS (Azure SQL) • Data is highly normalised. • Database schemas are required and enforced. • Many-to-many relationships between data entities in the database. • Constraints are defined in the schema and imposed on any data in the database. • Data requires high integrity. Indexes and relationships need to be maintained accurately.

Document Databases • Data can be managed in denormalised way. • Insert and update operations are common. Both the creation of new records and updates to existing data happen regularly. • No object-relational impedance mismatch. Documents can better match the object structures used in application code. • Optimistic concurrency is more commonly used. • Each document type can use its own schema. • Individual documents are retrieved and written as a single block. • Document data is semi-structured, meaning that data types of each field are not strictly defined.

Key / Value Store • Data is identified and accessed using a single ID key, like a dictionary. • Massively scalable. • No joins, lock or unions are required. • No aggregation mechanisms are used. • Data size tends to be large. • Each key is associated with a single value, which is an unmanaged data BLOB. • There is no schema enforcement. • No relationships between entities.

Graph Databases • The relationships between data items are very complex, involving many hops between related data items. • The relationship between data items are dynamic and change over time. • Relationships between objects are first-class citizens, without requiring foreign-keys and joins to traverse. • Data is comprised of nodes and relationships. • Relationships are just as important as nodes and are exposed directly in the query language. • Composite objects, such as a person with multiple phone numbers, tend to be broken into separate, smaller nodes, combined with traversable relationships.

Column-Family Databases • Update and delete operations are rare. • Designed to provide high throughput and low-latency access. • Data is stored in tables consisting of a key column and one or more column families.

Search Engine Databases • Indexing data from multiple sources and services. • Queries are ad hoc and can be complex. • Requires aggregation. • Full text search is required. • Ad hoc self-service query is required. • Data analysis with index on all fields is required.

Time series databases • An overwhelmingly proportion of operations (95-99%) are writes. • Records are generally appended sequentially in time order. • Updates are rare. • Deletes occur in bulk and are made to contiguous blocks or records. • Read req • A time stamp that is used as the primary key and sorting mechanism. • Measurements from the entry or descriptions of what the entry represents.uests can be larger than available memory.

How to design for the Cloud? • Self-healing • Redundant • Relaxed :) • Scalable

Self-healing services • Protect failing remote services (Circuit Breaker). • Checkpoint long-running transactions. • Throttle clients. • Test with fault injection. • Degrade gracefully • Compensate failed transactions, make smaller ones if possible.

Redundancy at its core • Replicate databases. • Enable geo-replication • Use automatic failover, but manual failback (Azure Traffic Manager can do that!)

Design to scale out • Avoid instance stickiness • Offload resource-intensive tasks • Use built-in autoscaling features • Clients/consumers of a service should support transient fault handling and retry. • For long-running tasks, consider breaking up the work, using checkpoints or the Pipes and Filters pattern. • Put work items in a queue so that another instance can pick up the work, if an instance is removed in the middle of processing.

Minimise coordination • Embrace eventual consistency • Use domain events to synchronise state • Consider patterns such as CQRS and event sourcing • Design idempotent operations. • Use asynchronous parallel processing • Use optimistic concurrency when possible

Reference Architectures

Serverless scalability

Multi-Region Web App

Serverless Event Processing

Serverless Web App

Enterprise Integrations

Resources • Azure Application Architecture Guide: https://drn.fyi/2T8aphy • Resilliancy Patterns: https://drn.fyi/2Ufi5Lp • Azure Reference Architectures: https://drn.fyi/2BZ4v81

Thanks http://daron.me | @daronyondem Download slides here; http://daron.me/decks Sample code here; https://drn.fyi/2XbzK8D