Building modern microservices for scale with Red Hat Data Grid and Quarkus

Once you have decided on that path to microservices. you need to be able to manage the state and also ensure that your applications is high performing and secure. Enter the cloud native world with Red Hat Data Grid, that ensures a true cloud native experience and enables microservices to perform faster and scale as your architecture grows. In this live demo session you will learn more about Data Grid and its abilities to scale across clouds.

Join this session and learn how it integrates with modern frameworks like Quarkus, Node, etc.

Transcript

1. 1 Building modern microservices for scale with Red Hat Data Grid and Quarkus
2. dn.dev/kubemaster2 Technical Presentation and demo sshaaf@redhat • Java developer, advocate, architect, engineer… • Open source enthusiast, contributor • Technical evangelist, developer advocate, marketing • InfoQ Java Technical Editor • Volunteer, coach, trainer.. • Ask me about #Java, backends, architecture, containers.. fosstodon.org/@shaaf @syshaaf sshaaf https://www.linkedin.com/in/shaaf/ shaaf.dev
3. dn.dev/kubemaster2 Agenda - Quarkus the ultimate KubeNative framework - DataGrid a distribution of Infinispan - Caching, Security, DataTypes etc. - Cross-Site replicated cache over DCs - Operators, Cache configs, Failovers,
4. dn.dev/kubemaster2 Cloud Native Microservices Serverless
5. dn.dev/kubemaster2 • Live code • Dev services • Dev UI • Continuous testing • Cli • Remote-dev • …. < Coding with Joy />
6. dn.dev/kubemaster2 • Stores data like KVS • Support for a variety of data types • Set data lifespan • Detect data changes • Distributed processing where data is located • Easily integrates with other data stores • Automatically caches frequently used data on the app side • Access by query without key • Supports a variety of languages < development />
7. dn.dev/kubemaster2 • All nodes read/write data • Scale out (and in) to suit requirements • Deployment strategies to suit requirements • Automatic data replication and distribution • Cross-region deployments • Encryption of all routes • Runs on-premises as well as in the cloud and containers • Integration with various de facto standards • Comprehensive set of operational tools < infra ops />
8. dn.dev/kubemaster2 < applications /> • IoT • Mobile applications • User information such as ads and games • Temporary data like session information • E-commerce site carts • Caching of high-load application processing
9. dn.dev/kubemaster2 Put : Store data by associating a key with a value. Get : Retrieve the value associated with a key Both keys and values can be types consisting of multiple elements. The combination of a key and a value is called an entry as well as a Map. A collection of data like table in RDBMS, where data is stored, is called a cache in DataGrid. Data is stored as keys and values as in the Key-Value-Store < Accessing data /> Key Value Ally { name: “Alice”, mail : “alice@...” } Bob { name: “Robert”, mail : “robert@...” } Carol { name: “Caroline”, mail : “caroline@...” } … … put key : Ally value : {name : “Alice”, mail : “alice@...” } get key : Ally {name : “Alice”, mail : “alice@...” } Application User Cache
10. dn.dev/kubemaster2 Use of collections in Java objects allows multiple objects to be stored for a single key. Supports any text, numeric, and binary. Stored as protobuf format by annotating the class that defines the data to be stored. HTTP sessions can also be stored. Java Objects Collections Scalar Types Stores complex structures such as XML and JSON documents as strings. Document Types Value < Variety of data types, from simple strings to complex objects />
11. dn.dev/kubemaster2 <Data change detection /> Data changes can be detected and any processing can be executed on a per-cache level*. Data Grid Developer Guide : 3.3. Listeners and Notifications Execute any process Notified application performs any processing, such as a real-time aggregation and analysis Data Change Delivery Deliver data changes to various processes by transferring data to queues such as Kafka Detects various changes Detects events such as creation, update, deletion, expiration, etc. Events update Notice Application Notice Queue Application
12. dn.dev/kubemaster2 < Query and Index /> Data is retrieved by SQL-like query language rather than accessed by key Querying Data Grid Caches : Query your data in Data Grid caches Statistics Provides information such as confirmation of indexes used and processing times. This allows users to tune performance. Uses a language that is a subset of JPQL. Pagination support, parameters, and full-text search are also supported. Setup using annotations or API, choose file or heap as storage location Query Index Monitors query results, notifies when results change, and can perform any processing Notify changes in results
13. dn.dev/kubemaster2 < Security /> Encryption of all routes and access control Hot Rod Java Client Guide : 3.4. Configuring Hot Rod client encryption role-based access control Restricted to only those users with the necessary privileges for cache access and interaction with resources. SSL/TLS encryption is implemented and certificates are presented to Hot Rod clients to establish a trusted and secure connection. Encryption by exchanging SSL/TLS certificates as well as inter-server encryption. Client encryption Inter-region encryption Deploy clusters using SSL/TLS encryption of communications and certificate authentication. Inter-Server Encryption
14. dn.dev/kubemaster2 < Using Data Stores /> If data is not loaded, it is automatically retrieved from other data stores. Reflects after data update Optimizes performance and cost by caching only the most frequently used data in the data store, rather than caching all data. Cache only used data Keep the system running with only frequently used data in the cache even if the data store is down. Application works if datastore stop When data in the cache is updated, it is reflected to the data store (a)synchronously. Reflects updated data No need to implement fetching from the cache if data is in the cache, or from the data store if not. No separate processing by data availability All the data Automatically retrieved Automatic update Update data Down Accessible Collaboration Frequently Used Data
15. dn.dev/kubemaster2 < Automatic cache size control /> Optimize performance and cost by automatically eliminating unused data Storing eliminated data in another datastore Optimize performance and cost by storing eliminated data in other data stores such as RDBMS or storage. Maintain a constant cache size by automatically eliminating recently unused data from cache after a certain period of time since the last access. This optimizes performance and cost. Manage data lifecycle as desired by setting a lifetime for the data Eliminate unused data Automatic data deletion Cache (up to 3) Cache (delete in 10min) Prevent the cache from depleting the heap by setting the maximum number of data and the maximum amount of memory used Suppresses memory depletion
16. dn.dev/kubemaster2 < Data Redundancy /> Automatically replicate data without application awareness Data grid replicated to other nodes • When a client writes data, the data grid replicates that data to other nodes. • Synchronous replication returns processing to the client after replication. Asynchronous replication returns processing to the client immediately after writing data to the first node, and replicates asynchronously with it. • The order of priority of replicas is different sites, different racks, different machines and nodes. • The number of replicas and whether they are synchronous or asynchronous replicas can be changed
17. dn.dev/kubemaster2 < Scattered /> Automatically replicate data without application awareness
18. dn.dev/kubemaster2 < Redistribution /> Automatically replicate data without application awareness
19. dn.dev/kubemaster2 DG holds only frequently used data, and remaining data is stored in block storage accessible by DG; NFS/SMB not supported.* < How to hold data /> Flexible data holding to meet availability, performance, and cost requirements Data is stored in the Java heap as well as Java objects. Java Heap FileStores DG holds only frequently used data and remaining data is stored in the RDBMS Data Stores Data is stored in memory outside the Java heap, and data changes have no effect on GC. Heap is used for metadata for data management. Off Heap Place to store inside the node Place to store the entire data
20. dn.dev/kubemaster2 @Entity public class Inventory extends PanacheEntity { public String itemId; public String location; public int quantity; public String link; }
21. dn.dev/kubemaster2 public class Product{ private String itemId; private String title; private String desc; private BigDecimal price; private int quantity; private String image; private String category; }
22. dn.dev/kubemaster2 public class Cart { private double cartItemTotal = 0.0f; private double cartItemPromoSavings = 0.0f; private double shippingTotal = 0.0f; private double shippingPromoSavings = 0.0f; private double cartTotal = 0.0f; private String cartId; private List<CartItem> cartItemList = new ArrayList<CartItem>() ; }
23. dn.dev/kubemaster2
24. dn.dev/kubemaster2
25. dn.dev/kubemaster2 • [TP] New connector for Redis protocol ◦ Initial support for a subset of commands of the RESP3 protocol: set, get, del, mget, mset, incr, decr, publish, subscribe, auth, ping. ◦ Aiming at the typical caching usage. ◦ Additional commands will be implemented in subsequent versions. ◦ The connector integrates with RHDG security and protocol auto-detections in the single-port endpoint. • Memcached improvements ◦ Non-blocking implementation for higher throughput. RHDG 8.4 - Multi-protocol support Documentation: • Using the RESP protocol endpoint with Data Grid Persistence Transactions Memory Management Console Prometheus JMX Clustering RHDG SERVER Cross Site Replication Indexing / Query Events and Listeners Distributed Execution Security Memcached RESP REST Hot Rod
26. dn.dev/kubemaster2 Thank you for listening… fosstodon.org/@shaaf @syshaaf sshaaf https://www.linkedin.com/in/shaaf/ shaaf.dev Building modern microservices for scale with Red Hat Data Grid and Quarkus