Slide 1

Slide 1 text

1 Building modern microservices for scale with Red Hat Data Grid and Quarkus

Slide 2

Slide 2 text

dn.dev/kubemaster2 Technical Presentation and demo sshaaf@redhat ● Java developer, advocate, architect, engineer… ● Open source enthusiast, contributor ● Technical evangelist, developer advocate, marketing ● InfoQ Java Technical Editor ● Volunteer, coach, trainer.. ● Ask me about #Java, backends, architecture, containers.. fosstodon.org/@shaaf @syshaaf sshaaf https://www.linkedin.com/in/shaaf/ shaaf.dev

Slide 3

Slide 3 text

dn.dev/kubemaster2 Agenda - Quarkus the ultimate KubeNative framework - DataGrid a distribution of Infinispan - Caching, Security, DataTypes etc. - Cross-Site replicated cache over DCs - Operators, Cache configs, Failovers,

Slide 4

Slide 4 text

dn.dev/kubemaster2 Cloud Native Microservices Serverless

Slide 5

Slide 5 text

dn.dev/kubemaster2 ● Live code ● Dev services ● Dev UI ● Continuous testing ● Cli ● Remote-dev ● …. < Coding with Joy />

Slide 6

Slide 6 text

dn.dev/kubemaster2 ● Stores data like KVS ● Support for a variety of data types ● Set data lifespan ● Detect data changes ● Distributed processing where data is located ● Easily integrates with other data stores ● Automatically caches frequently used data on the app side ● Access by query without key ● Supports a variety of languages < development />

Slide 7

Slide 7 text

dn.dev/kubemaster2 ● All nodes read/write data ● Scale out (and in) to suit requirements ● Deployment strategies to suit requirements ● Automatic data replication and distribution ● Cross-region deployments ● Encryption of all routes ● Runs on-premises as well as in the cloud and containers ● Integration with various de facto standards ● Comprehensive set of operational tools < infra ops />

Slide 8

Slide 8 text

dn.dev/kubemaster2 < applications /> ● IoT ● Mobile applications ● User information such as ads and games ● Temporary data like session information ● E-commerce site carts ● Caching of high-load application processing

Slide 9

Slide 9 text

dn.dev/kubemaster2 Put : Store data by associating a key with a value. Get : Retrieve the value associated with a key Both keys and values can be types consisting of multiple elements. The combination of a key and a value is called an entry as well as a Map. A collection of data like table in RDBMS, where data is stored, is called a cache in DataGrid. Data is stored as keys and values as in the Key-Value-Store < Accessing data /> Key Value Ally { name: “Alice”, mail : “alice@...” } Bob { name: “Robert”, mail : “robert@...” } Carol { name: “Caroline”, mail : “caroline@...” } … … put key : Ally value : {name : “Alice”, mail : “alice@...” } get key : Ally {name : “Alice”, mail : “alice@...” } Application User Cache

Slide 10

Slide 10 text

dn.dev/kubemaster2 Use of collections in Java objects allows multiple objects to be stored for a single key. Supports any text, numeric, and binary. Stored as protobuf format by annotating the class that defines the data to be stored. HTTP sessions can also be stored. Java Objects Collections Scalar Types Stores complex structures such as XML and JSON documents as strings. Document Types Value < Variety of data types, from simple strings to complex objects />

Slide 11

Slide 11 text

dn.dev/kubemaster2 Data changes can be detected and any processing can be executed on a per-cache level*. Data Grid Developer Guide : 3.3. Listeners and Notifications Execute any process Notified application performs any processing, such as a real-time aggregation and analysis Data Change Delivery Deliver data changes to various processes by transferring data to queues such as Kafka Detects various changes Detects events such as creation, update, deletion, expiration, etc. Events update Notice Application Notice Queue Application

Slide 12

Slide 12 text

dn.dev/kubemaster2 < Query and Index /> Data is retrieved by SQL-like query language rather than accessed by key Querying Data Grid Caches : Query your data in Data Grid caches Statistics Provides information such as confirmation of indexes used and processing times. This allows users to tune performance. Uses a language that is a subset of JPQL. Pagination support, parameters, and full-text search are also supported. Setup using annotations or API, choose file or heap as storage location Query Index Monitors query results, notifies when results change, and can perform any processing Notify changes in results

Slide 13

Slide 13 text

dn.dev/kubemaster2 < Security /> Encryption of all routes and access control Hot Rod Java Client Guide : 3.4. Configuring Hot Rod client encryption role-based access control Restricted to only those users with the necessary privileges for cache access and interaction with resources. SSL/TLS encryption is implemented and certificates are presented to Hot Rod clients to establish a trusted and secure connection. Encryption by exchanging SSL/TLS certificates as well as inter-server encryption. Client encryption Inter-region encryption Deploy clusters using SSL/TLS encryption of communications and certificate authentication. Inter-Server Encryption

Slide 14

Slide 14 text

dn.dev/kubemaster2 < Using Data Stores /> If data is not loaded, it is automatically retrieved from other data stores. Reflects after data update Optimizes performance and cost by caching only the most frequently used data in the data store, rather than caching all data. Cache only used data Keep the system running with only frequently used data in the cache even if the data store is down. Application works if datastore stop When data in the cache is updated, it is reflected to the data store (a)synchronously. Reflects updated data No need to implement fetching from the cache if data is in the cache, or from the data store if not. No separate processing by data availability All the data Automatically retrieved Automatic update Update data Down Accessible Collaboration Frequently Used Data

Slide 15

Slide 15 text

dn.dev/kubemaster2 < Automatic cache size control /> Optimize performance and cost by automatically eliminating unused data Storing eliminated data in another datastore Optimize performance and cost by storing eliminated data in other data stores such as RDBMS or storage. Maintain a constant cache size by automatically eliminating recently unused data from cache after a certain period of time since the last access. This optimizes performance and cost. Manage data lifecycle as desired by setting a lifetime for the data Eliminate unused data Automatic data deletion Cache (up to 3) Cache (delete in 10min) Prevent the cache from depleting the heap by setting the maximum number of data and the maximum amount of memory used Suppresses memory depletion

Slide 16

Slide 16 text

dn.dev/kubemaster2 < Data Redundancy /> Automatically replicate data without application awareness Data grid replicated to other nodes ● When a client writes data, the data grid replicates that data to other nodes. ● Synchronous replication returns processing to the client after replication. Asynchronous replication returns processing to the client immediately after writing data to the first node, and replicates asynchronously with it. ● The order of priority of replicas is different sites, different racks, different machines and nodes. ● The number of replicas and whether they are synchronous or asynchronous replicas can be changed

Slide 17

Slide 17 text

dn.dev/kubemaster2 < Scattered /> Automatically replicate data without application awareness

Slide 18

Slide 18 text

dn.dev/kubemaster2 < Redistribution /> Automatically replicate data without application awareness

Slide 19

Slide 19 text

dn.dev/kubemaster2 DG holds only frequently used data, and remaining data is stored in block storage accessible by DG; NFS/SMB not supported.* < How to hold data /> Flexible data holding to meet availability, performance, and cost requirements Data is stored in the Java heap as well as Java objects. Java Heap FileStores DG holds only frequently used data and remaining data is stored in the RDBMS Data Stores Data is stored in memory outside the Java heap, and data changes have no effect on GC. Heap is used for metadata for data management. Off Heap Place to store inside the node Place to store the entire data

Slide 20

Slide 20 text

dn.dev/kubemaster2 @Entity public class Inventory extends PanacheEntity { public String itemId; public String location; public int quantity; public String link; }

Slide 21

Slide 21 text

dn.dev/kubemaster2 public class Product{ private String itemId; private String title; private String desc; private BigDecimal price; private int quantity; private String image; private String category; }

Slide 22

Slide 22 text

dn.dev/kubemaster2 public class Cart { private double cartItemTotal = 0.0f; private double cartItemPromoSavings = 0.0f; private double shippingTotal = 0.0f; private double shippingPromoSavings = 0.0f; private double cartTotal = 0.0f; private String cartId; private List cartItemList = new ArrayList() ; }

Slide 23

Slide 23 text

dn.dev/kubemaster2

Slide 24

Slide 24 text

dn.dev/kubemaster2

Slide 25

Slide 25 text

dn.dev/kubemaster2 ● [TP] New connector for Redis protocol ○ Initial support for a subset of commands of the RESP3 protocol: set, get, del, mget, mset, incr, decr, publish, subscribe, auth, ping. ○ Aiming at the typical caching usage. ○ Additional commands will be implemented in subsequent versions. ○ The connector integrates with RHDG security and protocol auto-detections in the single-port endpoint. ● Memcached improvements ○ Non-blocking implementation for higher throughput. RHDG 8.4 - Multi-protocol support Documentation: ● Using the RESP protocol endpoint with Data Grid Persistence Transactions Memory Management Console Prometheus JMX Clustering RHDG SERVER Cross Site Replication Indexing / Query Events and Listeners Distributed Execution Security Memcached RESP REST Hot Rod

Slide 26

Slide 26 text

dn.dev/kubemaster2 Thank you for listening… fosstodon.org/@shaaf @syshaaf sshaaf https://www.linkedin.com/in/shaaf/ shaaf.dev Building modern microservices for scale with Red Hat Data Grid and Quarkus