Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Big Data Redis Mongodb Dynamodb Sharding

Big Data Redis Mongodb Dynamodb Sharding

Building Cloud-Native App Series - Part 5 of 15
Microservices Architecture Series
Distributed Cache
Redis, Hazelcast, EHCache, Oracle Coherence
NoSQL vs. SQL
Redis, MongoDB, AWS DynamoDB, Neo4J
Big Data Design Patterns
Sharding, Partitions
Multi-Tenancy

Araf Karsh Hamid

June 01, 2022
Tweet

More Decks by Araf Karsh Hamid

Other Decks in Technology

Transcript

  1. @arafkarsh arafkarsh
    8 Years
    Network &
    Security
    6+ Years
    Microservices
    Blockchain
    8 Years
    Cloud
    Computing
    8 Years
    Distributed
    Computing
    Architecting
    & Building Apps
    a tech presentorial
    Combination of
    presentation & tutorial
    ARAF KARSH HAMID
    Co-Founder / CTO
    MetaMagic Global Inc., NJ, USA
    @arafkarsh
    arafkarsh
    1
    Microservice
    Architecture Series
    Building Cloud
    Native Apps
    Distributed Cache: Hazelcast, Redis, EHCache
    NoSQL Vs. SQL: Redis / MongoDB / DynamoDB
    Scalability: Shards and Partitions
    Multi-Tenancy: DB, Schema, Table
    Compliance and Data Security
    Part 5 of 15

    View full-size slide

  2. @arafkarsh arafkarsh 2
    Slides are color coded based on the topic colors.
    Distributed Cache
    EHCache, Hazelcast,
    Redis, Coherence
    1
    NoSQL Vs. SQL
    Redis
    MongoDB
    DynamoDB, Neo4J
    2
    Scalability
    Sharding &
    Partitions
    3
    Multi-Tenancy
    Compliance
    Data Security
    4

    View full-size slide

  3. @arafkarsh arafkarsh
    Agile
    Scrum (4-6 Weeks)
    Developer Journey
    Monolithic
    Domain Driven Design
    Event Sourcing and CQRS
    Waterfall
    Optional
    Design
    Patterns
    Continuous Integration (CI)
    6/12 Months
    Enterprise Service Bus
    Relational Database [SQL] / NoSQL
    Development QA / QC Ops
    3
    Microservices
    Domain Driven Design
    Event Sourcing and CQRS
    Scrum / Kanban (1-5 Days)
    Mandatory
    Design
    Patterns
    Infrastructure Design Patterns
    CI
    DevOps
    Event Streaming / Replicated Logs
    SQL NoSQL
    CD
    Container Orchestrator Service Mesh

    View full-size slide

  4. @arafkarsh arafkarsh
    Application Modernization – 3 Transformations
    4
    Monolithic SOA Microservice
    Physical
    Server
    Virtual
    Machine
    Cloud
    Waterfall Agile DevOps
    Source: IBM: Application Modernization > https://www.youtube.com/watch?v=RJ3UQSxwGFY
    Architecture
    Infrastructure
    Delivery
    Modernization
    1
    2
    3

    View full-size slide

  5. @arafkarsh arafkarsh
    Distributed Caching
    • EHCache
    • Hazelcast
    • Oracle Coherence
    • Redis
    5
    1

    View full-size slide

  6. @arafkarsh arafkarsh
    Distributed Cache Feature Set
    6
    1. Language Support: Refers to the programming languages for which the distributed caching solution provides APIs
    or client libraries.
    2. Partitioning & Replication: The ability to partition data across multiple nodes and maintain replicas for fault
    tolerance and availability.
    3. Eviction Policies: Strategies to remove data from the cache when it reaches capacity. Standard policies include
    Least Recently Used (LRU), and Least Frequently Used (LFU).
    4. Persistence: Storing cached data on disk allows cache recovery in case of node failure or restart.
    5. Querying: Support for querying cached data using a query language or API.
    6. Transactions: The ability to perform atomic operations and maintain data consistency across cache operations.
    7. High Availability & Fault Tolerance: Support for redundancy and automatic failover to ensure the cache remains
    operational in case of node failures.
    8. Performance: A measure of the cache's ability to handle read and write operations with low latency and high
    throughput.
    9. Data Structures: The types of data structures supported by the caching solution.
    10. Open Source: Whether the caching solution is open-source and freely available for use and modification

    View full-size slide

  7. @arafkarsh arafkarsh
    Distributed Cache Comparison
    7
    Feature EHCache Hazelcast Coherence Redis
    Language Support Java
    Java, .NET, C++, Python,
    Node.js, etc.
    Java
    Java, Python, .NET, C++,
    etc.
    Partitioning &
    Replication
    Terracotta integration
    (limited)
    Native support Native support
    Native support (Redis
    Cluster)
    Eviction Policies LRU, FIFO, custom LRU, LFU, custom LRU, custom
    LRU, LFU, volatile,
    custom
    Persistence Disk-based persistence Disk-based persistence Disk-based persistence
    Disk-based and in-
    memory persistence
    Querying Limited support
    SQL-like querying
    (Predicate API)
    SQL-like querying (Filter
    API)
    Limited querying support
    Transactions Limited support Native support Native support Native support
    High Availability & Fault
    Tolerance
    Limited (with Terracotta) Native support Native support
    Native support (via
    replication and
    clustering)
    Performance Moderate High High High
    Data Structures Key-value pairs
    Key-value pairs, queues,
    topics, etc.
    Key-value pairs, caches,
    and services
    Strings, lists, sets,
    hashes, etc.
    Open Source Yes Yes No (proprietary) Yes

    View full-size slide

  8. @arafkarsh arafkarsh
    Operational In-Memory Computing
    8
    Cache Topology
    Standalone This setup consists of a single node containing
    all the cached data. It’s equivalent to a single-
    node cluster and does not collaborate with
    other running instances.
    Distributed Data is spread across multiple nodes in a cache
    such that only a single node is responsible for
    fetching a particular entry. This is possible by
    distributing/partitioning the cluster
    in a balanced manner (i.e., all the nodes have
    the same number of entries and are hence
    load balanced). Failover is handled via
    configurable backups on each node.
    Replicated Data is spread across multiple nodes in a cache
    such that each node consists of the complete
    cache data, since each cluster node contains all
    the data; failover is not a concern.
    Caching Strategies
    Read
    Through
    A process by which a missing cache entry is fetched from the integrated
    backend store.
    Write
    Through
    A process by which changes to a cache entry (create, update, delete) are
    pushed into the backend data store.
    It is important to note that the business logic for Read-Through and Write-
    Through operations for a specific cache are confined within the caching layer
    itself. Hence, your application remains insulated from the specifics of the
    cache and its backing system-of-record.
    Caching Mode
    Embedded When the cache and the application co-exist within the same JVM, the
    cache can be said to be operating in embedded mode. The cache lives and
    dies with the application JVM. This strategy should be used when:
    • Tight coupling between your application and the cache is not a concern
    • The application host has enough capacity (memory) to accommodate the
    demands of the cache
    Client /
    Server
    In this setup, the application acts as the client to a standalone (remote)
    caching layer. This should be leveraged when:
    • The caching infrastructure and application need to evolve independently
    • Multiple applications use a unified caching layer which can be scaled up
    without affecting client applications.
    Java Cache API: JSR 107 [ Distributed Caching /
    Distributed Computing / Distributed Messaging ]

    View full-size slide

  9. @arafkarsh arafkarsh
    Cache Deployment Models
    9
    Application
    Standalone
    Embedded Cache
    JVM
    Application
    Node 1
    Embedded Cache
    JVM
    Application
    Node 2
    Embedded Cache
    JVM
    Distributed
    or
    Replicated
    Cache
    Cluster
    Application
    Using Client API
    JVM
    Standalone
    Remote Cache
    JVM
    Stand Alone
    Client Server
    Cache
    Distributed or
    Replicated
    Cache Cluster
    Node 1
    Remote Cache
    JVM
    Node 2
    Remote Cache
    JVM
    Application
    Using Client API
    JVM
    Stand Alone
    Embedded
    Cache

    View full-size slide

  10. @arafkarsh arafkarsh
    Spring Cache Example
    • Service definition with Cacheable Annotation
    • With Complex Object
    • With Custom Key Generator
    10

    View full-size slide

  11. @arafkarsh arafkarsh
    Cache Simple Example
    11
    import org.springframework.cache.annotation.Cacheable;
    import org.springframework.stereotype.Service;
    @Service
    public class MyCacheService {
    @Cacheable(value = ”healthCareCache", key = "#name")
    public String getGreeting(String name) {
    /
    // Simulating an expensive operation
    try {
    Thread.sleep(7000);
    } catch (InterruptedException e) { e.printStackTrace(); }
    return "Hello, " + name + "! How are you today?";
    }
    }
    The @Cacheable annotation is part of the Spring
    Cache abstraction.
    This annotation indicates that the result of a
    method invocation should be cached so that
    subsequent invocations with the same arguments
    can return the result from the cache.
    1. Before the method execution, Spring
    generates a cache key based on the method
    arguments and the specified cache name.
    2. Spring checks if the cache contains a value
    associated with the generated cache key.
    3. If a cached value is found, it is returned
    directly, and the method is not executed.
    4. The method is executed if no cached value is
    found, and the result is stored in the cache
    with the generated key.
    5. The result of the method is returned to the
    caller.

    View full-size slide

  12. @arafkarsh arafkarsh
    Cache Annotations
    12
    @Service
    @CacheConfig(cacheNames = " healthCareCache ")
    public class PatientService {
    @Cacheable(key = "#id")
    public Patient findPatientById(String id) {
    // Code to Fetch Data
    }
    @CachePut(key = "#patient.id")
    public Patient updatePatient(Patient patient) {
    // Code to Update data
    // Cache is also updated
    }
    CacheEvict(key = "#id")
    public void deletePatient(String id) {
    // Code to Delete data
    // Cache is Evicted
    }
    }
    @CacheEvict
    The @CacheEvict annotation is used to
    remove one or more entries from the cache.
    You can specify the cache entry key to evict
    or use the allEntries attribute to remove all
    entries from the specified cache
    @CachePut
    The @CachePut annotation is used to update
    the cache with the result of the method
    execution. Unlike @Cacheable, which only
    executes the method if the result is not present
    in the cache, @CachePut always executes the
    method and then updates the cache with the
    returned value. This annotation is useful when
    you want to update the cache after modifying
    data.

    View full-size slide

  13. @arafkarsh arafkarsh
    Cache Complex Example
    13
    public class Patient {
    private String firstName, lastName, dateOfBirth, gender, maritalStatus;
    private String phone, email, address;
    public Patient(String firstName, String lastName,
    String dateOfBirth, String gender, String maritalStatus,
    String phone, String email, String address) {
    this.firstName = firstName;
    this.lastName = lastName;
    this.dateOfBirth = dateOfBirth;
    this.gender = gender;
    this.maritalStatus = maritalStatus;
    this.phone = phone;
    this.email = email; this.address = address;
    }
    // Getters omitted for brevity
    }

    View full-size slide

  14. @arafkarsh arafkarsh
    Cache Complex Example
    14
    import org.springframework.cache.annotation.Cacheable;
    import org.springframework.stereotype.Service;
    @Service
    public class PatientService {
    @Cacheable(value = "patient1Cache", key = "#patient.firstName")
    public Patient findPatientByFirstName(Patient patient) {
    // Simulate a time-consuming operation
    try { Thread.sleep(3000); } catch (InterruptedException e) { e.printStackTrace(); }
    return patient;
    }
    @Cacheable(value = "patient2Cache", key = "#firstName + '_' + #lastName")
    public Patient findPatientByFirstAndLastName(String firstName, String lastName) {
    try { Thread.sleep(3000); } catch (InterruptedException e) { e.printStackTrace(); }
    return new Patient(firstName, lastName, "01-01-1995", "Female", ”Single",
    ”98765-12345", "[email protected]", "123 Main St");
    }
    }

    View full-size slide

  15. @arafkarsh arafkarsh
    Cache Example: Custom Key Generator
    15
    import org.springframework.cache.interceptor.KeyGenerator;
    import org.springframework.web.context.request.RequestContextHolder;
    import org.springframework.web.context.request.ServletRequestAttributes;
    import javax.servlet.http.HttpServletRequest;
    import java.lang.reflect.Method;
    public class CustomKeyGenerator implements KeyGenerator {
    @Override
    public Object generate(Object target, Method method, Object... params) {
    HttpServletRequest request = ((ServletRequestAttributes)
    RequestContextHolder.currentRequestAttributes()).getRequest();
    String customHeaderValue = request.getHeader("X-Custom-Header");
    StringBuilder keyBuilder = new StringBuilder();
    keyBuilder.append(method.getName()).append("-");
    if (customHeaderValue != null) {
    keyBuilder.append(customHeaderValue).append("-");
    }
    for (Object param : params) {
    keyBuilder.append(param.toString()).append("-");
    }
    return keyBuilder.toString();
    }
    }

    View full-size slide

  16. @arafkarsh arafkarsh
    Cache Example: Key Generator Bean
    16
    import org.springframework.cache.annotation.EnableCaching;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    @Configuration
    @EnableCaching
    public class CacheConfig {
    @Bean("customKeyGenerator")
    public CustomKeyGenerator customKeyGenerator() {
    return new CustomKeyGenerator();
    }
    }

    View full-size slide

  17. @arafkarsh arafkarsh
    Cache Complex Example
    17
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.beans.factory.annotation.Qualifier;
    import org.springframework.cache.annotation.Cacheable;
    import org.springframework.cache.interceptor.KeyGenerator;
    import org.springframework.stereotype.Service;
    @Service
    public class PatientService {
    @Autowired
    @Qualifier("customKeyGenerator")
    private KeyGenerator customKeyGenerator;
    @Cacheable(value = "patientCache", keyGenerator = "customKeyGenerator")
    public Patient findPatientByFirstAndLastName(String firstName, String lastName) {
    try { Thread.sleep(3000); } catch (InterruptedException e) { e.printStackTrace(); }
    return new Patient(firstName, lastName, "01-01-1995", "Female", ”Single", ”
    98765-12345", "[email protected]", "123 Main St");
    }
    }

    View full-size slide

  18. @arafkarsh arafkarsh
    EHCache Setup
    • POM File
    • Configuration File
    • Springboot Configuration
    18

    View full-size slide

  19. @arafkarsh arafkarsh
    EHCache Setup
    19

    org.springframework.boot
    spring-boot-starter-cache


    org.ehcache
    ehcache

    POM File
    xsi:noNamespaceSchemaLocation="http://www.ehcache.org/schema/ehcache-
    core.xsd">
    maxElementsInMemory="100"
    eternal="false"
    timeToIdleSeconds="120"
    timeToLiveSeconds="120"
    overflowToDisk="true"/>

    Configuration – ehcache.xml

    View full-size slide

  20. @arafkarsh arafkarsh
    EHCache Properties
    20
    • name: The unique name of the cache.
    • maxElementsInMemory: The maximum number of elements that can be stored in memory. Once this limit is
    reached, elements can be evicted or overflow to disk, depending on the configuration. The value is a positive
    integer.
    • eternal: A boolean value that indicates whether the elements in the cache should never expire. If set to true,
    the timeToIdleSeconds and timeToLiveSeconds attributes are ignored. The value is either true or false.
    • timeToIdleSeconds: The maximum number of seconds an element can be idle (not accessed) before it expires.
    A value of 0 means there's no limit on the idle time. The value is a non-negative integer.
    • timeToLiveSeconds: The maximum number of seconds an element can exist in the cache, regardless of idle
    time. A value of 0 means there's no limit on the element's lifespan. The value is a non-negative integer.
    • overflowToDisk: A boolean value that indicates whether elements can be moved from memory to disk when
    the maxElementsInMemory limit is reached. This attribute is deprecated in EHCache 2.10 and removed in
    EHCache 3.x. Instead, you should use the diskPersistent attribute or configure a disk store element. The value
    is either true or false.

    View full-size slide

  21. @arafkarsh arafkarsh
    EHCache Setup for Distributed
    21

    org.ehcache.modules
    ehcache-clustered
    get-latest-version

    POM File

    View full-size slide

  22. @arafkarsh arafkarsh
    EHCache Spring Boot Config
    22
    import org.springframework.cache.annotation.EnableCaching;
    import org.springframework.cache.ehcache.EhCacheCacheManager;
    import org.springframework.cache.ehcache.EhCacheManagerFactoryBean;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.core.io.ClassPathResource;
    @Configuration
    @EnableCaching
    public class CacheConfiguration { @Bean
    public EhCacheCacheManager cacheManager() {
    return new EhCacheCacheManager(ehCacheCacheManager().getObject());
    } @Bean
    public EhCacheManagerFactoryBean ehCacheCacheManager() {
    EhCacheManagerFactoryBean factory = new EhCacheManagerFactoryBean();
    factory.setConfigLocation(new ClassPathResource("ehcache.xml")); factory.setShared(true);
    return factory;
    }
    }

    View full-size slide

  23. @arafkarsh arafkarsh
    Hazelcast Setup
    • POM File
    • Configuration File
    • Springboot Configuration
    23

    View full-size slide

  24. @arafkarsh arafkarsh
    Hazelcast Setup
    24

    org.springframework.boot
    spring-boot-starter-cache


    com.hazelcast
    hazelcast


    com.hazelcast
    hazelcast-spring

    POM File
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.hazelcast.com/schema/config
    http://www.hazelcast.com/schema/config/hazelcast-config.xsd">

    max-size-policy="ENTRY_COUNT”
    eviction-policy="LRU" />

    expiry-policy-type="TOUCHED"
    duration-amount="120"
    time-unit="SECONDS"/>



    Configuration – hazelcast.xml

    View full-size slide

  25. @arafkarsh arafkarsh
    Hazelcast Properties
    25
    1. cache name: The unique name of the cache.
    2. eviction:
    • size: The maximum number of elements in the cache before eviction occurs. The value is a positive integer.
    • max-size-policy: The cache size policy. The possible values are ENTRY_COUNT, USED_HEAP_SIZE, and
    USED_HEAP_PERCENTAGE.
    • eviction-policy: The eviction policy for the cache. The possible values are LRU (Least Recently Used), LFU
    (Least Frequently Used), RANDOM, and NONE.
    3. expiry-policy-factory: Configures the cache's expiration policy.
    • timed-expiry-policy-factory: A factory for creating a timed expiry policy.
    • expiry-policy-type: The type of expiry policy. Possible values are CREATED, ACCESSED, MODIFIED, and
    TOUCHED. CREATED expires based on the creation time, ACCESSED expires based on the last access
    time, MODIFIED expires based on the last modification time, and TOUCHED expires based on the last
    access or modification time.
    • duration-amount: The duration amount for the expiry policy. The value is a positive integer.
    • time-unit: The time unit for the duration amount. Possible values are NANOSECONDS,
    MICROSECONDS, MILLISECONDS, SECONDS, MINUTES, HOURS, and DAYS.

    View full-size slide

  26. @arafkarsh arafkarsh
    Hazelcast Setup for Distributed
    26
    xsi:schemaLocation="http://www.hazelcast.com/schema/config http://www.hazelcast.com/schema/config/hazelcast-config.xsd">





    machine1:5701
    machine2:5701


    -->




    120

    Configuration – hazelcast.xml

    View full-size slide

  27. @arafkarsh arafkarsh
    Hazelcast Spring Boot Config
    27
    import com.hazelcast.config.Config;
    import com.hazelcast.core.Hazelcast;
    import com.hazelcast.core.HazelcastInstance;
    import org.springframework.cache.annotation.EnableCaching;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.core.io.ClassPathResource;
    import org.springframework.cache.CacheManager;
    import org.springframework.cache.hazelcast.HazelcastCacheManager;
    @Configuration
    @EnableCaching
    public class CacheConfiguration { @Bean
    public CacheManager cacheManager() {
    return new HazelcastCacheManager(hazelcastInstance());
    } @Bean
    public HazelcastInstance hazelcastInstance() {
    Config config = new Config(); config.setConfigurationFile(
    new ClassPathResource("hazelcast.xml").getFile());
    return Hazelcast.newHazelcastInstance(config);
    }
    }

    View full-size slide

  28. @arafkarsh arafkarsh
    Oracle Coherence Setup
    • POM File
    • Configuration File
    • Springboot Configuration
    28

    View full-size slide

  29. @arafkarsh arafkarsh
    Oracle Coherence Setup
    29

    org.springframework.boot
    spring-boot-starter-cache



    com.oracle.coherence.spring
    coherence-spring-boot-starter
    3.3.2

    POM File

    xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
    xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">


    healthCareCache
    example-distributed




    example-distributed



    true



    Configuration – coherence.xml

    View full-size slide

  30. @arafkarsh arafkarsh
    Coherence Properties
    30
    1.cache-config: The root element for the Coherence cache configuration.
    2.caching-scheme-mapping: Contains the mapping of cache names to caching schemes.
    1. cache-mapping: Defines the mapping between a cache name and a caching scheme.
    1. cache-name: The unique name of the cache.
    2. scheme-name: The name of the caching scheme that this cache should use.
    3.caching-schemes: Contains the caching scheme definitions.
    1. distributed-scheme: The distributed caching scheme. This scheme provides a distributed cache,
    partitioned across the cluster members.
    1. scheme-name: The name of the distributed caching scheme.
    2. backing-map-scheme: The backing map scheme that defines the storage strategy for the distributed
    cache.
    1. local-scheme: A local backing map scheme that stores the cache data in the local member's
    memory.
    3. autostart: A boolean value that indicates whether the cache should start automatically when the
    cache service starts. The value is either true or false.

    View full-size slide

  31. @arafkarsh arafkarsh
    Oracle Coherence Spring Boot Config
    31
    import com.tangosol.net.CacheFactory;
    import com.tangosol.net.ConfigurableCacheFactory;
    import com.tangosol.net.NamedCache;
    import org.springframework.cache.CacheManager;
    import org.springframework.cache.annotation.EnableCaching;
    import org.springframework.cache.support.SimpleCacheManager;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.core.io.ClassPathResource;
    import java.util.Collections;
    @Configuration
    @EnableCaching
    public class CacheConfiguration {
    @Bean@Bean
    public CacheManager cacheManager() {
    NamedCache cache = getCache(”healthCareCache");
    SimpleCacheManager cacheManager = new SimpleCacheManager();
    cacheManager.setCaches(Collections.singletonList(cache));
    return cacheManager;
    }
    @Bean
    public ConfigurableCacheFactory configurableCacheFactory() {
    return CacheFactory.getCacheFactoryBuilder()
    .getConfigurableCacheFactory(new ClassPathResource(
    "coherence.xml").getFile());
    }
    private NamedCache getCache(String cacheName) {
    return configurableCacheFactory().ensureCache(cacheName, null);
    }
    }

    View full-size slide

  32. @arafkarsh arafkarsh
    Redis Cache Setup
    • POM File
    • Configuration File
    • Springboot Configuration
    32

    View full-size slide

  33. @arafkarsh arafkarsh
    Redis Setup Standalone & Distributed
    33

    org.springframework.boot
    spring-boot-starter-data-redis


    org.springframework.boot
    spring-boot-starter-cache
    >
    POM File
    spring.redis.host=127.0.0.1
    spring.redis.port=6379
    Standalone
    Configuration – application.properties
    Distributed
    Configuration – application.properties
    spring.redis.cluster.nodes=node1:6379,node2:6380
    cluster-enabled yes
    cluster-config-file nodes.conf
    cluster-node-timeout 5000
    Update this for each Redis instance
    Nodes.conf is automatically created by Redis
    when running in cluster mode
    redis.conf

    View full-size slide

  34. @arafkarsh arafkarsh
    Redis Spring Boot Config
    34
    import org.springframework.cache.CacheManager;
    import org.springframework.cache.annotation.EnableCaching;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.data.redis.cache.RedisCacheConfiguration;
    import org.springframework.data.redis.cache.RedisCacheManager;
    import org.springframework.data.redis.connection.RedisConnectionFactory;
    import org.springframework.data.redis.core.RedisTemplate;
    import java.time.Duration;
    @Configuration
    @EnableCaching
    public class CacheConfiguration { @Bean
    public CacheManager cacheManager(RedisConnectionFactory redisConnectionFactory) {
    RedisCacheConfiguration configuration = RedisCacheConfiguration.defaultCacheConfig()
    .entryTtl(Duration.ofSeconds(120)); // Set the time-to-live for the cache entries
    return RedisCacheManager.builder(redisConnectionFactory) .
    cacheDefaults(configuration) .build();
    }
    @Bean public RedisTemplate redisTemplate(RedisConnectionFactory redisConnectionFactory) {
    RedisTemplate redisTemplate = new RedisTemplate<>();
    redisTemplate.setConnectionFactory(redisConnectionFactory);
    return redisTemplate;
    }
    }

    View full-size slide

  35. @arafkarsh arafkarsh
    2
    NoSQL Databases
    o Cap Theorem
    o Sharding / Partitioning
    o Geo Partitioning
    o Oracle Sharding and Geo Partitioning
    35

    View full-size slide

  36. @arafkarsh arafkarsh
    ACID Vs. BASE
    36
    # Property ACID BASE
    1 Acronym Atomicity, Consistency, Isolation, Durability
    Basically Available, Soft state, Eventual
    consistency
    2 Focus
    Strong consistency, data integrity, transaction
    reliability
    High availability, partition tolerance, high
    scalability
    3 Applicability Traditional relational databases (RDBMS) Distributed NoSQL databases
    4 Transactions Ensures all-or-nothing transactions Allows partial transactions, more flexible
    5 Consistency Guarantees strong consistency Supports eventual consistency
    6 Isolation
    Ensures transactions are isolated from each
    other
    Transactions may not be fully isolated
    7 Durability
    Guarantees data is permanently stored once
    committed
    Data durability may be delayed, relying on
    eventual consistency
    8 Latency
    Higher latency due to stricter consistency
    constraints
    Lower latency due to relaxed consistency
    constraints
    9 Use Cases Financial systems, inventory management, etc.
    Social networks, recommendation
    systems, search engines, etc.

    View full-size slide

  37. @arafkarsh arafkarsh
    CAP Theorem by Eric Allen Brewer
    37
    Pick Any 2!! Say NO to 2 Phase Commit J
    Source: https://en.wikipedia.org/wiki/CAP_theorem | http://en.wikipedia.org/wiki/Eric_Brewer_(scientist)
    CAP 12 years later: How the “Rules have changed”
    “In a network subject to communication failures, it is
    impossible for any web service to implement an atomic
    read / write shared memory that guarantees a response
    to every request.”
    Partition Tolerance (Key in Cloud)
    The system continues to operate despite an arbitrary
    number of messages being dropped (or delayed) by
    the network between nodes.
    Consistency
    Every read receives the
    most recent write or an
    error.
    Availability
    Every request receives a (non-error) response – without
    guarantee that it contains the most recent write.
    Old Single
    Node
    RDBMS

    View full-size slide

  38. @arafkarsh arafkarsh
    Databases that Support CA
    38
    Aster (Teradata Aster Database): A parallel, distributed, and columnar
    database is designed to perform advanced analytics and manage large-
    scale data. It provides high performance and availability but does not
    explicitly focus on partition tolerance.
    Greenplum: It is an open-source MPP database that is based on
    PostgreSQL. It is designed for handling large-scale analytical workloads
    and provides high performance and availability. Greenplum is also
    designed for fault tolerance and can recover from failures; however, it
    does not explicitly focus on partition tolerance.
    Vertica is a columnar MPP (Massively Parallel Processing) database
    designed for high-performance analytics and large-scale data
    management. Vertica offers high availability through data replication
    and automated failover, ensuring the system's resilience in case of
    node failures. However, Vertica does not explicitly focus on partition
    tolerance.
    Traditional RDBMS
    (Single Node Implementation)
    1. DB2
    2. MS SQL
    3. MySQL
    4. Oracle
    5. PostgreSQL

    View full-size slide

  39. @arafkarsh arafkarsh
    Databases that support both AP / CP
    39
    1. MongoDB
    2. Cassandra
    3. Amazon DynamoDB
    4. Couchbase
    5. Riak
    6. ScyllaDB
    • Network partitions are
    considered inevitable in
    modern distributed
    systems, and most
    databases and systems
    now prioritize partition
    tolerance by default.
    • The challenge is to find
    the right balance
    between consistency
    and availability in the
    presence of partitions.

    View full-size slide

  40. @arafkarsh arafkarsh
    MongoDB: Consistency / Partition Tolerance
    40
    import com.mongodb.MongoClientSettings;
    import com.mongodb.ReadConcern;
    import com.mongodb.WriteConcern;
    import com.mongodb.client.MongoClient;
    import com.mongodb.client.MongoClients;
    import com.mongodb.client.MongoDatabase;
    public class MongoDBExample {
    public static void main(String[] args) {
    MongoClientSettings settings = MongoClientSettings.builder()
    .applyConnectionString(new ConnectionString("mongodb://localhost:27017"))
    .readConcern(ReadConcern.MAJORITY)
    .writeConcern(WriteConcern.MAJORITY)
    .build();
    MongoClient mongoClient = MongoClients.create(settings);
    MongoDatabase exampleDb = mongoClient.getDatabase(”healthcare_db");
    }
    }

    View full-size slide

  41. @arafkarsh arafkarsh
    MongoDB: Availability / Partition Tolerance
    41
    import com.mongodb.MongoClientSettings;
    import com.mongodb.ReadConcern;
    import com.mongodb.WriteConcern;
    import com.mongodb.client.MongoClient;
    import com.mongodb.client.MongoClients;
    import com.mongodb.client.MongoDatabase;
    public class MongoDBExample {
    public static void main(String[] args) {
    MongoClientSettings settings = MongoClientSettings.builder() .
    .applyConnectionString(new ConnectionString("mongodb://localhost:27017"))
    .readConcern(ReadConcern.LOCAL)
    .writeConcern(WriteConcern.W1)
    .build();
    MongoClient mongoClient = MongoClients.create(settings);
    MongoDatabase exampleDb = mongoClient.getDatabase(”healthcare_db");
    }
    }

    View full-size slide

  42. @arafkarsh arafkarsh
    Cassandra: Consistency / Partition Tolerance
    42
    import com.datastax.driver.core.Cluster;
    import com.datastax.driver.core.Session;
    import com.datastax.driver.core.ConsistencyLevel;
    import com.datastax.driver.core.SimpleStatement;
    Import com.datastax.driver.core.Statement;
    public class CassandraAPExample {
    public static void main(String[] args) {
    Cluster cluster = Cluster.builder().addContactPoint("127.0.0.1").build();
    Session session = cluster.connect(" healthcare_keyspace");
    Statement writeStatement = new SimpleStatement("INSERT INTO diagnosis_t (id, value) VALUES (1, 'test')");
    writeStatement.setConsistencyLevel(ConsistencyLevel.QUORUM);
    session.execute(writeStatement);
    Statement readStatement = new SimpleStatement("SELECT * FROM diagnosis_t WHERE id = 1");
    readStatement.setConsistencyLevel(ConsistencyLevel.QUORUM);
    session.execute(readStatement);
    }
    }

    View full-size slide

  43. @arafkarsh arafkarsh
    Cassandra: Availability / Partition Tolerance
    43
    import com.datastax.driver.core.Cluster;
    import com.datastax.driver.core.Session;
    import com.datastax.driver.core.ConsistencyLevel;
    import com.datastax.driver.core.SimpleStatement;
    Import com.datastax.driver.core.Statement;
    public class CassandraAPExample {
    public static void main(String[] args) {
    Cluster cluster = Cluster.builder().addContactPoint("127.0.0.1").build();
    Session session = cluster.connect(" healthcare_keyspace");
    Statement writeStatement = new SimpleStatement("INSERT INTO diagnosis_t (id, value) VALUES (1,
    'test')");
    writeStatement.setConsistencyLevel(ConsistencyLevel.ONE);
    session.execute(writeStatement);
    Statement readStatement = new SimpleStatement("SELECT * FROM diagnosis_t WHERE id = 1");
    readStatement.setConsistencyLevel(ConsistencyLevel.ONE);
    session.execute(readStatement);
    }
    }

    View full-size slide

  44. @arafkarsh arafkarsh
    AWS DynamoDB: Consistency / Partition Tolerance
    44
    import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
    import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder;
    import com.amazonaws.services.dynamodbv2.document.DynamoDB;
    import com.amazonaws.services.dynamodbv2.document.Item;
    import com.amazonaws.services.dynamodbv2.document.spec.GetItemSpec;
    import com.amazonaws.services.dynamodbv2.document.spec.PutItemSpec;
    public class DynamoDBAPExample {
    public static void main(String[] args) {
    AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard().build();
    DynamoDB dynamoDB = new DynamoDB(client);
    // Write an item (CP-like behavior)
    PutItemSpec putItemSpecCP = new PutItemSpec()
    .withItem(new Item().withPrimaryKey("id", 1)
    .withString("value", "test"));
    dynamoDB.getTable("diagnosis_t").putItem(putItemSpecCP);
    // Read with strongly consistent read (CP-like behavior)
    GetItemSpec getItemSpecStronglyConsistent = new GetItemSpec()
    .withPrimaryKey("id", 1) .withConsistentRead(true);
    Item itemCP = dynamoDB.getTable("diagnosis_t ")
    .getItem(getItemSpecStronglyConsistent);
    System.out.println("CP: " + itemCP);
    }
    }

    View full-size slide

  45. @arafkarsh arafkarsh
    AWS DynamoDB: Availability / Partition Tolerance
    45
    import com.amazonaws.services.dynamodbv2.AmazonDynamoDB;
    import com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder;
    import com.amazonaws.services.dynamodbv2.document.DynamoDB;
    import com.amazonaws.services.dynamodbv2.document.Item;
    import com.amazonaws.services.dynamodbv2.document.spec.GetItemSpec;
    import com.amazonaws.services.dynamodbv2.document.spec.PutItemSpec;
    public class DynamoDBAPExample {
    public static void main(String[] args) {
    AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard().build();
    DynamoDB dynamoDB = new DynamoDB(client);
    // Write an item (CP-like behavior)
    PutItemSpec putItemSpecCP = new PutItemSpec()
    .withItem(new Item().withPrimaryKey("id", 1)
    .withString("value", "test"));
    dynamoDB.getTable(" diagnosis_t").putItem(putItemSpecCP);
    // Read with eventually consistent read (AP-like behavior)
    GetItemSpec getItemSpecEventuallyConsistent = new GetItemSpec()
    .withPrimaryKey("id", 1) .withConsistentRead(false);
    Item itemAP = dynamoDB.getTable(" diagnosis_t")
    .getItem(getItemSpecEventuallyConsistent);
    System.out.println("AP: " + itemAP);
    }
    }

    View full-size slide

  46. @arafkarsh arafkarsh
    NoSQL Databases
    46
    2

    View full-size slide

  47. @arafkarsh arafkarsh
    NoSQL Databases
    47
    Database Type ACID Query Use Case
    Couchbase
    Doc Based,
    Key Value
    Open Source Yes N1QL
    Financial Services, Inventory,
    IoT
    Cassandra Wide Column Open Source No CQL
    Social Analytics
    Retail, Messaging
    Neo4J Graph
    Open Source
    Commercial
    Yes Cypher
    AI, Master Data Mgmt
    Fraud Protection
    Redis Key Value Open Source Yes Many languages Caching, Queuing
    Mongo DB Doc Based
    Open Source
    Commercial
    Yes JS
    IoT, Feal Time Analytics
    Inventory,
    Amazon
    Dynamo DB
    Key Value
    Doc based
    Vendor Yes DQL
    Gamming, Retail, Financial
    Services
    Source: https://searchdatamanagement.techtarget.com/infographic/NoSQL-database-comparison-to-help-you-choose-the-right-store.

    View full-size slide

  48. @arafkarsh arafkarsh
    SQL Vs NoSQL
    48
    SQL NoSQL
    Database Type Relational Non-Relational
    Schema Pre-Defined Dynamic Schema
    Database Category Table Based
    1. Documents
    2. Key Value Stores
    3. Graph Stores
    4. Wide Column Stores
    Queries
    Complex Queries (Standard SQL for
    all Relational Databases)
    Need to apply Special Query language for
    each type of NoSQL DB.
    Hierarchical Storage Not a Good Fit Perfect
    Scalability
    Scales well for traditional
    Applications
    Scales well for Modern heavy data-oriented
    Application
    Query Language
    SQL – Standard Language across all
    the Databases
    Non-Standard Query Language as each of the
    NoSQL DB is different.
    ACID Support Yes For some of the Database (Ex. MongoDB)
    Data Size Good for traditional Applications
    Handles massive amount of Data for the
    Modern App requirements.

    View full-size slide

  49. @arafkarsh arafkarsh
    SQL Vs NoSQL (MongoDB)
    49
    1. In MongoDB Transactional Properties are scoped at Doc Level.
    2. One or More fields can be atomically written in a Single Operation.
    3. With Updates to multiple sub documents including nested arrays.
    4. Any Error results in the entire operation to Roll back.
    5. This is at par with Data Integrity Guarantees provided Traditional Databases.

    View full-size slide

  50. @arafkarsh arafkarsh
    Multi Table / Doc ACID Transactions
    50
    Examples – Systems of Record or Line of Business (LoB) Applications
    1. Finance
    1. Moving funds between Bank Accounts,
    2. Payment Processing Systems
    3. Trading Platforms
    2. Supply Chain
    • Transferring ownership of Goods & Services through Supply
    Chains and Booking Systems – Ex. Adding Order and Reducing
    inventory.
    3. Billing System
    1. Adding a Call Detail Record and then updating Monthly Plan.
    Source: ACID Transactions in MongoDB

    View full-size slide

  51. @arafkarsh arafkarsh
    Redis
    • Data Structures
    • Design Patterns
    51
    2020 2019 NoSQL Database Model
    1 1 Redis Key-Value, Multi Model
    2 2 Amazon DynamoDB Multi Model
    3 3 Microsoft Cosmos Multi Model
    4 4 Memcached Key-Value
    In-Memory Databases

    View full-size slide

  52. @arafkarsh arafkarsh
    Why do you need In-Memory Databases
    52
    1 Users 1 Million +
    2 Data Volume Terabytes to Petabytes
    3 Locality Global
    4 Performance Microsecond Latency
    5 Request Rate Millions Per Second
    6 Access Mobile, IoT, Devices
    7 Economics Pay as you go
    8 Developer Access Open API
    Source: AWS re:Invent 2020: https://www.youtube.com/watch?v=2WkJeofqIJg

    View full-size slide

  53. @arafkarsh arafkarsh
    Tables / Docs (JSON) – Why Redis is different?
    53
    • Redis is a Multi data model Key Store
    • Commands operate on Keys
    • Data types of Keys can change overtime
    Source: https://www.youtube.com/watch?v=ELk_W9BBTDU

    View full-size slide

  54. @arafkarsh arafkarsh
    Keys, Values & Data Types
    54
    movie:StarWars “Sold Out”
    Key Name Value
    String
    Hash
    List
    Set
    Sorted Set
    Basic Data Types
    Key Properties
    • Unique
    • Binary Safe (Case Sensitive)
    • Max Size = 512 MB
    Expiration / TTL
    • By Default – Keys are retained
    • Time in Seconds, Milli Second, Unix Epoch
    • Added / Removed from Key Ø SET movie:StarWars ex 5000 (Expires in 5000 seconds)
    Ø PEXPIRE movie:StarWars 5 (set for 5 milli seconds)
    https://redis.io/commands/set

    View full-size slide

  55. @arafkarsh arafkarsh
    Redis – Remote Dictionary Server
    55
    Distributed In-Memory Data Store
    String Standard String data
    Hash { A: “John Doe”, B: “New York”, C:USA” }
    List [ A -> B -> C -> D. -> E ]
    Set { A, B, C, D, E }
    Sorted Set { A:10, B:12, C:14:, D:20, E:32 }
    Stream … msg1, msg2, msg3
    Pub / Sub … msg1, msg2, msg3
    https://redis.io/topics/data-types

    View full-size slide

  56. @arafkarsh arafkarsh
    Data Type: Hash
    56
    movie:The-Force-Awakens
    Value
    J. J. Abrams
    L. Kasdan, J. J. Abrams, M. Arndt
    Dan Mindel
    Ø HGET movie:The-Force-Awakens Director
    “J. J. Abrams”
    • Field & Value Pairs
    • Single Level
    • Add and Remove Fields
    • Set Operations
    • Intersect
    • Union
    https://redis.io/topics/data-types
    https://redis.io/commands#hash
    Key Name
    Director
    Writer
    Cinematography
    Field
    Use Cases
    • Session Cache
    • Rate Limiting

    View full-size slide

  57. @arafkarsh arafkarsh
    Data Type: List
    57
    movies
    Key Name
    “Force Awakens, The” “Last Jedi, The” “Rise of Skywalker, The”
    Ø LPOP movies
    “Force Awakens, The”
    Ø LPOP movies
    “Last Jedi, The”
    Ø RPOP movies
    “Rise of Skywalker, The”
    Ø RPOP movies
    “Last Jedi, The”
    • Ordered List (FIFO or LIFO)
    • Duplicates Allowed
    • Elements added from Left or Right or By Position
    • Max 4 Billion elements per List
    Type of Lists
    • Queues
    • Stacks
    • Capped List
    https://redis.io/topics/data-types
    https://redis.io/commands#list
    Use Cases
    • Communication
    • Activity List

    View full-size slide

  58. @arafkarsh arafkarsh
    Data Type: Set
    58
    movies
    Member / Element
    “Force Awakens, The”
    “Last Jedi, The”
    “Rise of Skywalker, The”
    Ø SMEMBERS movies
    “Force Awakens, The”
    “Last Jedi, The”
    “Rise of Skywalker, The”
    • Un-Ordered List of Unique
    Elements
    • Set Operations
    • Difference
    • Intersect
    • Union
    https://redis.io/topics/data-types
    https://redis.io/commands#set
    Key Name
    Use Cases
    • Unique Visitors

    View full-size slide

  59. @arafkarsh arafkarsh
    Data Type: Sorted Set
    59
    movies
    Value
    “Force Awakens, The”
    “Last Jedi, The”
    “Rise of Skywalker, The”
    Ø ZRANGE movies 0 1
    “Last Jedi, The”
    “Rise of Skywalker, The”
    • Ordered List of Unique
    Elements
    • Set Operations
    • Intersect
    • Union
    https://redis.io/topics/data-types
    https://redis.io/commands#set
    Key Name
    3
    1
    2
    Score
    Use Cases
    • Leaderboard
    • Priority Queues

    View full-size slide

  60. @arafkarsh arafkarsh
    Redis: Transactions
    60
    • Transactions are
    • Atomic
    • Isolated
    • Redis commands are
    queue
    • All the Queued commands
    are executed sequentially
    as an Atomic unit
    Ø MULTI
    Ø SET movie:The-Force-Awakens:Review Good
    Ø INCR movie:The-Force-Awakens:Rating
    Ø EXEC

    View full-size slide

  61. @arafkarsh arafkarsh
    Redis In-Memory Data Store Use cases
    61
    Machine
    Learning
    Message
    Queues
    Gaming
    Leaderboards
    Geospatial
    Session
    Store
    Media
    Streaming
    Real-time
    Analytics
    Caching

    View full-size slide

  62. @arafkarsh arafkarsh
    Use Case: Sorted Set – Leader Board
    62
    • Collection of Sorted Distinct
    Entities
    • Set Operations and Range
    Queries based on Score
    value: John
    score: 610
    value : Jane
    score: 987
    value : Sarah
    score: 1597
    value : Maya
    score: 144
    value : Fred
    score: 233
    value : Ann
    score: 377
    Game Scores
    Ø ZADD game:1 987 Jane 1597 Sarah 377 Maya 610 John 144
    Ann 233 Fred
    Ø ZREVRANGE game:1 0 3 WITHSCORES. (Get top 4 Scores)
    • Sarah 1597
    • Jane 987
    • John 610
    • Ann 377
    Source: AWS re:Invent 2020: https://www.youtube.com/watch?v=2WkJeofqIJg
    https://redis.io/commands/zadd

    View full-size slide

  63. @arafkarsh arafkarsh
    Use Case: Geospatial
    63
    • Compute distance between
    members
    • Find all members within a
    radius
    Source: AWS re:Invent 2020: https://www.youtube.com/watch?v=2WkJeofqIJg
    Ø GEOADD cities 87.6298 41.8781 Chicago
    Ø GEOADD cities 122.3321 447.6062 Seattle
    Ø ZRANGE cities0 -1
    • “Chicago”
    • “Seattle”
    Ø GEODIST cities Chicago Seattle mi
    • “1733.4089”
    Ø GEORADIUS cities 122.4194 37..7749
    1000 mi WITHDIST
    • “Seattle”
    • “679.4848”
    o m for meters
    o km for
    kilometres
    o mi for miles
    o ft for feet
    https://redis.io/commands/geodist

    View full-size slide

  64. @arafkarsh arafkarsh
    Use Case: Streams
    64
    • Ordered collection of Data
    • Efficient for consuming
    from the tail
    • Multiple Consumers
    support similar to Kafka
    {
    “order”: “xy2123adbcd”
    {
    “item”: “book1”,
    “qty”: 1
    }
    }
    {
    “order”: “xy2123adbcd”
    {
    “item”: “book1”,
    “qty”: 1
    }
    }
    {
    “order”: “xy2123adbcd”
    {
    “item”: “book1”,
    “qty”: 1
    }
    }
    {
    “order”: “xy2123adbcd”
    {
    “item”: “book1”,
    “qty”: 1
    }
    }
    {
    “order”: “xy2123adbcd”
    {
    “item”: “book1”,
    “qty”: 1
    }
    }
    {
    “order”: “xy2123adbcd”
    {
    “item”: “book1”,
    “qty”: 1
    }
    }
    {
    “order”: “xy2123adbcd”
    {
    “item”: “book1”,
    “qty”: 1
    }
    }
    {
    “order”: “xy2123adbcd”
    {
    “item”: “book1”,
    “qty”: 1
    }
    }
    {
    “order”: “xy2123adbcd”
    {
    “item”: “book1”,
    “qty”: 1
    }
    }
    {
    “order”: “xy2123adbcd”
    {
    “item”: “book1”,
    “qty”: 1
    }
    }
    {
    “order”: “xy2123adbcd”
    {
    “item”: “book1”,
    “qty”: 1
    }
    }
    START
    END
    Consumer 1
    Consumer 2
    Consumer n
    Consumer G1
    Consumer G2
    Consumer Group G
    Ø XADD orderStream * orderId1:item1:qty1
    Ø XADD orderStream * orderId2:item1:qty2
    https://redis.io/commands/xadd
    * Autogenerates the Uniq ID
    Producer
    Ø XREAD BLOCK 20 STREAMS orderStream $
    • orderId2
    • Item1
    • qty2
    Consumer

    View full-size slide

  65. @arafkarsh arafkarsh
    MongoDB: Design Patterns
    1. Prefer Embedding
    2. Embrace Duplication
    3. Know when Not to Embed
    4. Relationships and Join
    65

    View full-size slide

  66. @arafkarsh arafkarsh
    MongoDB Docs – Prefer Embedding
    66
    Use
    Structure
    to use
    Data
    within a
    Document
    Include
    Bounded
    Arrays to
    have
    multiple
    records

    View full-size slide

  67. @arafkarsh arafkarsh
    MongoDB Docs – Embrace Duplication
    67
    Field Info
    Duplicated
    from
    Customer
    Profile
    Address
    Duplicated
    from
    Customer
    Profile

    View full-size slide

  68. @arafkarsh arafkarsh
    Know When Not to Embed
    68
    As Item is used outside
    of Order, You don’t
    need to embed the
    whole Object here.
    Instead give the Item
    Reference ID.
    (Not to Embed)
    Name is given to
    decouple it from Item
    (Product) Service.
    (Embrace Duplication)

    View full-size slide

  69. @arafkarsh arafkarsh
    Relationships and Joins
    69
    Reviews are joined to Product
    Collection using Item UUID
    Bi-Directional Joins are also
    supported

    View full-size slide

  70. @arafkarsh arafkarsh
    MongoDB – Tips & Best Practices
    70
    1. MongoDB Will Abort any Multi Document transaction that runs for more
    than 60 seconds.
    2. No More than 1000 documents should be modified within a
    Transaction.
    3. Developers need to try logic to retry the transaction in case transaction
    is aborted due to network error.
    4. Transactions that affects Multiple Shards incur a greater performance
    Cost as operations are coordinated across multiple participating nodes
    over the network.
    5. Performance will be impacted if a transaction runs against a collection
    that is subject to rebalancing.

    View full-size slide

  71. @arafkarsh arafkarsh
    Amazon DynamoDB
    DynamoDB Concepts
    DynamoDB Design Patterns
    Performance

    View full-size slide

  72. @arafkarsh arafkarsh
    Amazon DynamoDB Concept
    Customer ID Name Category State
    Order
    Order
    Customer
    Cart
    Payments
    Order
    Cart
    Catalogue
    Catalogue
    Table
    Product ID Name Value Description Image
    Item ID Quantity Value Currency
    User ID + Item ID
    Attributes
    1. A single Table holds multiple Entities (Customer, Catalogue, Cart, Order
    etc.) aka Items.
    2. Item contains a collection of Attributes.
    3. Primary Key plays a key role in Performance, Scalability and avoiding Joins
    (in a typical RDBMS way).
    4. Primary Key contains a Partition Key and an option Sort Key.
    5. Item Data Model is JSON, and Attribute can be a field or a Custom Object.
    Items
    Primary Key

    View full-size slide

  73. @arafkarsh arafkarsh
    DynamoDB – Under the Hood
    One Single table Multiple Entities with multiple documents (Records in RDBMS style)
    1 Org Record
    2 Employee Record
    1 Org Record
    2 Employee Record
    1. DynamoDB Structure is JSON (Document Model) – However, it has no resemblance to MongoDB in terms DB
    implementation or Schema Design Patterns.
    2. Multiple Entities are part of the Single Table and this helps to avoid expensive joins. For Ex. PK = ORG#Magna
    will retrieve all the 3 records. 1 Record from Org Entity and 2 Records from Employee Entity.
    3. Partition Key helps in Sharding and Horizontal Scalability.

    View full-size slide

  74. @arafkarsh arafkarsh
    Neo4J Graph Database
    74

    View full-size slide

  75. @arafkarsh arafkarsh
    Features
    75
    In a Graph Database, data is represented as nodes (also called vertices) and edges (also called
    relationships or connections).
    • Nodes represent entities or objects, while
    • Edges represent the relationships or connections between those entities.
    • Both nodes and edges can have properties (key-value pairs) that store additional information
    about the entities or their relationships.
    The main components of a Graph Database are:
    1. Nodes: The fundamental units representing entities, such as people, products, or locations.
    2. Edges: The connections between nodes, representing relationships, such as "friends with,"
    "purchased," or "lives in."
    3. Properties: Key-value pairs that store additional information about nodes or edges, such as
    names, ages, or timestamps.

    View full-size slide

  76. @arafkarsh arafkarsh
    Advantages
    76
    1. Flexibility: Graph Databases can quickly adapt to changes in the
    data model and accommodate the addition or removal of nodes,
    edges, or properties without significant disruption.
    2. Performance: Graph Databases are optimized for querying
    connected data, allowing for faster traversal of relationships
    compared to traditional relational databases.
    3. Intuitive Representation: Graph Databases represent data in a way
    that closely mirrors real-world entities and their relationships,
    making it easier to understand and work with the data.

    View full-size slide

  77. @arafkarsh arafkarsh
    Example: Healthcare App
    77
    Nodes:
    1. Patients
    2. Doctors
    3. Hospitals
    4. Diagnoses
    5. Treatments
    6. Medications
    7. Insurances
    Edges: The edges would represent
    relationships between these entities,
    such as:
    1. Patient 'visited' Doctor
    2. Doctor 'works at a' Hospital
    3. Patient 'diagnosed with' Diagnosis
    4. Diagnosis 'treated with' Treatment
    5. Treatment 'involves' Medication
    6. Patient 'covered by’ Insurance
    Properties: Nodes and edges could
    have properties to store additional
    information about the entities or
    their relationships, such as:
    1. Patient: name, age, gender, medical
    history
    2. Doctor: name, specialty, experience,
    ratings
    3. Hospital: name, location, facilities,
    ratings
    4. Diagnosis: name, description,
    prevalence, risk factors
    5. Treatment: name, type, duration,
    success rate
    6. Medication: name, dosage, side
    effects, interactions
    7. Insurance: company, coverage,
    premium, limitations
    8. Visited: Date & Time, Doctors Name,
    Hospital

    View full-size slide

  78. @arafkarsh arafkarsh
    How the data is represented in Graph DB
    78
    P1
    D1
    p1.d1.v1, Date, Clinic
    p1.d1.v2, Date, Clinic
    p1.d1.v3, Date, Clinic
    Node P1
    Patient Name
    DOB
    Phone
    Node D1
    Doctor Name
    Phone
    Specialty
    D2
    p1.d2.v1, Date, Clinic
    p1.d2.v2, Date, Clinic
    P2
    Edges
    Node D2
    Doctor Name
    Phone
    Specialty
    p2.d2.v1, Date, Clinic
    L1
    Node L1
    Lab Name
    Phone
    Specialty
    Node P2
    Patient Name
    DOB
    Phone
    p2.l1.v1, Date,
    M1
    p2.m
    1.v1, Date,
    p1.m
    1.v1, Date,
    p1.m
    1.v2, Date,
    Node M1
    Pharmacy
    Phone
    Type (Internal)
    Edges with
    Properties
    Nodes
    with
    Properties

    View full-size slide

  79. @arafkarsh arafkarsh
    Sample Code
    79
    MATCH (p:Patient {patient_id: "P1"}), (d:Doctor {doctor_id: "D1"})
    CREATE (p)-[:VISITED {date: "2023-05-01"}]->(d)
    MATCH (p:Patient {patient_id: "P1"}), (diag:Diagnosis {diagnosis_id: "Dg1"})
    CREATE (p)-[:DIAGNOSED {date: "2023-05-02"}]->(diag)
    MATCH (p:Patient {patient_id: "P1"}), (l:Lab {lab_id: "L1"})
    CREATE (p)-[:HAD_XRAY {date: "2023-05-03"}]->(l)
    CREATE (p:Patient {patient_id: "P1", name: "John Doe"})
    CREATE (d:Doctor {doctor_id: "D1", name: "Dr. Smith"})
    CREATE (diag:Diagnosis {diagnosis_id: "Dg1", name: "Flu"})
    CREATE (l:Lab {lab_id: "L1", name: "X-Ray Lab"})

    View full-size slide

  80. @arafkarsh arafkarsh
    Neo4J – Graph Data Science (GDS) Library
    80
    1. Graph traversal algorithms:
    1. Depth-First Search (DFS)
    2. Breadth-First Search (BFS)
    2. Shortest path algorithms:
    1. Dijkstra's algorithm
    2. A* (A-Star) algorithm
    3. All Pairs Shortest Path (APSP)
    3. Centrality algorithms:
    1. Degree Centrality: Computes a node's incoming and outgoing number of relationships.
    2. Closeness Centrality: Measures how central a node is to its neighbors.
    3. Betweenness Centrality: Measures the importance of a node based on the number of
    shortest paths passing through it.
    4. PageRank: A popular centrality algorithm initially designed to rank web pages based on
    the idea that more important nodes will likely receive more connections from other
    nodes.

    View full-size slide

  81. @arafkarsh arafkarsh
    Neo4J – Graph Data Science (GDS) Library
    81
    4. Community detection algorithms:
    1. Label Propagation: A fast algorithm for detecting communities within a graph based on
    propagating labels to form clusters.
    2. Louvain Modularity: An algorithm for detecting communities by optimizing a modularity
    score.
    3. Weakly Connected Components: Identifies groups of nodes where each node is reachable
    from any other node within the same group, disregarding the direction of relationships.
    5.Similarity algorithms:
    1. Jaccard Similarity: Measures the similarity between two sets by comparing their intersection
    and union.
    2. Cosine Similarity: Measures the similarity between two vectors based on the cosine of the
    angle between them.
    3. Pearson Similarity: Measures the similarity between two vectors based on their Pearson
    correlation coefficient.
    6.Pathfinding algorithms:
    1. Minimum Weight Spanning Tree: Computes the minimum weight spanning tree for a
    connected graph, using algorithms like Kruskal's or Prim's.

    View full-size slide

  82. @arafkarsh arafkarsh
    Scalability: Sharding / Partitions
    • Scale Cube
    • eBay Case Study
    • Sharding and Partitions
    82
    3

    View full-size slide

  83. @arafkarsh arafkarsh
    Scalability
    • Scale Cube
    • eBay Case Study
    83

    View full-size slide

  84. @arafkarsh arafkarsh
    App Scalability based
    on micro services
    architecture
    Source: The NewStack. Based on the Art of Scalability by By Martin Abbot
    & Michael Fisher
    84

    View full-size slide

  85. @arafkarsh arafkarsh
    Scale Cube and Micro Services
    85
    1. Functional
    Decomposition
    2. Avoid locks by
    Database Sharding
    3. Cloning Services

    View full-size slide

  86. @arafkarsh arafkarsh
    Scalability Best Practices : Lessons from
    Best Practices Highlights
    #1 Partition By Function
    • Decouple the Unrelated Functionalities.
    • Selling functionality is served by one set of applications, bidding by another, search by yet another.
    • 16,000 App Servers in 220 different pools
    • 1000 logical databases, 400 physical hosts
    #2 Split Horizontally
    • Break the workload into manageable units.
    • eBay’s interactions are stateless by design
    • All App Servers are treated equal and none retains any transactional state
    • Data Partitioning based on specific requirements
    #3
    Avoid Distributed
    Transactions
    • 2 Phase Commit is a pessimistic approach comes with a big COST
    • CAP Theorem (Consistency, Availability, Partition Tolerance). Apply any two at any point in time.
    • @ eBay No Distributed Transactions of any kind and NO 2 Phase Commit.
    #4
    Decouple Functions
    Asynchronously
    • If Component A calls component B synchronously, then they are tightly coupled. For such systems to
    scale A you need to scale B also.
    • If Asynchronous A can move forward irrespective of the state of B
    • SEDA (Staged Event Driven Architecture)
    #5
    Move Processing to
    Asynchronous Flow
    • Move as much processing towards Asynchronous side
    • Anything that can wait should wait
    #6 Virtualize at All Levels • Virtualize everything. eBay created their on O/R layer for abstraction
    #7 Cache Appropriately • Cache Slow changing, read-mostly data, meta data, configuration and static data.
    86
    Source: http://www.infoq.com/articles/ebay-scalability-best-practices

    View full-size slide

  87. @arafkarsh arafkarsh
    Sharding & Partitions
    • Horizontal Sharding
    • Vertical Sharding
    • Partitioning (Vertical)
    • Geo Partitioning
    87

    View full-size slide

  88. @arafkarsh arafkarsh
    Sharding / Partitioning
    88
    Method Scalability Table
    Sharding Horizontal Rows Same Schema with
    Uniq Rows
    Sharding Vertical Columns Different Schema
    Partition Vertical Rows Same Schema with
    Uniq Rows
    1. Optimize the Database
    2. Separate Rows or Columns into multiple smaller tables
    3. Each table has either Same Schema with Unique Rows
    4. Or has a Schema that is subset of the Original
    Customer ID Customer
    Name
    DOB City
    1 ABC Bengaluru
    2 DEF Tokyo
    3 JHI Kochi
    4 KLM Pune
    Original Table
    Customer ID Customer
    Name
    DOB City
    1 ABC Bengaluru
    2 DEF Tokyo
    Customer ID Customer
    Name
    DOB City
    3 JHI Kochi
    4 KLM Pune
    Horizontal Sharding - 1
    Horizontal Sharding - 2
    Customer ID Customer
    Name
    DOB
    1 ABC
    2 DEF
    3 JHI
    4 KLM
    Customer ID City
    1 Bengaluru
    2 Tokyo
    3 Kochi
    4 Pune
    Vertical Sharding - 1 Vertical Sharding - 2

    View full-size slide

  89. @arafkarsh arafkarsh
    Sharding Scenarios
    89
    1. Horizontal Scaling: Single Server is unable to handle the load
    even after partitioning the data sets.
    2. Data can be partitioned in such a way that specific server(s)
    can serve the search query based on the partition. For Ex. In
    an e-Commerce Application Searching the data based on
    1. Product Type
    2. Product Brand
    3. Sellers Region (for Local Shipping)
    4. Orders based on Year / Months

    View full-size slide

  90. @arafkarsh arafkarsh
    Geo Partitioning
    90
    • Geo-partitioning is the ability to control the location of
    data at the row level.
    • CockroachDB lets you control which tables are replicated
    to which nodes. But with geo-partitioning, you can control
    which nodes house data with row-level granularity.
    • This allows you to keep customer data close to the user,
    which reduces the distance it needs to travel, thereby
    reducing latency and improving user experience.
    Source: https://www.cockroachlabs.com/blog/geo-partition-data-reduce-latency/

    View full-size slide

  91. @arafkarsh arafkarsh
    Oracle Database – Geo Partitioning
    91
    Source: https://www.oracle.com/a/tech/docs/sharding-wp-12c.pdf

    View full-size slide

  92. @arafkarsh arafkarsh
    Oracle Sharding and Geo
    92
    CREATE SHARDED TABLE customers (
    cust_id NUMBER NOT NULL ,
    name VARCHAR2(50) ,
    address VARCHAR2(250) ,
    geo VARCHAR2(20) ,
    class VARCHAR2(3) ,
    signup_date DATE ,
    CONSTRAINT cust_pk PRIMARY KEY(geo, cust_id) )
    PARTITIONSET BY LIST (geo)
    PARTITION BY CONSISTENT HASH (cust_id)
    PARTITIONS AUTO (
    PARTITIONSET AMERICA VALUES (‘AMERICA’) TABLESPACE SET tbs1,
    PARTITIONSET ASIA VALUES (‘ASIA’) TABLESPACE SET tbs2
    );
    Primary
    Shard
    Standby
    Shards
    Read / Write
    Tx / Second
    Read Only
    Tx / Second
    25 25 1.18 Million 1.62 Million
    50 50 2.11 Million 3.26 Million
    75 75 3.57 Million 5.05 Million
    100 100 4.38 Million 6.82 Million
    Linear Scalability
    Source: https://www.oracle.com/a/tech/docs/sharding-wp-12c.pdf

    View full-size slide

  93. @arafkarsh arafkarsh
    Oracle
    Sharding
    Compared with
    Cassandra and
    MongoDB
    93

    View full-size slide

  94. @arafkarsh arafkarsh
    MongoDB: Cluster
    1. Replication
    2. Automatic Failover
    3. Sharding
    94

    View full-size slide

  95. @arafkarsh arafkarsh
    MongoDB Replication
    95
    Application
    (Client App Driver)
    Replica Set1
    (mongos)
    RS 2
    (mongos)
    RS 3
    (mongos)
    Secondary Servers
    Primary Server
    Replication
    Replication
    Heartbeat
    Source: MongoDB Replication https://docs.mongodb.com/manual/replication/
    ü Provides redundancy
    High Availability.
    ü It provides Fault
    Tolerance as
    multiple copies of
    data on different
    database servers
    ensures that the loss
    of a single database
    server will not affect
    the Application.
    1. Replicate the primary's oplog and
    2. Apply the operations to their data
    sets such that the secondaries' data
    sets reflect the primary's data set.
    3. Secondary apply the operations to
    their data sets asynchronously
    What Secondary does?
    What Primary does?
    1. Receives all write operations
    mongodb://
    mongodb0.example.com:27017,
    mongodb1.example.com:27017,
    mongodb2.example.com:27017/?
    replicaSet=myRepl
    Use Secure Connection
    mongodb://myDBReader:D1fficultP%40ssw0rd
    @mongodb0.example.com:27017
    Replica Set Connection Configuration

    View full-size slide

  96. @arafkarsh arafkarsh
    MongoDB Replication: Automatic Failover
    96
    Source: MongoDB Replication https://docs.mongodb.com/manual/replication/
    ü If the Primary is NOT reachable
    more than the configured
    electionTimeoutMillis (default 10
    seconds) then
    ü One of the Secondary will become
    the Primary after an election
    process.
    ü Most updated Secondary will
    become the next Primary.
    ü Election should not take more
    than 12 seconds to elect a Primary.
    Replica Set1
    (mongos)
    RS 2
    (mongos)
    RS 3
    (mongos)
    Secondary Servers
    Primary Server
    Heartbeat
    Election for new Primary
    Replica Set1
    (mongos)
    Primary
    (mongos)
    RS 3
    (mongos)
    Secondary Servers
    Primary Server
    Heartbeat
    Election for new Primary
    Replication
    ü The write Operations will be blocked until the new Primary is selected.
    ü The Secondary Replica Set can serve the Read Operations while the election is in progress provided its configured for that.
    ü MongoDB 4.2+ compatible drivers enable retryable writes by default
    ü MongoDB 4.0 and 3.6-compatible drivers must explicitly enable retryable writes by including retryWrites=true in the connection
    string.

    View full-size slide

  97. @arafkarsh arafkarsh
    MongoDB Replication: Arbiter
    97
    Application
    (Client App Driver)
    Replica Set1
    (mongos)
    RS 2
    (mongos)
    Arbiter
    (mongos)
    Secondary Servers
    Primary Server
    Replication
    ü An Arbiter can be used to save the
    cost of adding an additional
    Secondary Server.
    ü Arbiter will handle only the election
    process to select a Primary.
    Source: MongoDB Replication https://docs.mongodb.com/manual/replication/

    View full-size slide

  98. @arafkarsh arafkarsh
    MongoDB Replication: Secondary Reads
    98
    Replica Set1
    (mongos)
    RS 2
    (mongos)
    RS 3
    (mongos)
    Secondary Servers
    Primary Server
    Replication
    Replication
    Heartbeat
    Source: MongoDB Replication https://docs.mongodb.com/manual/core/read-preference/
    ü Asynchronous replication to secondaries means
    that reads from secondaries may return data that
    does not reflect the state of the data on the
    primary.
    ü Multi-document transactions that contain read
    operations must use read preference primary. All
    operations in a given transaction must route to
    the same member.
    Write to Primary and Read from Secondary
    Application
    (Client App Driver)
    Read from
    the
    Secondary
    Write
    mongo ‘mongodb://mongodb0,mongodb1,mongodb2/?replicaSet=rsOmega&readPreference=secondary’
    $ >

    View full-size slide

  99. @arafkarsh arafkarsh
    MongoDB – Deploy Replica Set
    99
    mongod --replSet “rsOmega” --bind_ip localhost,
    $ >
    replication:
    replSetName: "rsOmega"
    net:
    bindIp: localhost,
    Config File
    mongod --config
    $ >
    Use Config file to set the Replica Config to each Mongo Instance
    Use Command Line to set Replica details to each Mongo Instance
    1
    Source: MongoDB Replication https://docs.mongodb.com/manual/tutorial/deploy-replica-set/

    View full-size slide

  100. @arafkarsh arafkarsh
    MongoDB – Deploy Replica Set
    100
    mongo
    $ >
    Initiate the Replica Set
    Connect to Mongo DB
    2
    > rs.initiate( {
    _id : "rsOmega",
    members: [
    { _id: 0, host: "mongodb0.host.com:27017" },
    { _id: 1, host: "mongodb1.host.com :27017" },
    { _id: 2, host: "mongodb2.host.com :27017" }
    ]
    })
    3
    Run rs.initiate() on just one and only one mongod instance for
    the replica set.
    Source: MongoDB Replication https://docs.mongodb.com/manual/tutorial/deploy-replica-set/

    View full-size slide

  101. @arafkarsh arafkarsh
    MongoDB – Deploy Replica Set
    101
    mongo ‘mongodb://mongodb0,mongodb1,mongodb2/?replicaSet=rsOmega’
    $ >
    > rs.conf()
    Show Config
    Show the Replica Config
    4
    Source: MongoDB Replication
    https://docs.mongodb.com/manual/tutorial/deploy-replica-set/
    > rs.status()
    5 Ensure that the replica set has a primary
    mongo
    $ >
    6 Connect to the Replica Set

    View full-size slide

  102. @arafkarsh arafkarsh
    MongoDB Sharding
    102
    Application
    (Client App Driver)
    Config Server
    (mongos)
    Config
    (mongos)
    Config
    (mongos)
    Secondary Servers
    Primary Server
    Router
    (mongos)
    Router
    (mongos)
    Replica Set1
    (mongos)
    RS 2
    (mongos)
    RS 3
    (mongos)
    Secondary Servers
    Primary Server
    Shard 1
    Replica Set1
    (mongos)
    RS 2
    (mongos)
    RS 3
    (mongos)
    Secondary Servers
    Primary Server
    Shard 2
    Replica Set1
    (mongos)
    RS 2
    (mongos)
    RS 3
    (mongos)
    Secondary Servers
    Primary Server
    Shard 3

    View full-size slide

  103. @arafkarsh arafkarsh
    Scalability Best Practices : Lessons from
    Best Practices Highlights
    #1 Partition By Function
    • Decouple the Unrelated Functionalities.
    • Selling functionality is served by one set of applications, bidding by another, search by yet another.
    • 16,000 App Servers in 220 different pools
    • 1000 logical databases, 400 physical hosts
    #2 Split Horizontally
    • Break the workload into manageable units.
    • eBay’s interactions are stateless by design
    • All App Servers are treated equal and none retains any transactional state
    • Data Partitioning based on specific requirements
    #3
    Avoid Distributed
    Transactions
    • 2 Phase Commit is a pessimistic approach comes with a big COST
    • CAP Theorem (Consistency, Availability, Partition Tolerance). Apply any two at any point in time.
    • @ eBay No Distributed Transactions of any kind and NO 2 Phase Commit.
    #4
    Decouple Functions
    Asynchronously
    • If Component A calls component B synchronously, then they are tightly coupled. For such systems to
    scale A you need to scale B also.
    • If Asynchronous A can move forward irrespective of the state of B
    • SEDA (Staged Event Driven Architecture)
    #5
    Move Processing to
    Asynchronous Flow
    • Move as much processing towards Asynchronous side
    • Anything that can wait should wait
    #6 Virtualize at All Levels • Virtualize everything. eBay created their on O/R layer for abstraction
    #7 Cache Appropriately • Cache Slow changing, read-mostly data, meta data, configuration and static data.
    Source: http://www.infoq.com/articles/ebay-scalability-best-practices

    View full-size slide

  104. @arafkarsh arafkarsh
    4
    Multi-Tenancy
    o Multi-Tenancy
    o Compliance
    o Data Security
    104

    View full-size slide

  105. @arafkarsh arafkarsh
    Multi-Tenancy
    • Separate Database
    • Shared Database Separate Schema
    • Shared Database Shared Schema
    105

    View full-size slide

  106. @arafkarsh arafkarsh
    Type of Multi-Tenancy
    106
    Tenant B
    Tenant A
    App
    Tenant A Tenant B
    1. Separate Database
    Tenant B
    Tenant A
    App
    Shared DB
    Tenant A
    Schema
    Tenant B
    Schema
    2. Shared Database,
    Separate Schema
    Tenant B
    Tenant A
    App
    Shared DB
    3. Shared Database,
    Shared Entity
    Entity
    • Tenant A
    • Tenant B

    View full-size slide

  107. @arafkarsh arafkarsh
    Hibernate: 1. Separate DB
    107
    import org.hibernate.SessionFactory;
    import org.hibernate.boot.registry.StandardServiceRegistryBuilder;
    import org.hibernate.cfg.Configuration;
    public class HibernateUtil {
    public static SessionFactory getSessionFactory(String tenantId) {
    Configuration configuration = new Configuration();
    configuration.configure("hibernate.cfg.xml");
    configuration.setProperty("hibernate.connection.url", "jdbc:postgresql://localhost:5432/" +
    tenantId);
    configuration.setProperty("hibernate.default_schema", tenantId);
    StandardServiceRegistryBuilder builder = new
    StandardServiceRegistryBuilder()
    .applySettings(configuration.getProperties());
    return configuration.buildSessionFactory(builder.build());
    }
    }

    View full-size slide

  108. @arafkarsh arafkarsh
    Hibernate: 2. Shared DB Separate Schema
    108
    import org.hibernate.boot.model.naming.Identifier;
    Import org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl;
    import org.hibernate.engine.jdbc.env.spi.JdbcEnvironment;
    import org.hibernate.SessionFactory;
    import org.hibernate.boot.registry.StandardServiceRegistryBuilder;
    import org.hibernate.cfg.Configuration;
    public class TenantAwareNamingStrategy extends PhysicalNamingStrategyStandardImpl {
    private String tenantId;
    public TenantAwareNamingStrategy(String tenantId) {
    this.tenantId = tenantId;
    } @Override
    public Identifier toPhysicalTableName(Identifier name, JdbcEnvironment context) {
    String tableName = tenantId + "_" + name.getText();
    return new Identifier(tableName, name.isQuoted());
    }
    }
    public class HibernateUtil {
    public static SessionFactory getSessionFactory(String tenantId) {
    Configuration configuration = new Configuration();
    configuration.configure("hibernate.cfg.xml");
    configuration.setPhysicalNamingStrategy(new TenantAwareNamingStrategy(tenantId));
    StandardServiceRegistryBuilder builder = new StandardServiceRegistryBuilder()
    .applySettings(configuration.getProperties());
    return configuration.buildSessionFactory(builder.build());
    }
    }

    View full-size slide

  109. @arafkarsh arafkarsh
    Hibernate: 3. Shared DB Shared Entity
    109
    import javax.persistence.Column;
    import javax.persistence.Entity;
    import javax.persistence.Table;
    import org.hibernate.annotations.Filter;
    import org.hibernate.annotations.FilterDef;
    import org.hibernate.annotations.ParamDef; I
    import org.hibernate.SessionFactory;
    import org.hibernate.boot.registry.StandardServiceRegistryBuilder;
    Import org.hibernate.cfg.Configuration;
    @Entity @Table(name = "employees")
    @FilterDef(name = "tenantFilter", parameters = {@ParamDef(name = "tenantId", type = "string")})
    @Filter(name = "tenantFilter", condition = "tenant_id = :tenantId")
    public class Employee {
    @Column(name = "tenant_id", nullable = false)
    private String tenantId; //
    // other columns …….. getters and setters
    }
    public class HibernateUtil {
    public static SessionFactory getSessionFactory() {
    Configuration configuration = new Configuration();
    configuration.configure("hibernate.cfg.xml");
    StandardServiceRegistryBuilder builder = new StandardServiceRegistryBuilder()
    .applySettings(configuration.getProperties());
    return configuration.buildSessionFactory(builder.build());
    }
    }

    View full-size slide

  110. @arafkarsh arafkarsh
    Multi-Tenancy in MongoDB, Redis, Cassandra
    110
    Feature MongoDB Redis Cassandra
    Separate databases
    / instances per
    tenant
    Separate databases per
    tenant
    Separate instances per
    tenant
    Separate keyspaces per
    tenant
    Strong data isolation,
    individual tenant backup
    and restoration.
    Complete data isolation,
    more resources and
    management overhead.
    Strong data isolation,
    simplifies tenant-specific
    backup and restoration.
    Shared database /
    instance with
    separate collections
    / tables /
    namespaces per
    tenant
    Shared database with
    separate collections per
    tenant
    Shared instance with
    separate namespaces per
    tenant
    Shared keyspace with
    separate tables per tenant
    Balances data isolation
    and resource utilization.
    Some level of data isolation,
    better resource utilization.
    Balances data isolation and
    resource utilization.
    Shared database /
    instance and shared
    collections / tables
    Shared database and
    shared collections
    Not applicable
    Shared keyspace and shared
    tables
    Optimizes resource
    utilization, requires
    careful implementation.
    Optimizes resource utilization,
    requires careful
    implementation to avoid data
    leaks.

    View full-size slide

  111. @arafkarsh arafkarsh
    Compliance
    • Best Practices
    • Case study: Health Care App
    • Case study: eCommerce App
    111

    View full-size slide

  112. @arafkarsh arafkarsh
    Best Practices: Health Care App
    112
    1. Understand the regulations: Familiarize yourself with HIPAA rules, including the Privacy Rule,
    Security Rule, and Breach Notification Rule. These rules outline the standards and requirements for
    protecting sensitive patient data, known as Protected Health Information (PHI).
    2. Implement Access Controls: Ensure only authorized users can access PHI by implementing robust
    authentication mechanisms like multi-factor authentication (MFA), role-based access controls
    (RBAC), and proper password policies.
    3. Encrypt Data: Use encryption for data at rest and in transit. Implement encryption technologies like
    SSL/TLS for data in transit and AES-256 for data at rest. Store encryption keys securely and
    separately from the data they protect.
    4. Regular Audits and Monitoring: Regularly audit and monitor your systems for security
    vulnerabilities and potential breaches. Implement logging mechanisms to track access to PHI and
    use intrusion detection systems to monitor for unauthorized access.
    5. Data Backups and Disaster Recovery: Implement a robust data backup and disaster recovery plan
    to ensure the availability and integrity of PHI in case of data loss or system failures.

    View full-size slide

  113. @arafkarsh arafkarsh
    Best Practices: Health Care App
    113
    6. Regular Risk Assessments: Conduct regular risk assessments to identify potential risks to PHI's
    confidentiality, integrity, and availability. Develop a risk management plan to address these risks and
    ensure continuous improvement of your security posture.
    7. Implement a Privacy Policy: Develop and maintain a privacy policy that clearly outlines how your
    organization handles and protects PHI. This policy should be easily accessible to users and should be
    updated regularly to reflect changes in your organization’s practices or regulations.
    8. Employee Training: Train employees on HIPAA regulations, your organization’s privacy policy, and
    security best practices. Regularly update and reinforce this training to ensure continued compliance.
    9. Business Associate Agreements (BAAs): Ensure that any third-party vendors, contractors, or partners
    who have access to PHI sign a Business Associate Agreement (BAA) that outlines their responsibilities
    for maintaining the privacy and security of PHI.
    10. Incident Response Plan: Develop an incident response plan to handle potential data breaches or
    security incidents. This plan should include procedures for identifying, containing, and mitigating
    breaches and notifying affected individuals and relevant authorities.

    View full-size slide

  114. @arafkarsh arafkarsh
    Best Practices: eCommerce App
    114
    1. Build and Maintain a Secure Network:
    1. Install and maintain a firewall configuration to protect cardholder data.
    2. Change vendor-supplied defaults for system passwords and other security parameters.
    2. Protect Cardholder Data:
    1. Encrypt transmission of cardholder data across open, public networks using SSL/TLS or other
    encryption methods.
    2. Avoid storing cardholder data whenever possible. If you must store data, use strong encryption and
    access controls, and securely delete data when no longer needed.
    3. Maintain a Vulnerability Management Program:
    1. Regularly update and patch all systems, including operating systems, applications, and security
    software, to protect against known vulnerabilities.
    2. Use and regularly update anti-virus software or programs.
    4. Implement Strong Access Control Measures:
    1. Restrict access to cardholder data by business need-to-know by implementing role-based access
    controls (RBAC).
    2. Implement robust authentication methods, such as multi-factor authentication (MFA), for all users
    with access to cardholder data.
    3. If applicable, restrict physical access to cardholder data by implementing physical security measures
    like secure storage, access controls, and surveillance.

    View full-size slide

  115. @arafkarsh arafkarsh
    Best Practices: eCommerce App
    115
    5. Regularly Monitor and Test Networks:
    1. Track and monitor all access to network resources and cardholder data using logging mechanisms and
    monitoring tools.
    2. Regularly test security systems and processes, including vulnerability scans, penetration tests, and file
    integrity monitoring.
    6. Maintain an Information Security Policy:
    1. Develop, maintain, and disseminate a comprehensive information security policy that addresses all
    PCI-DSS requirements and is reviewed and updated at least annually.
    7. Use Tokenization or Third-Party Payment Processors:
    1. Consider using tokenization or outsourcing payment processing to a PCI-DSS-compliant third-party
    provider. This can reduce the scope of compliance and protect cardholder data by minimizing the
    exposure of sensitive information within your systems.
    8. Educate and Train Employees:
    1. Train employees on PCI-DSS requirements, your company's security policies, and best practices for
    handling cardholder data securely. Regularly reinforce and update this training.
    9. Regularly Assess and Update Security Measures:
    1. Conduct regular risk assessments and security audits to identify potential risks and vulnerabilities.
    Update your security measures accordingly to maintain compliance and ensure the continued
    protection of cardholder data.

    View full-size slide

  116. @arafkarsh arafkarsh
    Data Security
    • Oracle
    • PostgreSQL
    • MySQL
    116

    View full-size slide

  117. @arafkarsh arafkarsh
    Oracle Data Security
    117
    • Row-level security: Oracle's Virtual Private Database (VPD) feature
    enables row-level security by adding a dynamic WHERE clause to SQL
    statements. You can use this feature to restrict access to specific rows
    based on user roles or attributes.
    • Column-level security: Oracle provides column-level security through the
    use of column masking with Data Redaction. This feature allows you to
    mask sensitive data in specific columns for unauthorized users, ensuring
    only authorized users can view the data.
    • Encryption: Oracle's Transparent Data Encryption (TDE) feature
    automatically encrypts data at rest without requiring changes to the
    application code. TDE encrypts data within the database files and
    automatically decrypts it when accessed by authorized users.

    View full-size slide

  118. @arafkarsh arafkarsh
    PostgreSQL Data Security
    118
    • Row-level security: PostgreSQL supports row-level security using Row
    Security Policies. These policies allow you to define access rules for
    specific rows based on user roles or attributes.
    • Column-level security: PostgreSQL provides column-level security using
    column privileges. You can grant or revoke specific privileges (e.g., SELECT,
    INSERT, UPDATE) on individual columns to control access to sensitive data.
    • Encryption: PostgreSQL does not have built-in transparent data
    encryption like Oracle. However, you can use third-party solutions, such
    as pgcrypto, to encrypt data within the database. You can use file-system
    level encryption or full-disk encryption for data at rest.

    View full-size slide

  119. @arafkarsh arafkarsh
    MySQL Data Security
    119
    • Row-level security: MySQL does not have built-in row-level security features
    like Oracle and PostgreSQL. However, you can implement row-level security by
    adding appropriate WHERE clauses to SQL statements in your application code,
    restricting access to specific rows based on user roles or attributes.
    • Column-level security: MySQL supports column-level security through column
    privileges. Like PostgreSQL, you can grant or revoke specific privileges on
    individual columns to control access to sensitive data.
    • Encryption: MySQL Enterprise Edition provides Transparent Data Encryption
    (TDE) to encrypt data at rest automatically. For the Community Edition, you can
    use file-system level encryption or full-disk encryption to protect data at rest.
    Data in transit can be encrypted using SSL/TLS.

    View full-size slide

  120. @arafkarsh arafkarsh 120
    100s Microservices
    1,000s Releases / Day
    10,000s Virtual Machines
    100K+ User actions / Second
    81 M Customers Globally
    1 B Time series Metrics
    10 B Hours of video streaming
    every quarter
    Source: NetFlix: : https://www.youtube.com/watch?v=UTKIT6STSVM
    10s OPs Engineers
    0 NOC
    0 Data Centers
    So what do NetFlix think about DevOps?
    No DevOps
    Don’t do lot of Process / Procedures
    Freedom for Developers & be Accountable
    Trust people you Hire
    No Controls / Silos / Walls / Fences
    Ownership – You Build it, You Run it.

    View full-size slide

  121. @arafkarsh arafkarsh 121
    50M Paid Subscribers
    100M Active Users
    60 Countries
    Cross Functional Team
    Full, End to End ownership of features
    Autonomous
    1000+ Microservices
    Source: https://microcph.dk/media/1024/conference-microcph-2017.pdf
    1000+ Tech Employees
    120+ Teams

    View full-size slide

  122. @arafkarsh arafkarsh 122
    Design Patterns are
    solutions to general
    problems that
    software developers
    faced during software
    development.
    Design Patterns

    View full-size slide

  123. @arafkarsh arafkarsh 123
    DREAM | AUTOMATE | EMPOWER
    Araf Karsh Hamid :
    India: +91.999.545.8627
    http://www.slideshare.net/arafkarsh
    https://www.linkedin.com/in/arafkarsh/
    https://www.youtube.com/user/arafkarsh/playlists
    http://www.arafkarsh.com/
    @arafkarsh
    arafkarsh

    View full-size slide

  124. @arafkarsh arafkarsh 124
    Source Code: https://github.com/MetaArivu Web Site: https://metarivu.com/ https://pyxida.cloud/

    View full-size slide

  125. @arafkarsh arafkarsh 125
    http://www.slideshare.net/arafkarsh

    View full-size slide

  126. @arafkarsh arafkarsh
    References
    126
    1. July 15, 2015 – Agile is Dead : GoTo 2015 By Dave Thomas
    2. Apr 7, 2016 - Agile Project Management with Kanban | Eric Brechner | Talks at Google
    3. Sep 27, 2017 - Scrum vs Kanban - Two Agile Teams Go Head-to-Head
    4. Feb 17, 2019 - Lean vs Agile vs Design Thinking
    5. Dec 17, 2020 - Scrum vs Kanban | Differences & Similarities Between Scrum & Kanban
    6. Feb 24, 2021 - Agile Methodology Tutorial for Beginners | Jira Tutorial | Agile Methodology Explained.
    Agile Methodologies

    View full-size slide

  127. @arafkarsh arafkarsh
    References
    127
    1. Vmware: What is Cloud Architecture?
    2. Redhat: What is Cloud Architecture?
    3. Cloud Computing Architecture
    4. Cloud Adoption Essentials:
    5. Google: Hybrid and Multi Cloud
    6. IBM: Hybrid Cloud Architecture Intro
    7. IBM: Hybrid Cloud Architecture: Part 1
    8. IBM: Hybrid Cloud Architecture: Part 2
    9. Cloud Computing Basics: IaaS, PaaS, SaaS
    1. IBM: IaaS Explained
    2. IBM: PaaS Explained
    3. IBM: SaaS Explained
    4. IBM: FaaS Explained
    5. IBM: What is Hypervisor?
    Cloud Architecture

    View full-size slide

  128. @arafkarsh arafkarsh
    References
    128
    Microservices
    1. Microservices Definition by Martin Fowler
    2. When to use Microservices By Martin Fowler
    3. GoTo: Sep 3, 2020: When to use Microservices By Martin Fowler
    4. GoTo: Feb 26, 2020: Monolith Decomposition Pattern
    5. Thought Works: Microservices in a Nutshell
    6. Microservices Prerequisites
    7. What do you mean by Event Driven?
    8. Understanding Event Driven Design Patterns for Microservices

    View full-size slide

  129. @arafkarsh arafkarsh
    References – Microservices – Videos
    129
    1. Martin Fowler – Micro Services : https://www.youtube.com/watch?v=2yko4TbC8cI&feature=youtu.be&t=15m53s
    2. GOTO 2016 – Microservices at NetFlix Scale: Principles, Tradeoffs & Lessons Learned. By R Meshenberg
    3. Mastering Chaos – A NetFlix Guide to Microservices. By Josh Evans
    4. GOTO 2015 – Challenges Implementing Micro Services By Fred George
    5. GOTO 2016 – From Monolith to Microservices at Zalando. By Rodrigue Scaefer
    6. GOTO 2015 – Microservices @ Spotify. By Kevin Goldsmith
    7. Modelling Microservices @ Spotify : https://www.youtube.com/watch?v=7XDA044tl8k
    8. GOTO 2015 – DDD & Microservices: At last, Some Boundaries By Eric Evans
    9. GOTO 2016 – What I wish I had known before Scaling Uber to 1000 Services. By Matt Ranney
    10. DDD Europe – Tackling Complexity in the Heart of Software By Eric Evans, April 11, 2016
    11. AWS re:Invent 2016 – From Monolithic to Microservices: Evolving Architecture Patterns. By Emerson L, Gilt D. Chiles
    12. AWS 2017 – An overview of designing Microservices based Applications on AWS. By Peter Dalbhanjan
    13. GOTO Jun, 2017 – Effective Microservices in a Data Centric World. By Randy Shoup.
    14. GOTO July, 2017 – The Seven (more) Deadly Sins of Microservices. By Daniel Bryant
    15. Sept, 2017 – Airbnb, From Monolith to Microservices: How to scale your Architecture. By Melanie Cubula
    16. GOTO Sept, 2017 – Rethinking Microservices with Stateful Streams. By Ben Stopford.
    17. GOTO 2017 – Microservices without Servers. By Glynn Bird.

    View full-size slide

  130. @arafkarsh arafkarsh
    References
    130
    Domain Driven Design
    1. Oct 27, 2012 What I have learned about DDD Since the book. By Eric Evans
    2. Mar 19, 2013 Domain Driven Design By Eric Evans
    3. Jun 02, 2015 Applied DDD in Java EE 7 and Open Source World
    4. Aug 23, 2016 Domain Driven Design the Good Parts By Jimmy Bogard
    5. Sep 22, 2016 GOTO 2015 – DDD & REST Domain Driven API’s for the Web. By Oliver Gierke
    6. Jan 24, 2017 Spring Developer – Developing Micro Services with Aggregates. By Chris Richardson
    7. May 17. 2017 DEVOXX – The Art of Discovering Bounded Contexts. By Nick Tune
    8. Dec 21, 2019 What is DDD - Eric Evans - DDD Europe 2019. By Eric Evans
    9. Oct 2, 2020 - Bounded Contexts - Eric Evans - DDD Europe 2020. By. Eric Evans
    10. Oct 2, 2020 - DDD By Example - Paul Rayner - DDD Europe 2020. By Paul Rayner

    View full-size slide

  131. @arafkarsh arafkarsh
    References
    131
    Event Sourcing and CQRS
    1. IBM: Event Driven Architecture – Mar 21, 2021
    2. Martin Fowler: Event Driven Architecture – GOTO 2017
    3. Greg Young: A Decade of DDD, Event Sourcing & CQRS – April 11, 2016
    4. Nov 13, 2014 GOTO 2014 – Event Sourcing. By Greg Young
    5. Mar 22, 2016 Building Micro Services with Event Sourcing and CQRS
    6. Apr 15, 2016 YOW! Nights – Event Sourcing. By Martin Fowler
    7. May 08, 2017 When Micro Services Meet Event Sourcing. By Vinicius Gomes

    View full-size slide

  132. @arafkarsh arafkarsh
    References
    132
    Kafka
    1. Understanding Kafka
    2. Understanding RabbitMQ
    3. IBM: Apache Kafka – Sept 18, 2020
    4. Confluent: Apache Kafka Fundamentals – April 25, 2020
    5. Confluent: How Kafka Works – Aug 25, 2020
    6. Confluent: How to integrate Kafka into your environment – Aug 25, 2020
    7. Kafka Streams – Sept 4, 2021
    8. Kafka: Processing Streaming Data with KSQL – Jul 16, 2018
    9. Kafka: Processing Streaming Data with KSQL – Nov 28, 2019

    View full-size slide

  133. @arafkarsh arafkarsh
    References
    133
    Databases: Big Data / Cloud Databases
    1. Google: How to Choose the right database?
    2. AWS: Choosing the right Database
    3. IBM: NoSQL Vs. SQL
    4. A Guide to NoSQL Databases
    5. How does NoSQL Databases Work?
    6. What is Better? SQL or NoSQL?
    7. What is DBaaS?
    8. NoSQL Concepts
    9. Key Value Databases
    10. Document Databases
    11. Jun 29, 2012 – Google I/O 2012 - SQL vs NoSQL: Battle of the Backends
    12. Feb 19, 2013 - Introduction to NoSQL • Martin Fowler • GOTO 2012
    13. Jul 25, 2018 - SQL vs NoSQL or MySQL vs MongoDB
    14. Oct 30, 2020 - Column vs Row Oriented Databases Explained
    15. Dec 9, 2020 - How do NoSQL databases work? Simply Explained!
    1. Graph Databases
    2. Column Databases
    3. Row Vs. Column Oriented Databases
    4. Database Indexing Explained
    5. MongoDB Indexing
    6. AWS: DynamoDB Global Indexing
    7. AWS: DynamoDB Local Indexing
    8. Google Cloud Spanner
    9. AWS: DynamoDB Design Patterns
    10. Cloud Provider Database Comparisons
    11. CockroachDB: When to use a Cloud DB?

    View full-size slide

  134. @arafkarsh arafkarsh
    References
    134
    Docker / Kubernetes / Istio
    1. IBM: Virtual Machines and Containers
    2. IBM: What is a Hypervisor?
    3. IBM: Docker Vs. Kubernetes
    4. IBM: Containerization Explained
    5. IBM: Kubernetes Explained
    6. IBM: Kubernetes Ingress in 5 Minutes
    7. Microsoft: How Service Mesh works in Kubernetes
    8. IBM: Istio Service Mesh Explained
    9. IBM: Kubernetes and OpenShift
    10. IBM: Kubernetes Operators
    11. 10 Consideration for Kubernetes Deployments
    Istio – Metrics
    1. Istio – Metrics
    2. Monitoring Istio Mesh with Grafana
    3. Visualize your Istio Service Mesh
    4. Security and Monitoring with Istio
    5. Observing Services using Prometheus, Grafana, Kiali
    6. Istio Cookbook: Kiali Recipe
    7. Kubernetes: Open Telemetry
    8. Open Telemetry
    9. How Prometheus works
    10. IBM: Observability vs. Monitoring

    View full-size slide

  135. @arafkarsh arafkarsh
    References
    135
    1. Feb 6, 2020 – An introduction to TDD
    2. Aug 14, 2019 – Component Software Testing
    3. May 30, 2020 – What is Component Testing?
    4. Apr 23, 2013 – Component Test By Martin Fowler
    5. Jan 12, 2011 – Contract Testing By Martin Fowler
    6. Jan 16, 2018 – Integration Testing By Martin Fowler
    7. Testing Strategies in Microservices Architecture
    8. Practical Test Pyramid By Ham Vocke
    Testing – TDD / BDD

    View full-size slide

  136. @arafkarsh arafkarsh 136
    1. Simoorg : LinkedIn’s own failure inducer framework. It was designed to be easy to extend and
    most of the important components are plug- gable.
    2. Pumba : A chaos testing and network emulation tool for Docker.
    3. Chaos Lemur : Self-hostable application to randomly destroy virtual machines in a BOSH-
    managed environment, as an aid to resilience testing of high-availability systems.
    4. Chaos Lambda : Randomly terminate AWS ASG instances during business hours.
    5. Blockade : Docker-based utility for testing network failures and partitions in distributed
    applications.
    6. Chaos-http-proxy : Introduces failures into HTTP requests via a proxy server.
    7. Monkey-ops : Monkey-Ops is a simple service implemented in Go, which is deployed into an
    OpenShift V3.X and generates some chaos within it. Monkey-Ops seeks some OpenShift
    components like Pods or Deployment Configs and randomly terminates them.
    8. Chaos Dingo : Chaos Dingo currently supports performing operations on Azure VMs and VMSS
    deployed to an Azure Resource Manager-based resource group.
    9. Tugbot : Testing in Production (TiP) framework for Docker.
    Testing tools

    View full-size slide

  137. @arafkarsh arafkarsh
    References
    137
    CI / CD
    1. What is Continuous Integration?
    2. What is Continuous Delivery?
    3. CI / CD Pipeline
    4. What is CI / CD Pipeline?
    5. CI / CD Explained
    6. CI / CD Pipeline using Java Example Part 1
    7. CI / CD Pipeline using Ansible Part 2
    8. Declarative Pipeline vs Scripted Pipeline
    9. Complete Jenkins Pipeline Tutorial
    10. Common Pipeline Mistakes
    11. CI / CD for a Docker Application

    View full-size slide

  138. @arafkarsh arafkarsh
    References
    138
    DevOps
    1. IBM: What is DevOps?
    2. IBM: Cloud Native DevOps Explained
    3. IBM: Application Transformation
    4. IBM: Virtualization Explained
    5. What is DevOps? Easy Way
    6. DevOps?! How to become a DevOps Engineer???
    7. Amazon: https://www.youtube.com/watch?v=mBU3AJ3j1rg
    8. NetFlix: https://www.youtube.com/watch?v=UTKIT6STSVM
    9. DevOps and SRE: https://www.youtube.com/watch?v=uTEL8Ff1Zvk
    10. SLI, SLO, SLA : https://www.youtube.com/watch?v=tEylFyxbDLE
    11. DevOps and SRE : Risks and Budgets : https://www.youtube.com/watch?v=y2ILKr8kCJU
    12. SRE @ Google: https://www.youtube.com/watch?v=d2wn_E1jxn4

    View full-size slide

  139. @arafkarsh arafkarsh
    References
    139
    1. Lewis, James, and Martin Fowler. “Microservices: A Definition of This New Architectural Term”, March 25, 2014.
    2. Miller, Matt. “Innovate or Die: The Rise of Microservices”. e Wall Street Journal, October 5, 2015.
    3. Newman, Sam. Building Microservices. O’Reilly Media, 2015.
    4. Alagarasan, Vijay. “Seven Microservices Anti-patterns”, August 24, 2015.
    5. Cockcroft, Adrian. “State of the Art in Microservices”, December 4, 2014.
    6. Fowler, Martin. “Microservice Prerequisites”, August 28, 2014.
    7. Fowler, Martin. “Microservice Tradeoffs”, July 1, 2015.
    8. Humble, Jez. “Four Principles of Low-Risk Software Release”, February 16, 2012.
    9. Zuul Edge Server, Ketan Gote, May 22, 2017
    10. Ribbon, Hysterix using Spring Feign, Ketan Gote, May 22, 2017
    11. Eureka Server with Spring Cloud, Ketan Gote, May 22, 2017
    12. Apache Kafka, A Distributed Streaming Platform, Ketan Gote, May 20, 2017
    13. Functional Reactive Programming, Araf Karsh Hamid, August 7, 2016
    14. Enterprise Software Architectures, Araf Karsh Hamid, July 30, 2016
    15. Docker and Linux Containers, Araf Karsh Hamid, April 28, 2015

    View full-size slide

  140. @arafkarsh arafkarsh
    References
    140
    16. MSDN – Microsoft https://msdn.microsoft.com/en-us/library/dn568103.aspx
    17. Martin Fowler : CQRS – http://martinfowler.com/bliki/CQRS.html
    18. Udi Dahan : CQRS – http://www.udidahan.com/2009/12/09/clarified-cqrs/
    19. Greg Young : CQRS - https://www.youtube.com/watch?v=JHGkaShoyNs
    20. Bertrand Meyer – CQS - http://en.wikipedia.org/wiki/Bertrand_Meyer
    21. CQS : http://en.wikipedia.org/wiki/Command–query_separation
    22. CAP Theorem : http://en.wikipedia.org/wiki/CAP_theorem
    23. CAP Theorem : http://www.julianbrowne.com/article/viewer/brewers-cap-theorem
    24. CAP 12 years how the rules have changed
    25. EBay Scalability Best Practices : http://www.infoq.com/articles/ebay-scalability-best-practices
    26. Pat Helland (Amazon) : Life beyond distributed transactions
    27. Stanford University: Rx https://www.youtube.com/watch?v=y9xudo3C1Cw
    28. Princeton University: SAGAS (1987) Hector Garcia Molina / Kenneth Salem
    29. Rx Observable : https://dzone.com/articles/using-rx-java-observable

    View full-size slide