Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Coherence user meet up (Moscow, 14 Mar 2013) - Gene Gleyzer - Coherence Overview

aragozin
March 14, 2013

Coherence user meet up (Moscow, 14 Mar 2013) - Gene Gleyzer - Coherence Overview

aragozin

March 14, 2013
Tweet

More Decks by aragozin

Other Decks in Programming

Transcript

  1. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    <Insert Picture Here> Oracle Coherence Clustering Overview March 2013 Gene Gleyzer Architect, Oracle Coherence
  2. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
  3. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    “A Data Grid is a system composed of multiple servers that work together to manage information and related operations - such as computations - in a distributed environment.” What is Oracle Coherence?
  4. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    <Insert Picture Here> Coherence Clustering Concepts
  5. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Coherence Clustering: Tangosol Clustered Messaging Protocol • Completely asynchronous yet ordered messaging built on various transports • Truly Peer-to-Peer: equal responsibility for both producing and consuming the services of the cluster • Self Healing - Quorum based diagnostics • Linearly scalable mesh architecture. • TCP-like features • Messaging throughput scales to the network infrastructure
  6. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Coherence Clustering: Service Hierarchy • One ClusterService (per ClassLoader) • Any number of uniquely named Services (e.g. caching service) • For each caching service there are any number of uniquely named Caches
  7. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Coherence Clustering: The Cluster Service • Clustering – Discovery & Member addressing – Message delivery & transport – Death Detection – Membership events – Cluster health • Witness protocol • Tombstones • Quorum (split brain prevention)
  8. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Coherence Clustering: Death Detection • Purpose – Maintain cluster HA/stability • Description – Detect member failures using one of several methods: • TCP Socket monitoring • IP Address monitoring • Datagram delivery SLAs • Adaptive Heartbeats – Report detected member death to ClusterService for handling
  9. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Coherence Clustering: Messaging • Purpose – Communications between service members within the cluster • Description – Reliable, in-order message delivery within a channel – Multiple transports: • UDP (unicast/multicast) • TCP (SSL) • “Exabus” (TCP/SDP/RDMA)
  10. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Exabus – Transport Optimizations Direct Memory I/O for Java • New Java APIs and Exalogic Elastic Cloud Software - Low Latency Java support for Infiniband - Optimized implementation for Exalogic Infiniband • Surfacing low-level advanced networking capabilities
  11. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Exabus - MessageBus • Binary low-level message transport – Multi-point addressing – Reliable ordered delivery – Asynchronous event based programming model • Pluggable provider based framework – SocketBus (TCP/SDP) – Native RDMA
  12. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    <Insert Picture Here> Data Management Options
  13. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Partitioning • Purpose – Partitioned data storage and request processing • Description – Distributed key-value store with backup/failover – Partitioned requests processed and backed up atomically – Support for pluggable data affinity – Highly customizable behaviors (“Roll your own API”) • Read-through/write-through/write-behind to DB/store • Triggers, listeners, events (client and server-side)
  14. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Partitioned Cache: Data Access • Data spread and backed up across Servers • Data location transparent (but observable) to developers • Members have access to all Data • All Data locations are known – no lookup & no registry • Access cost stays constant regardless of cluster size
  15. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Partitioned Cache: Data Update • Synchronous and asynchronous updates • Avoid Data Loss & Corruption • Predictable Performance • Backups and Primaries on different machines for resilience • Partitioning strategy is automatically and dynamically managed
  16. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Partitioned Cache: Local Storage • Some members are used to manage data • Other members are temporary in a cluster, or do not have memory to spare for managing data • They should not cause repartitioning • Specialization of roles within a Data Grid: Clients and Servers
  17. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Partitioning • Traditional/Basic Usage – Key-based data retrieval/update – Predicate-based queries – Data-driven Events • Advanced Usage – Server-side programming – Partition-based transactions – “Patterns” (e.g. Messaging, Processing, Push-replication, etc) – Live Objects
  18. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    <Insert Picture Here> Data Processing Options
  19. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Data Processing: Events - JavaBean Event Model • Listen to all events for all keys – ENTRY_DELETED – ENTRY_INSERTED – ENTRY_UPDATED • Listen to changes to a specific key • Listen to a changes to data that match a specific criteria (i.e. Filter)
  20. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Data Processing: Parallel Query • Programmatic query mechanism • Queries performed in parallel across the grid • Standard indexes provided out-of-the-box and supports implementing your own custom indexes • Cost-based analysis of Filter application • Standard Filters provided out-of-the-box (e.g. OR, AND, ALL, EQUALS, etc.)
  21. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Data Processing: Continuous Query Cache • Automatically, transparently and dynamically maintains a view locally based on a specific criteria (i.e. Filter) • Same API as all other Coherence caches • Support local listeners. • Supports layered views
  22. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Data Processing: Invocable Map • The inverse of caching • Sends the processing (e.g. EntryProcessors) to where the data is in the grid • Standard EntryProcessors provided Out-of-the-box • Once and only once guarantees • Processing is automatically fault-tolerant and idempotent • Processing can be: • Targeted to a specific key • Targeted to a collection of keys • Targeted to entries that match a specific criteria • Targeted to a partition
  23. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    Data Processing: Triggers • Inject pre-processing logic to data being added to a cache. • Similar to EntryProcessors, but fired before a mutation takes place. • They allow your “process” method to override, replace, decorate, remove or fail a cache mutation. • Adds veto ability to data insertion. • Common Uses: • Prevent invalid transactions; • Enforce complex security authorizations; • Enforce complex business rules; • Gather statistics on data modifications;
  24. Copyright © 2013, Oracle and/or its affiliates. All rights reserved.

    12.1.2 Committed Features • Exabus Hardening • ECID Integration • Multiple-backup hardening • Coherence Container • Federated Caches • Coherence*Extend Improvements – Container support – Thread-pooling improvements – REST enhancements • Unified Event Model