Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CosmosDB: Jack Of All Trades, Master Of Many

Daron Yondem
November 04, 2017

CosmosDB: Jack Of All Trades, Master Of Many

This is the slide deck I used during SQL Saturday event in Istanbul.
Last updated on 12/21/2017

Daron Yondem

November 04, 2017
Tweet

More Decks by Daron Yondem

Other Decks in Technology

Transcript

  1. CosmosDB
    Jack of All Trades, Master of Many
    Daron Yöndem
    http://daron.me | @daronyondem

    View Slide

  2. A little history
    2010 – Project Florence
    2015 – DocumentDB
    2017 – CosmosDB

    View Slide

  3. News Flash! “It is just JSON”
    https://www.documentdb.com/sql/demo

    View Slide

  4. 1-Click Global Replication
    • Ring 0 Service
    • Multi-Homing
    • Priorities for
    regions
    • Manual or
    automatic failover

    View Slide

  5. 99.99% SLA for Low latency reads + writes
    • Reads and writes served from local region
    • Guaranteed millisecond latency worldwide
    • Write optimized, latch-free database engine
    • Automatically indexed SSD storage
    Reads (1KB) Indexed writes (1KB)
    Read < 2 ms
    Writes < 6 ms
    Read < 10 ms
    Writes < 15 ms
    99%
    50%
    • Synchronous and automatic
    indexing at sustained
    ingestion rates
    • No schema or index
    management needed
    • No schema versioning needed
    • No schema migration needed

    View Slide

  6. Provisioned
    Throughput

    View Slide

  7. What is RU?
    • Request Unit
    • Not all requests are equal.
    • A normalized quantity of request unit based on
    the amount of computation (CPU, memory, and
    IOPS) required to serve the request.

    View Slide

  8. How to calculate?
    Item Size Reads/second Writes/second Request units
    1 KB 500 100 (500 * 1) + (100 * 5) = 1,000 RU/s
    1 KB 500 500 (500 * 1) + (500 * 5) = 3,000 RU/s
    4 KB 500 100 (500 * 1.3) + (100 * 7) = 1,350 RU/s
    4 KB 500 500 (500 * 1.3) + (500 * 7) = 4,150 RU/s
    64 KB 500 100 (500 * 10) + (100 * 48) = 9,800 RU/s
    64 KB 500 500 (500 * 10) + (500 * 48) = 29,000 RU/s
    See: https://www.documentdb.com/capacityplanner

    View Slide

  9. For example
    • SELECT * FROM c
    • (2.87 RU)
    • SELECT * FROM c where
    Contains (c.Name, "Sample")
    • (2.45 RU)

    View Slide

  10. DEMO
    Calculating RU On-The-Fly

    View Slide

  11. public static async Task> GetItemsAsync(Expression>
    predicate)
    {
    double queryCost = 0;
    IDocumentQuery query = client.CreateDocumentQuery(
    UriFactory.CreateDocumentCollectionUri(DatabaseId, CollectionId),
    new FeedOptions { MaxItemCount = -1 })
    .Where(predicate)
    .AsDocumentQuery();
    List results = new List();
    while (query.HasMoreResults)
    {
    var response = await query.ExecuteNextAsync();
    queryCost += response.RequestCharge;
    results.AddRange(response);
    }
    Debug.WriteLine(queryCost.ToString());
    return results;
    }

    View Slide

  12. Request Unit Management
    Single Partition
    Container
    Partitioned Container
    Minimum Throughput 400 RU/sec 1.000 RU/sec
    Maximum Throughput 10.000 RU/sec Unlimited
    Offer offer = client.CreateOfferQuery()
    .Where(r => r.ResourceLink == collection.SelfLink)
    .AsEnumerable().SingleOrDefault();
    offer = new OfferV2(offer, 12000);
    client.ReplaceOfferAsync(offer);
    A partition key is
    required to scale your
    collection's throughput
    beyond 2,500 request
    units in the future

    View Slide

  13. Affecting RUs are;
    • Item size
    • Item property count (Indexing)
    • Data consistency (Strong or Bounded Staleness)
    • Indexed properties (lazy indexing can help)
    • Document indexing (Disable if you don’t need)
    • Query patterns (predicates, UDFs, data source size)
    • Script usage (SPs, triggers)

    View Slide

  14. View Slide

  15. What if you exceed?
    HTTP Status 429 Status Line:
    RequestRateTooLarge x-ms-retry-after-ms :100

    View Slide

  16. Partitioning
    Partition schema is
    immutable

    View Slide

  17. Reading data with partitions.

    View Slide

  18. Reading data with partitions.

    View Slide

  19. How to detect
    Hot Partitions?

    View Slide

  20. View Slide

  21. View Slide

  22. Choose Your Consistency Level
    01
    Strong
    Bounded
    Staleness
    Session
    Consistent
    Prefix
    Eventual
    Clear Tradeoffs
    • Latency
    • Availability
    • Throughput
    Lower latency, higher availability, better read scalability.

    View Slide

  23. Bounded Staleness
    01
    Strong
    Bounded
    Staleness
    Session
    Consistent
    Prefix
    Eventual
    When choosing bounded
    staleness, the "staleness" can
    be configured in two ways:
    number of versions K of the
    item by which the reads lag
    behind the writes, and the
    time interval t
    Lower latency, higher availability, better read scalability.

    View Slide

  24. Consistent Prefix
    01
    Strong
    Bounded
    Staleness
    Session
    Consistent
    Prefix
    Eventual
    Consistent prefix guarantees
    that reads never see out of
    order writes. If writes were
    performed in the order A, B,
    C, then a client sees either A,
    A,B, or A,B,C, but never out of
    order like A,C or B,A,C.
    Lower latency, higher availability, better read scalability.

    View Slide

  25. Multi-Model API

    View Slide

  26. Native Support for Multiple Data Models
    • Database engine operates on atom-record-sequence (ARS) based type system
    • All data models are efficiently translated to ARS
    • API and wire protocols are supported via extensible modules
    • Instance of a given data model can be materialized as trees
    • Graph, documents, key-value, column-family, … more to come
    KEY-VALUE COLUMN-FAMILY DOCUMENT GRAPH

    View Slide

  27. View Slide

  28. Auto Indexing
    • IndexingMode
    • Consistent (Collection Consistency
    applies)
    • Lazy (ingest now, query later)
    • None (EnableScanInQuery)
    • DataTypes
    • String, Number, Point,
    Polygon
    • Index Types
    • Hash (joins)
    • Range (<, >)
    • Spatial
    • Precision can be defined.

    View Slide

  29. 4 Axis SLA
    Latency @ 99th
    percentile SLA
    Throughput SLA
    Consistency SLA
    Availability SLA
    2
    4
    3
    1
    Cosmos DB:
    99.99% HA within a single region
    99.999% across regions
    99.99 SLA throughput, latency,
    consistency all at the 99th
    percentile

    View Slide

  30. Security
    • Documents and backups are encrypted at rest
    • IP-based access controls
    • Role-based access controls
    • Automated online backups
    • Attack monitoring
    • Geo-fencing

    View Slide

  31. Disclaimer
    • Cosmos DB is not a SQL Database, no complex
    table joins. (you are doing it wrong)
    • Other NoSQL databases are good at doing one or
    two things really well but not native to Cloud.

    View Slide

  32. Thanks!
    Slides: http://daron.me/decks
    Daron Yöndem
    http://daron.me | @daronyondem

    View Slide