Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Elastic Cloud - Architecture deep dive

Elastic Cloud - Architecture deep dive

Igor Kupczynski, an Elastic Cloud Engineer, takes you into the Cloud architecture and cover a high level talk on the inner workings of Elastic Cloud, our hosted version of the Elastic Stack.

This talk was presented on the Elastic NL meetup http://www.meetup.com/Elastic-NL/events/230223068/

Dd9d954997353b37b4c2684f478192d3?s=128

Elastic Co

May 24, 2016
Tweet

More Decks by Elastic Co

Other Decks in Technology

Transcript

  1. 1 Elastic Cloud Architecture Morten Ingebrigtsen, Igor Kupczyński

  2. 2 Agenda Expertise Architecture / Index / Shard Design Cluster

    Management (Tuning) Query Performance Optimization Dev to Production Migration & Upgrades Best Practices (Elastic Stack, X-Pack) • Introduction & demo • Dive into architecture • Use case: recommendations with Graph • Q&A
  3. 3 Introduction

  4. 4 Demo

  5. 5 Architecture

  6. 6 Architecture — key features • Services • AWS infrastructure

    • Regions and availability zones • EC2 — virtual servers (i2) • Elastic Load Balancer — front proxy / load balancer • S3 — backup and binary storage • RDS — PostgreSQL • Docker all-the-things • Zookeeper — state of the system + coordination
  7. 7 Lets talk numbers • 7 AWS regions • Thousands

    of active elasticsearch clusters • Billions of elasticsearch requests per day • 15 team members • both SREs and devs, with some overlap in responsibilities • it started with 4
  8. 8 High availability 2TKOCT[PQFG 4GRNKECPQFG 6KGDTGCMGT #XCKNCDKNKV[ \QPG \QPG \QPG

    #OC\QPņU 'NCUVKE.QCF $CNCPEGT 2TQZ[ 2TQZ[ 2TQZ[ )'6AUGCTEJ
  9. 9 No-downtime scaling & upgrades 0QFG# XGT 2TQZ[ 2TQZ[ 2TQZ[

      0QFG$ XGT   4GRNKECVKPIFCVC
  10. 10 Architecture <QQMGGRGT 2QUVITG 53. 2TQZ[ %QPUVTWEVQT 1VJGTUGTXKEGU #FOKP %QPUQNG

    7UGT%QPUQNG #NNQECVQT '5 0QFG
  11. 11 Zookeeper <QQMGGRGT • Distributed data store for coordination and

    storing the global state • Filesystem tree-like structure — znodes • CAP — consistent in the event of network partitions • Fast reads, but only simple queries are possible • Writes need ACK from a quorum, so slower; data should fit in memory • Writes are linear — all clients see them in the same order • Possible to set watches • Elastic Cloud — zookeeper stores the state of the whole system
  12. 12 Use Zookeeper. It’s mature, well-designed, and battle-tested. Because the

    consequences of its connection model and linearizability properties are subtle, you should, wherever possible, take advantage of tested recipes and client libraries like Curator, which do their best to correctly handle the complex state transitions associated with session and connection loss. https://aphyr.com/posts/291-jepsen-zookeeper Kyle Kingsbury a.k.a. Aphyr
  13. 13 Allocator & elasticsearch nodes • Manages elasticsearch nodes •

    All allocators in a region form an allocator pool • Advertises its capacity and free resources in zookeeper • Each node is a docker container • Guarantees memory, CPU, IOPS, disk space • Isolates the nodes #NNQECVQT '5 0QFG
  14. 14 Constructor • The brains behind allocator • Calculates what

    needs to be changed when a cluster is added or reconfigured • Monitors new requests from user & admin console • Writes to a znode • Allocator watches it and acts • Assigns new clusters to a proper allocator %QPUVTWEVQT
  15. 15 Security • Stunnels between containers • znode ACLs (each

    container can access only part of the state important for it) • Dockerized elasticsearch nodes • Oauth & Time-based One Time Passwords in the admin console • Shield (Elastic Security) for all the clusters • Dedicated service to manage secrets in the cloud • Backups every 30 mins, retention 48 hours • Isolated S3 bucket
  16. 16 Use case: Graph

  17. 17 Q&A Go to https://cloud.elastic.co and start your trial today!