Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Roblox Elasticsearch Meetup

Dd9d954997353b37b4c2684f478192d3?s=47 Elastic Co
May 05, 2016
210

Roblox Elasticsearch Meetup

Slides from Guru's presentation at the Elastic Silicon Valley meetup on 5/5/2016.

Dd9d954997353b37b4c2684f478192d3?s=128

Elastic Co

May 05, 2016
Tweet

More Decks by Elastic Co

Transcript

  1. May 6, 2016 Elasticsearch Meetup Guru

  2. What is ROBLOX ROBLOX is the Imagination Platform™ Every month

    over 12 Million users and 30 Million visitors come to ROBLOX to play, adventure, imagine and create with their friends. • #1 gaming platform for Kids and Teens • Primarily user generated content • 360k concurrent players at peak times • 800K+ requests/sec
  3. What is ROBLOX

  4. How ROBLOX uses Elasticsearch Group Search Message Search Games Search

    ROBLOX Imagination Platform Catalog Search Leaderboards User Search Virtual Sales
  5. What’s Under the Hood

  6. What’s Under the Hood

  7. • The ROBLOX Catalog contains items uploaded by the community

    and by us • Makes use of boosting, filtering, custom scripts, tokenizers, and analyzers • eCPM sort • 350 Million documents (11 shards) • Seven nodes User Generated Catalog
  8. How ROBLOX uses Elasticsearch Group Search Message Search Games Search

    ROBLOX Imagination Platform Catalog Search Leaderboards User Search Virtual Sales
  9. • Game developers can award as many points as they

    want to the players • Each game gets a leaderboard but all the per-game leaderboard documents go into the same index • Aggregate leaderboards viewable by day, week, month and all time Leaderboards
  10. • Used Amazon DynamoDB to count the time-bucketed points per

    game • 200 Million documents (six shards) across six nodes • Watch out for the default query generated by your client Leaderboards (continued) Valid query 1: { "filter": { "term": { “gameId": "8888" } } } Valid query 2: { "query": { "filtered": { "filter": { "term": { " gameId": "8888" } } } } } took : 774 ms took : 53 ms
  11. How ROBLOX uses Elasticsearch Group Search Message Search Games Search

    ROBLOX Imagination Platform Catalog Search Leaderboards User Search Virtual Sales
  12. • 124 Million documents, 9 shards, 3 nodes • Partial

    word matches User Search
  13. How ROBLOX uses Elasticsearch Group Search Message Search Games Search

    ROBLOX Imagination Platform Catalog Search Leaderboards User Search Virtual Sales
  14. Query 1: { "and": { "filters": [ { "range": {

    "id": { "gt": "10" } } }, { "range": { "id": { "lt": "110" } } } ] } } • Gives the ability for our Customer Service team to respond to purchase anomalies and also ensure security • 800 Million documents (11 shards) across three nodes • 4 Million documents being added per day • Range query performance Users + Catalog = Virtual Sales Query 2: { "range": { "id": { "gt": "10", "lt": "110" } } } took: 5543 ms took:15 ms
  15. • Migrated our servers along with elasticsearch clusters from Toronto

    to Chicago • Considered approaches like cross data-center replication of indices • Scheduled snapshots copied over by automated scripts • Snapshots restored in the new data center • Once the rest of the site migrated over, the data started flowing into the new indices Data Center Migration
  16. ROBLOX + Elasticsearch Keeps the Imagination Platform Running Thank You