How Warner Bros. is Using Elastic to Solve Entertainment and Media Problems at Scale

Warner Bros. processes billions of records each day globally between its web assets, digital content distribution, OTT streaming services, online and mobile games, technical operations, anti-piracy programs, social media, and retail point of sale transactions. Despite having large MPP clusters, a significant amount of dark data remained trapped in Web Logs. In this presentation, we will discuss how Warner Bros. and Decision Lab leveraged the new Elastic Stack 5.x coupled with Apache Spark, to deliver scalable insights and new capabilities to support business needs.

Drew Malone l Senior DevOps Expert l Decision Lab
Nathan Necaise l CEO and Co-Founder l Decision Lab
Brian Kursar l Vice President Data Intelligence - Strategy and Architecture l Warner Brothers

Transcript

1. Warner Bros. Entertainment
   March 8, 2017
   @briankursar
   How Warner Bros. is Using Elastic to
   Solve Entertainment and Media
   Problems at Scale
   Brian Kursar, VP – Data Strategy and Architecture
   Nathan Necaise - CEO and Co-Founder, Decision Lab
   

   View Slide

2. MARCH 8, 2017
   WB DATA
   @ELASTICON
   Brian Kursar – VP Data Intelligence
   @briankursar
   

   View Slide

3. WARNER BROS.
   Games
   Theatrical Television Home
   Entertainment
   Consumer
   Products
   

   View Slide

4. STUDIO DATA
   Pre-
   Production
   Sales
   Supply
   Chain
   Production
   Marketing
   Physical
   Ratings
   Consumer
   Exhibitor
   Social
   Digital
   Linear
   Operations
   Behavior
   Inventory
   Spend
   Competitors
   Content
   Protection
   Over the
   Top
   Studio Tours
   Theatrical
   Consumer
   Products
   Television
   Games
   Home
   Entertainment
   

   View Slide

5. MARKETING CHALLENGE
   Billions of records need to be processed multiple times each day
   to support Marketing Audience queries.
   Existing architecture using MapReduce was taking 24-48 hours to
   process highly complex metadata joins and text based queries on
   Third Party Platform just to receive Audience total unique counts.
   

   View Slide

6. USE CASE
   WB Campaign Management Operations Team needed a solution
   that would be able to execute these same queries over Multi-
   Billion record datasets multiple times a day and reduce the query
   time from days to seconds.
   

   View Slide

7. DATA SOURCES
   DATABASE STORAGE
   Elasticsearch
   Kibana
   X-pack
   WEB APP SERVER
   AUTHENTICATION NOTIFICATION
   X-pack
   LDAP AD SSO
   Instances (X)
   Master Nodes (3)
   Ingest Nodes (X)
   Data Nodes - Hot (X)
   Data Nodes - Warm (X)
   AMAZON S3
   AMAZON REDSHIFT
   HDFS
   MESSAGING QUEUE
   HADOOP ECOSYSTEM
   ES-Hadoop
   SALT MASTER
   BASTION HOST
   External DMP
   ARCHITECTURE
   USERS
   ADMIN
   

   View Slide

8. RESULTS
   Converting from Multi-Day
   Batch jobs to Near Real-Time
   • Lightning Fast Cardinality
   Counts across Billions of
   Records in less than 10
   seconds
   • Campaign Ops Team can
   now build and syndicate
   Audiences in near-real time.
   Tech Tip
   • Team leveraged murmur3
   hash and saw approx. 50%
   performance improvement
   

   View Slide

9. TEXT ‘WARNERBROS’ to 67076
   We’re Hiring
   WarnerBrosCareers.com
   Brian Kursar – VP Data Intelligence
   @briankursar
   

   View Slide