How Warner Bros. is Using Elastic to Solve Entertainment and Media Problems at Scale

Warner Bros. processes billions of records each day globally between its web assets, digital content distribution, OTT streaming services, online and mobile games, technical operations, anti-piracy programs, social media, and retail point of sale transactions. Despite having large MPP clusters, a significant amount of dark data remained trapped in Web Logs. In this presentation, we will discuss how Warner Bros. and Decision Lab leveraged the new Elastic Stack 5.x coupled with Apache Spark, to deliver scalable insights and new capabilities to support business needs.

Drew Malone l Senior DevOps Expert l Decision Lab
Nathan Necaise l CEO and Co-Founder l Decision Lab
Brian Kursar l Vice President Data Intelligence - Strategy and Architecture l Warner Brothers

Transcript

1. Warner Bros. Entertainment March 8, 2017 @briankursar How Warner Bros. is Using Elastic to Solve Entertainment and Media Problems at Scale Brian Kursar, VP – Data Strategy and Architecture Nathan Necaise - CEO and Co-Founder, Decision Lab
2. MARCH 8, 2017 WB DATA @ELASTICON Brian Kursar – VP Data Intelligence @briankursar
3. WARNER BROS. Games Theatrical Television Home Entertainment Consumer Products
4. STUDIO DATA Pre- Production Sales Supply Chain Production Marketing Physical Ratings Consumer Exhibitor Social Digital Linear Operations Behavior Inventory Spend Competitors Content Protection Over the Top Studio Tours Theatrical Consumer Products Television Games Home Entertainment
5. MARKETING CHALLENGE Billions of records need to be processed multiple times each day to support Marketing Audience queries. Existing architecture using MapReduce was taking 24-48 hours to process highly complex metadata joins and text based queries on Third Party Platform just to receive Audience total unique counts.
6. USE CASE WB Campaign Management Operations Team needed a solution that would be able to execute these same queries over Multi- Billion record datasets multiple times a day and reduce the query time from days to seconds.
7. DATA SOURCES DATABASE STORAGE Elasticsearch Kibana X-pack WEB APP SERVER AUTHENTICATION NOTIFICATION X-pack LDAP AD SSO Instances (X) Master Nodes (3) Ingest Nodes (X) Data Nodes - Hot (X) Data Nodes - Warm (X) AMAZON S3 AMAZON REDSHIFT HDFS MESSAGING QUEUE HADOOP ECOSYSTEM ES-Hadoop SALT MASTER BASTION HOST External DMP ARCHITECTURE USERS ADMIN
8. RESULTS Converting from Multi-Day Batch jobs to Near Real-Time • Lightning Fast Cardinality Counts across Billions of Records in less than 10 seconds • Campaign Ops Team can now build and syndicate Audiences in near-real time. Tech Tip • Team leveraged murmur3 hash and saw approx. 50% performance improvement
9. TEXT ‘WARNERBROS’ to 67076 We’re Hiring WarnerBrosCareers.com Brian Kursar – VP Data Intelligence @briankursar