Revolutionizing the Fan Experience with Search at Ticketmaster

Elastic{ON} Tour LA - November 2, 2017

Helping fans find the perfect seat is a top priority for Ticketmaster. It's also a considerable challenge for their data system. In this presentation, Ticketmaster will demonstrate how they use search technology to solve for the difficult problem of helping users find seats, a volatile commodity as inventory changes. They will also highlight other areas, including marketing and abuse prevention, where they are leveraging search technology to help fans get tickets to the events they want to see.

John Carnahan | CDO, EVP of Data Science |Ticketmaster

Transcript

1. ElasticON Tour LA 2017 Ticketmaster Stories John Carnahan
2. 2 TICKETMASTER + ELASTIC Piecewise Modernization of 40 Year Data Company Overview Ticketmaster Data Story Many places where Elasticsearch fits our overall data strategy • Client Example • Fan Example
3. 3 TICKETMASTER’S STORY
4. 4 TICKETMASTER’S STORY
5. 5 TICKETMASTER’S STORY
6. TICKETMASTER’S STORY 1976 2000 1985 MARKET INTRODUCTION GROWTH
7. 7 TICKETMASTER’S STORY EDB1.TMOL.LAX0.WEBSYS.TMCS US Production Tools Target (Origin for all “B” data) EVENTDB sectionworks offering branding IVR tfopen venuemapdb wps release ORD0 Artemis Master edb1.tix.ord0.websys.tmcs que1.tmol.phx1.websys.tmcs que2.tmol.phx1.websys.tmcs que3.tmol.phx1.websys.tmcs que4.tmol.phx1.websys.tmcs PHX1 QUE ORD0 artemis art1.tix.ord0 local artemis db cleanup_num_days: 60 B’s A’s A’s edb1.tmol.phx1.websys.tmcs edb2.tmol.phx1.websys.tmcs edb3.tmol.phx1.websys.tmcs edb4.tmol.phx1.websys.tmcs edb5.tmol.phx1.websys.tmcs edb6.tmol.phx1.websys.tmcs edb7.tmol.phx1.websys.tmcs edb8.tmol.phx1.websys.tmcs edb9.tmol.phx1.websys.tmcs edb10.tmol.phx1.websys.tmcs edb11.tmol.phx1.websys.tmcs edb12.tmol.phx1.websys.tmcs edb13.tmol.phx1.websys.tmcs edb14.tmol.phx1.websys.tmcs edb15.tmol.phx1.websys.tmcs TMOL PHX1 SLAVES edb1.tmol.phx2.websys.tmcs edb2.tmol.phx2.websys.tmcs edb3.tmol.phx2.websys.tmcs edb4.tmol.phx2.websys.tmcs edb5.tmol.phx2.websys.tmcs edb6.tmol.phx2.websys.tmcs edb7.tmol.phx2.websys.tmcs edb8.tmol.phx2.websys.tmcs edb9.tmol.phx2.websys.tmcs edb10.tmol.phx2.websys.tmcs edb11.tmol.phx2.websys.tmcs edb12.tmol.phx2.websys.tmcs edb13.tmol.phx2.websys.tmcs edb14.tmol.phx2.websys.tmcs edb15.tmol.phx2.websys.tmcs TMOL PHX2 SLAVES mdb1.tmol.phx1.websys.tmcs PHX1 CLUSTER MASTER mdb1.tmol.phx2.websys.tmcs PHX2 CLUSTER MASTER que1.tmol.phx2.websys.tmcs que2.tmol.phx2.websys.tmcs que3.tmol.phx2.websys.tmcs que4.tmol.phx2.websys.tmcs PHX2 QUE que1.tmol.ash1.websys.tmcs que2.tmol.ash1.websys.tmcs que3.tmol.ash1.websys.tmcs que4.tmol.ash1.websys.tmcs ASH1 QUE edb1.tmol.ash1.websys.tmcs edb2.tmol.ash1.websys.tmcs edb3.tmol.ash1.websys.tmcs edb4.tmol.ash1.websys.tmcs edb5.tmol.ash1.websys.tmcs edb6.tmol.ash1.websys.tmcs edb7.tmol.ash1.websys.tmcs edb8.tmol.ash1.websys.tmcs edb9.tmol.ash1.websys.tmcs edb10.tmol.ash1.websys.tmcs edb11.tmol.ash1.websys.tmcs edb12.tmol.ash1.websys.tmcs edb13.tmol.ash1.websys.tmcs edb14.tmol.ash1.websys.tmcs edb15.tmol.ash1.websys.tmcs TMOL ASH1 SLAVES edb1.tmol.ash2.websys.tmcs edb2.tmol.ash2.websys.tmcs edb3.tmol.ash2.websys.tmcs edb4.tmol.ash2.websys.tmcs edb5.tmol.ash2.websys.tmcs edb6.tmol.ash2.websys.tmcs edb7.tmol.ash2.websys.tmcs edb8.tmol.ash2.websys.tmcs edb9.tmol.ash2.websys.tmcs edb10.tmol.ash2.websys.tmcs edb11.tmol.ash2.websys.tmcs edb12.tmol.ash2.websys.tmcs edb13.tmol.ash2.websys.tmcs edb14.tmol.ash2.websys.tmcs edb15.tmol.ash2.websys.tmcs TMOL ASH2 SLAVES mdb1.tmol.ash1.websys.tmcs ASH1 CLUSTER MASTER mdb1.tmol.ash2.websys.tmcs ASH2 CLUSTER MASTER que1.tmol.ash2.websys.tmcs que2.tmol.ash2.websys.tmcs que3.tmol.ash2.websys.tmcs que4.tmol.ash2.websys.tmcs ASH2 QUE ord0 Standby edb2.tix.ord0 LAX0 Standby edb2.tix.lax0 Simetra Venue Feed back to hosts LAX0 Mgr box mgr1.dba.lax0.websys.tmcs LAX0 artemis art1.tix.lax0 local artemis db cleanup_num_days: 60 LAX0 Artemis Master edb1.tix.lax0.websys.tmcs B’s A’s ORD0 Artemis Master edb1.stix.ord0.websys.tmcs A’s ORD0 artemis art1.stix.ord0 local artemis db cleanup_num_days: 360 ord0 Standby edb2.stix.ord0 edb1.shared.phx1.websys.tmcs PHX1 CLUSTER MASTER edb1.shared.phx2.websys.tmcs PHX2 CLUSTER MASTER edb2.shared.phx1.websys.tmcs edb3.shared.phx1.websys.tmcs edb4.shared.phx1.websys.tmcs edb5.shared.phx1.websys.tmcs edb6.shared.phx1.websys.tmcs edb7.shared.phx1.websys.tmcs edb8.shared.phx1.websys.tmcs edb9.shared.phx1.websys.tmcs edb10.shared.phx1.websys.tmcs SHARED PHX1 SLAVES edb2.shared.phx2.websys.tmcs edb3.shared.phx2.websys.tmcs edb4.shared.phx2.websys.tmcs edb5.shared.phx2.websys.tmcs edb6.shared.phx2.websys.tmcs edb7.shared.phx2.websys.tmcs edb8.shared.phx2.websys.tmcs edb9.shared.phx2.websys.tmcs edb10.shared.phx2.websys.tmcs SHARED PHX1 SLAVES LAX0 Artemis Master edb1.stix.lax0.websys.tmcs A’s LAX0 artemis art1.stix.lax0 local artemis db cleanup_num_days: 360 lax0 Standby edb2.stix.lax0 edb1.shared.ash1.websys.tmcs ASH1 CLUSTER MASTER edb1.shared.ash2.websys.tmcs ASH2 CLUSTER MASTER edb2.shared.ash1.websys.tmcs edb3.shared.ash1.websys.tmcs edb4.shared.ash1.websys.tmcs edb5.shared.ash1.websys.tmcs edb6.shared.ash1.websys.tmcs edb7.shared.ash1.websys.tmcs edb8.shared.ash1.websys.tmcs edb9.shared.ash1.websys.tmcs edb10.shared.ash1.websys.tmcs SHARED ASH1 SLAVES edb2.shared.ash2.websys.tmcs edb3.shared.ash2.websys.tmcs edb4.shared.ash2.websys.tmcs edb5.shared.ash2.websys.tmcs edb6.shared.ash2.websys.tmcs edb7.shared.ash2.websys.tmcs edb8.shared.ash2.websys.tmcs edb9.shared.ash2.websys.tmcs edb10.shared.ash2.websys.tmcs SHARED ASH2 SLAVES B’s B’s edb1.ivr.lax0.websys.tmcs IVR TOOLS MASTER edb1.ivr.ord0.websys.tmcs edb2.ivr.ord0.websys.tmcs edb3.ivr.ord0.websys.tmcs edb4.ivr.ord0.websys.tmcs IVR read only edb3.ntf.cft1.coreys.tmcs MySQL slave CRM UK DATA AD HOC UK DATA Replicated from the UK edb1.ntf.cft1.coresys.tmcs edb2.ntf.cft1.coresys.tmcs CRM NTF EDBs edb1.dba.ash1.coresys.tmcs PROMOTER POSTINGS TOOL cluster master edb2.dba.ash1.coresys.tmcs edb3.dba.ash1.coresys.tmcs edb4.dba.ash1.coresys.tmcs PROMOTER POSTINGS TOOL read only edb2.ivr.lax0.websys.tmcs edb3.ivr.lax0.websys.tmcs edb4.ivr.lax0.websys.tmcs edb5.ivr.lax0.websys.tmcs IVR read only edb1.dba.phx1.coresys.tmcs PROMOTER POSTINGS TOOL cluster master edb2.dba.phx1.coresys.tmcs edb3.dba.phx1.coresys.tmcs edb4.dba.phx1.coresys.tmcs PROMOTER POSTINGS TOOL read only Hosts ATL AGF ARZ CH1 AU2 AUS CH5 CH2 CH3 CH8 CH6 CH7 FL2 DAL DET LA2 FLO LA1 NCA MTN MXC NV2 NEV NTL NY2 NV3 NY1 NZL NY4 NY5 SCS PAN PHI T44 SEA T43 VAN TOR TWB WDC WES 1992: The move to relational data Event Meta Data moves to an RDBMS - Name - Date - Description
8. TICKETMASTER’S STORY 1976 2017 2010 2000 1985 MARKET INTRODUCTION GROWTH MATURITY DATA
9. 9 TICKETMASTER’S STORY 40 Year End Result What fundamental changes can we make that won’t break anything?
10. 10 TICKETMASTER CLIENTS How we can rock their world without really trying Modernization of Reporting Tools With Elasticsearch
11. 11 TICKETMASTER CLIENTS Pull Data in Real Time Reporting Archtics Archtics Reporting Pull Data in Real Time The Simple Solution
12. 12 TICKETMASTER CLIENTS
13. 13 TICKETMASTER CLIENTS Archtics
14. TICKETMASTER CLIENTS The Simple Solution Sucks Reporting Real-time Can’t compare events Dangerous to operational data store Brittle Connections
15. 15 TICKETMASTER CLIENTS Archtics ETL Ops Nightly Extract Nomalized Data DW Eng Data Eng De-Normalized Data Agg Data Reporting App Eng
16. 16 TICKETMASTER CLIENTS Can compare events 24-48 hour delay Brittle and Complex Aggregates Only Process and People Heavy
17. 17 TICKETMASTER CLIENTS Trxn Logs Reporting Archtics Norm Data De-Norm Data Trxn Logs Archtics Tap directly into data duplication Real-time Data for all events Idempotency for all Fine-grained data
18. TICKETMASTER CLIENTS Real-time Doesn’t touch the operational data store
19. TICKETMASTER CLIENTS Seat-level data Simple and Idemptent
20. TICKETMASTER CLIENTS Can compare events, in real-time, without affecting legacy databases
21. 21 TICKETMASTER FANS How we can rock their world without really trying VS Example of how everything is sold Powered by
22. TICKETMASTER FANS Long list of dates organized by date.
23. TICKETMASTER FANS No good seats left … try the next date
24. TICKETMASTER FANS Fan frustration as the time to find seats grows as more seats are sold. Slowing sales and panicking fans # SOLD Event Date Top of List Bottom of List
25. TICKETMASTER FANS And almost too much for single Host
26. TICKETMASTER FANS High Expectations following Hamilton – 10M unique visitors in the first minute looking for tickets
27. TICKETMASTER FANS Host Make content AND availability something to search and analyze
28. 28 TICKETMASTER FANS Host 15 Kiloseats per second per Host Real-time Data for all events Smart Collapse Better Indexing Reduce Noise of frequent changes Host R3 4XL to I3 Double Performance 6X Storqge Half Cost Crazy throughput Reduction
29. TICKETMASTER FANS Custom prototype interface that includes content and availability in an intuitive way Event Date Top of List Bottom of List
30. TICKETMASTER CONCLUSIONS (ELASTIC) 1. Pushing data from operational systems has to be core to every data strategy. 2. You can be flexible about where the data land as long as it is in a stream first 3. Turn things into a search problem - not a panacea for every problem but much to be gained if you can 4. Surprisingly indexing rate into Elastic was not the biggest worry 5. It can take some effort but adding plugins can make a huge difference (i.e. seat adjacency and price) 6. Elastic is not great for pulling data to the browser quickly en masse – again better for search 7. Watch compression settings closely – not obvious that compression would help at first but made a huge difference because data between nodes was large
31. TICKETMASTER Thanks Corey Porter David Nhim Brett Hale Chris Smith Gil Adam