Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Krossover & MongoDB

Krossover & MongoDB

Slides for the NY MongoDB User Group Meetup on July 24th, 2012

Carlo DiCelico

July 24, 2012
Tweet

Other Decks in Technology

Transcript

  1. >what we do, in a nutshell Krossover www.krossover.com twitter.com/krossovr We

    index (and generate stats from) game film uploaded by coaches.
  2. upload Coaches upload their raw video footage, including game and

    practice film, to our servers using our custom uploader. Transcode & Index we then transcode that video and tag every event in the film, grouping them into plays. This is done with our itag app. Deliver Once the video is transcoded and indexed, it is available to the coaches and players on our site and mobile app. we stream the video from our servers to client’s html5 video player via our intelligence app. 1 2 3 >What we do, in slightly more detail Krossover www.krossover.com twitter.com/krossovr
  3. >what we do, by the numbers Krossover www.krossover.com twitter.com/krossovr 16,716

    videos uploaded 29tb raw data uploaded 5.8tb compressed data stored 5,357 User accounts 6.3 million user actions in the past year 13.5 million searchable data points indexed in the past 2 basketball seasons 600k searchable data points indexed in the past 2 lacrosse seasons
  4. >where does mongo fit in? Krossover www.krossover.com twitter.com/krossovr what we

    provide indexing search analytics driven by mongo indexing search analytics
  5. >where does mongo fit in? indexing Krossover www.krossover.com twitter.com/krossovr An

    indexed play list of events indexed in current play Choice of next event When indexing a game, the possible next events, keyboard shortcuts, and event metadata (name, type, etc.) are all being pulled from mongo.
  6. >where does mongo fit in? Search Krossover www.krossover.com twitter.com/krossovr list

    of plays indexed in current game Choice of search filters that can be applied our search paradigm is filter, rather than keyword, based. A user applies one or more filters and plays are filtered out that don’t match the criteria provided.
  7. >where does mongo fit in? Search Krossover www.krossover.com twitter.com/krossovr list

    of plays after applying 3 search filters everything users interact with in search - the filters, the results, and the associated metadata, is stored in mongo.
  8. >where does mongo fit in? analytics Krossover www.krossover.com twitter.com/krossovr all

    of the data needed to render shot charts and drive charts for various sports is stored in mongo.
  9. >where does mongo fit in? analytics Krossover www.krossover.com twitter.com/krossovr Stats

    are generated in the background and stored in mongo. We then pull them out into our view.
  10. >how it works: High-level app architecture Krossover www.krossover.com twitter.com/krossovr itag

    app intelligence app json-rpc api vit widget vit (video intelligence tool) consumes the json-rpc api and detects whether it is in coach mode or indexing mode. In indexing mode, it loads the data needed for indexing. In coach mode, it loads plays. 1 intelligence app loads VIT on game overview and search pages in coach mode and publishes the json-rpc api. 2 itag app pulls vit in from the intelligence app and loads it in indexer mode. 3 json-rpc api allows the intelligence app, the itag app, and vit to communicate and access data based on context. 4
  11. >how it works: it’s all about event flow Krossover www.krossover.com

    twitter.com/krossovr event data point next event data point event play a game is made up of many plays, which are strings of events. Each event consists of multiple data points. every event has certain next events, which are events that can logically follow it in gameplay. time
  12. >how it works: event flow and mongo Krossover www.krossover.com twitter.com/krossovr

    events lie at the heart of krossover’s apps. the commonalities between event flow definitions across sports allow us to re- use code, while the differences demand flexibility. This requirement for a fluid yet similar event flow definition from sport to sport makes it a perfect candidate for being stored as a document in Mongo. event data point next event data point event play time
  13. > Krossover www.krossover.com twitter.com/krossovr A football field: Schema title chartTitle

    description field collection dimensions unit height width heightpx widthpx displayFieldDivisor fieldOriginX fieldOriginY timeLineUnit region regionId specification dimensions unit height width svg (svg data) region description region title All of our schema rely on some combination of embedding and linking. For modeling a football field, we embed most of the fields we need and link to regions, which are also used when indexing (i.e., did the event occur in the top, center, or bottom region?). We also use this kind of schema design for stats.
  14. > Krossover www.krossover.com twitter.com/krossovr A football field: code { "_id":

    ObjectId("4f0df205f403fe1a01000000"), "description": "Bottom rectangle", "region": "bottom", "title": "Bottom" } { "_id": ObjectId("4f0df1edf403fe2e01000000"), "description": "Center rectangle", "region": "center", "title": "Center" } { "_id": ObjectId("4f0df1d7f403fe2d01000000"), "description": "Top rectangle", "region": "top", "title": "Top" } { "_id": ObjectId("4f0de969f403fed702000000"), "chartTitle": "Field", "description": "Standard football field", "dimensions": { "unit": "yd", "height": 53.33, "width": 120, "widthPx": 1030, "heightPx": 462, "displayFieldDivisor": 1, "fieldOriginX": 10, "fieldOriginY": 0, "timelineUnit": 3 }, ...SVG DATA... "title": "Football field" }
  15. > Krossover www.krossover.com twitter.com/krossovr indexing: schema basic indexing collection Another

    technique we use is to pull in data from within the document. Here, we’re pulling player1 and location from fieldSeq and using this info in the script. we also use this technique for search. gameTeamId script shortcutKey fieldSeq player1 location event eventTitle player1 homePlayer awayPlayer location xcoord ycoord nextEvents status accepted declined { "_id": ObjectId("4f31669677a295f549000013"), "event": "dead ball penalty", "eventTitle": "Dead Ball Penalty", "fieldSeq": { "0": "player1", "1": "location" }, "gameTeamId": "player1", "location": { "0": "#xcoord", "1": "#ycoord" }, "nextEvents": [ ], "player1": { "0": "#homePlayer", "1": "#awayPlayer" }, "script": "After play was ruled dead, a penalty was called on {{{player1}}}at the {{positionFieldX}} yard line", "shortcutKey": "p", "status": { "0": "#accepted", "1": "#declined" }, "timeContext": "standalone" }
  16. > Krossover www.krossover.com twitter.com/krossovr indexing: code { "_id": ObjectId("4f18a9f05abe7e117c000002"), "events":

    { "0": { "event": "post-snap penalty", "eventId": ObjectId("4edc660bc469888065000008") }, "1": { "event": "blitz", "eventId": ObjectId("4edc6604c469888065000007") }, "2": { "event": "first down", "eventId": ObjectId("4f18a9d45abe7e3e29000000") } }, "eventsOld": { "0": "post-snap penalty", "1": "blitz", "2": "first down" }, "group": "setup", "shortcutKey": "u" } { "_id": ObjectId("4f16ed1f5abe7eb173000000"), "events": { "0": { "event": "non-forced fumble", "eventId": ObjectId("4ed70d57c469882675000000") }, "1": { "event": "forced fumble", "eventId": ObjectId("4ed716c8c469884212000000") } }, "eventsOld": { "0": "non-forced fumble", "1": "forced fumble" }, "group": "fumble", "shortcutKey": "f" } vit grouping collection examples using linking & embedding...
  17. > Krossover www.krossover.com twitter.com/krossovr Indexing: code { "_id": ObjectId("4eea5f3955e695a00b000002"), "eventType":

    "tackle", "eventTypeTitle": "Tackle" } { "_id": ObjectId("4eea5f2a55e695a00b000001"), "eventType": "down", "eventTypeTitle": "Down" } { "_id": ObjectId("4eea5f4555e6952410000000"), "eventType": "sack", "eventTypeTitle": "Sack" } { "_id": ObjectId("4f4d055d77a295715c000000"), "description": "summary line for basic indexing", "indexType": "basic-indexing", "summaryLine": { "0": { "priority": 1, "events": { "0": { "event": "post-snap penalty", "eventId": ObjectId("4edc660bc469888065000008") }, "1": { "event": "pre-snap penalty", "eventId": ObjectId("4ededea4c46988bc11000000") }, "2": { "event": "dead ball penalty", "eventId": ObjectId("4f31669677a295f549000013") } } }, basic indexing event type ...and more examples using both linking and embedding in our schema.
  18. >Credits Attributions “Cloud upload” symbol by adam whitcroft, “delivery man”

    symbol by jon trillana, “video” symbol by anas ramadan, “video search” symbol by luis rodriguez, from the noun project collection. “amazon” and the “Amazon web services” logo are registered trademarks of amazon.com, inc. “apache” and the “apache http server project” logo are registered trademarks of the apache software foundation. “git” logo by jason long. “html5” logo by w3c. “Jenkins” logo by the Jenkins Project. “krossover” and the “krossover” logo are registered trademarks of krossover intelligence, inc. “MongoDB”, “mongo”, and the “leaf” logo are registered trademarks of 10gen, inc. “mysql” and the “mysql” logo are registered trademarks of the oracle corporation. “Novecento” typeface by jan tonellato. “PHP” logo by the PHP project. “Rackspace” and the “Rackspace” logo are registered trademarks of rackspace US, inc. “Subversion” and the “subversion” logo are registered trademarks of the apache software foundation. “Yii framework” logo by the Yii Framework. krossover team mallik arjun developer chaitra bhat developer semih cicek designer carlo dicelico developer cyriel dikoume developer kevin haggerty developer stella km developer karthik KN developer abhinav kumar developer R S Sagar developer Abhishek srinivas developer girish walavalkar developer glenn yonemitsu developer vasu kulkarni ceo james piette, phd vp analytics alex kirtland vp products design/ux sandip chaudhari Cto david mayberry director of sales & marketing brett bivens directer of operations sean mccleary Project Manager amyn bandali business development alicia quintero client services Krossover www.krossover.com twitter.com/krossovr