Atlas - owl vs deer - Speaker Deck

Slide 1

Slide 1 text

Atlas 4.0 owl vs deer Fred van den Driessche Engineer @fredvdd

Slide 2

Slide 2 text

What is Atlas? “the global video and audio index”

Slide 3

Slide 3 text

What is Atlas? • aggregates metadata from multiple sources • stores in a uniform model • APIs + feeds provide consolidated data

Slide 4

Slide 4 text

What is Atlas? ATLAS DB BBC PA C4 etc... /content /schedules /topics sitemaps radioplayer interlinking

Slide 5

Slide 5 text

Atlas Data Model brand item series version broadcast location

Slide 6

Slide 6 text

History •2008: 0.5 • MySQL • URIs • RDF • POST to Update • /content

Slide 7

Slide 7 text

History •2009: 1.0 • Adapters • POST to Update

Slide 8

Slide 8 text

History •2010: 2.0 “otter” • MongoDB • denormalization • /discover • /schedules • uploading

Slide 9

Slide 9 text

History •2011: 3.0 “owl” • MongoDB :-( • less denormalization • /discover • annotations

Slide 10

Slide 10 text

DEMO

Slide 11

Slide 11 text

Objectives for 4.0 “deer” Easy scaling • Discovery • Faster updates • More consistent, rational API • extensive use of HTTP

Slide 12

Slide 12 text

External Changes - quite small • IDs not URIs • URL structures: /content/ not /content?uri= • More ﬂexible ﬁltering • Better annotations • Richer output • Better error handling • Consistently fast

Slide 13

Slide 13 text

Internal Changes - almost everything • New Technologies • New Persistence Layer • Flexible Denormalisation • Bootstrapping • Queue

Slide 14

Slide 14 text

DEMO

Slide 15

Slide 15 text

Optional Extras • improvements to storage and indexing • monitor all the things • small core • decentralization • queue-triggered processing • all data submitted via POST? • generalization of feed output • new equivalence algorithm?

Slide 16

Slide 16 text

Questions? more at http://atlas.metabroadcast.com follow @mb_atlas