Content State of the Union

A tech talk I gave to the Khan Academy team about the state of our content editing and publishing infrastructure.

Transcript

1. Content State of the Union Dylan Vassallo
2. - Root! ! - Domain! - Subject →Mission - Topic - Tutorial - Video - Exercise - Perseus item - Article - Scratchpad
3. 1,958! 6,398! 1,195! 15,671! 698! 182! 17! 26,119 topics videos exercises (1/3 KE, 2/3 Perseus) Perseus items articles scratchpads (non-UGC) missions total +
4. /devadmin/content /devadmin/content/topic/<slug> Medusa tutorial editor Medusa subject/topic editor Perseus Perseus One Knowledge Map editor ~10k edits in the past 7 days
8. BaseVideo title, description, youtube_id, author_names, … Video topic_parent_string_keys, source_table … VideoRevision content_id, author_key, do_not_publish, …
9. 1. Create a new revision (brand new or forked from existing) with the requested properties ! 2. Validate the revision ! 3. Update the ContentStage with the revision
10. FrozenRevisionStore ContentStage edit topic tree forked revision new revision
13. 1. Move the specified changes out of the ContentStage and into a new ContentCommit ! 2. Generate the new live topic tree ! 3. Write out a FrozenRevisionStore and FrozenModelStore ! 4. Prime some caches ! 5. Kick off a search indexing job ! 6. Update the last_published_commit setting Daily record: 55 successful publishes on 1/23/14
14. FrozenModelStore last_published_commit live topic tree FrozenRevisionStore FrozenModelStore FrozenRevisionStore ContentStage edit topic tree
15. - Did we present the publisher with a correct diff?! ! ! - Did the publisher choose the right options from the diff?! ! ! - Did we apply those selections to the ContentStage and ContentCommit correctly?! ! ! - Was the live topic tree generated correctly?! ! ! - To whom must I promise my firstborn to escape an eternity in devshell.py --prod?
16. - Did we present the publisher with a correct diff?
 PublishDiff (write better code and less of it, you dummie) ! - Did the publisher choose the right options from the diff?! Fix UX problems ! - Did we apply those selections to the ContentStage and ContentCommit correctly?! PublishDiff ! - Was the live topic tree generated correctly?! PublishEntityStore ! - To whom must I promise my firstborn to escape an eternity in devshell.py --prod? TBD
17. 1. Server creates and sends down a PublishDiff, a list of changes that each have a list of contexts in which they can be published. ! 2. The user browses the contexts and the changes within them and makes their selections. The client sends up the diff id and the list of selections. ! 3. The server updates the PublishDiff with the list of selections and the filtered list of changes. ! 4. The publish job uses the filtered list of changes to create the ContentCommit.
20. FIN