YourNextMP: Post-election Summary

76ec36568ba9b02bdb3fc160495ea75e?s=47 Mark Longair
May 14, 2015

YourNextMP: Post-election Summary

A talk I gave about the YourNextMP project a few days after the UK General Election in 2015 at Oxford Geek Night #40


Mark Longair

May 14, 2015


  1. YourNextMP: An open, transparently sourced database of candidates for the

    2015 UK general election Mark Longair <> Twitter: @mhl20 A Democracy Club project CC-BY-SA licensed data, from web-scraping and crowd-sourcing; AGPL licensed code Post-election Summary
  2. Photo by Simon Clayson

  3. None
  4. None
  5. None
  6. Provenance of the information A citation or explanation of how

    you know details about the candidate for every change:
  7. None
  8. None
  9. Candidate statistics • 3,971 candidates (69% of the eventual number

    of candidates in 2010) • 2,761 new candidates, 1,210 standing again (98 changes of party) • 90% with email • 73% with Twitter usernames • 86% with a photo
  10. Photo Moderation

  11. Photo Moderation

  12. User contributions There were 2004 users who made any contribution

  13. Top user contributions

  14. Official release of nominations • Deadline for nominating candidates: Thursday

    9th of April at 4pm • The “Statement of Persons Nominated” papers are published on council websites from about 5pm (the last were published the next Monday morning – Sutton) • Volunteers found 99% of these from ~400 council website within 24 hours: • Pasted URLs into a big Google spreadsheet • Imported all the documents into YourNextMP • Each constituency was checked to correct our candidate lists (about 1 in 10 needed some fix), then locked • All constituencies locked by early afternoon Saturday 9th April • Still some errors, found on subsequent passes by volunteers
  15. Technology PopIt YourNextMP uses PopIt as its datastore for people,

    organizations, posts and memberships: Provides an HTTP-based API and web interface for storing data about people and organizations in the data model specified by the Popolo project: Like all the Poplus components, PopIt is designed to make it easier for people to build civic tech sites and services. (PopIt is Node / MongoDB, the YourNextMP interface is Django-based.) yournextmp-popit (Django) PostgreSQL Memcached PopIt
  16. How did we do? Data reusers Google The Guardian The

    Daily Telegraph Trinity Mirror (Ampp3d) Wikimedia Votr 38 Degreees Verto Position Dial
  17. How did we do? Data reusers Google The Guardian The

    Daily Telegraph Trinity Mirror (Ampp3d) Wikimedia Votr 38 Degreees Verto
  18. How did we do? Data reusers Google The Guardian The

    Daily Telegraph Trinity Mirror (Ampp3d) Wikimedia Votr 38 Degreees Verto
  19. How did we do? Data reusers Google The Guardian The

    Daily Telegraph Trinity Mirror (Ampp3d) Wikimedia Votr 38 Degreees Verto Position Dial
  20. How did we do? Providing information directly

  21. Malicious or incorrect edits Changed policy over time • Initial

    emphasis on crowd-sourcing and bringing in volunteers: • Confirmed email address required for each user • Post-moderation (volunteers checking “Recent Changes”) • Except for photos: uploaded to a moderation queue • Only some users allowed to merge candidates • Edits restricted as we got closer to the election: • Locked candidates & parties after Statements of Persons Nominated were announced • No candidate renames allowed • No new accounts allowed • No non-staff edits allowed Less than 10 problematic users identified
  22. Election night “We're up all night to get MPs” (

    We attempted to record the winner of each constituency (though not vote counts) in real time from the BBC coverage: • TheyWorkForYou was updated in real time from that • @democlub tweeted each new MP • Atom feed of results These might have been the only free and structured election results data
  23. Election night

  24. Lessons: Copyright of uploaded photos Broadly: no one knows anything

    about copyright of photos. This is what we asked:
  25. Lessons: User support We got a lot of angry emails:

    • Most commonly from people who didn't realise their personal information was easily findable online anyway. • We ended up explaining Facebook privacy options quite frequently • Problems with delays in new party registrations appearing in the Electoral Commission What worked best: • Responding as quickly as possible • Being unreservedly apologetic (even when we thought people were being unreasonable)
  26. Lessons: Copyright and licensing of data Talk to a lawyer

    with lots of experience of intellectual property questions before starting a crowd-sourcing project like this. • Needed a legal entity to own the data • Understanding database rights • Problems with building on 2010 data • Needed the flexibility to relicense for some people This took a lot of time to sort out. Francis Davey gave us expert advice on this (
  27. Lessons: Electoral geekery Only descriptions (not necessarily party names) are

    listed on the SOPNs; also the chosen party emblem isn't shown. For example:
  28. Lessons: Electoral geekery Something I didn't know: you can stand

    jointly for two parties • e.g. the best known example: “Labour and Co-operative Party” Joint parties don't have pages on the Electoral Commission website; you have to infer them from descriptions, e.g. Left Unity Registered descriptions include: Left Unity - Trade Unionists and Socialists (Joint Descriptions with Trade Unionists and Socialists Coalition) Trade Unionist and Socialist Coalition Registered descriptions include: Left Unity - Trade Unionists and Socialists (Joint Description with Left Unity) We found 6 such joint parties.
  29. The future... What would I most want to change about

    the current system? • To not have to do this – it's ridiculous • Nominations should require an email address (or phone number) • It's impractical to verify candidate information without this • Councils should publish nominations as soon as they receive them • Nominations should be published as structured data • Nominations should be collected centrally, e.g. by the Electoral Commission Future for the code-base • Argentina, Chicago, Minnesota • UK elections in 2016: • Northern Ireland Assembly • National Assembly for Wales • Scottish Parliament
  30. Acknowledgements Other developers and designers: • Sym Roe, Tim Green,

    Andy Lulham, Zarino Zappia, Liz Conlan, Martin Wright • Matthew Somerville, Chris Mytton, James Baster PopIt developers: • Chris Mytton and Struan Donald Amazing volunteers: • andylolz, tfgg, JPCarrington, edent, RichCBury1, TimPatmore, sjorford, symroe, mattl, and many more Democracy Club: Joe Mitchell, Emily Randall Full Fact: Will Moy, Mevan Babakar For more information: • Democracy Club: Mark Longair <> Twitter: @mhl20