YourNextMP: Post-election Summary

A talk I gave about the YourNextMP project a few days after the UK General Election in 2015 at Oxford Geek Night #40

Transcript

1. YourNextMP: An open, transparently sourced database of candidates for the 2015 UK general election Mark Longair <mark@mysociety.org> Twitter: @mhl20 A Democracy Club project https://democracyclub.org.uk/ https://yournextmp.com CC-BY-SA licensed data, from web-scraping and crowd-sourcing; AGPL licensed code Post-election Summary
2. Photo by Simon Clayson
6. Provenance of the information A citation or explanation of how you know details about the candidate for every change:
9. Candidate statistics • 3,971 candidates (69% of the eventual number of candidates in 2010) • 2,761 new candidates, 1,210 standing again (98 changes of party) • 90% with email • 73% with Twitter usernames • 86% with a photo
10. Photo Moderation
11. Photo Moderation
12. User contributions There were 2004 users who made any contribution
13. Top user contributions
14. Official release of nominations • Deadline for nominating candidates: Thursday 9th of April at 4pm • The “Statement of Persons Nominated” papers are published on council websites from about 5pm (the last were published the next Monday morning – Sutton) • Volunteers found 99% of these from ~400 council website within 24 hours: • Pasted URLs into a big Google spreadsheet • Imported all the documents into YourNextMP • Each constituency was checked to correct our candidate lists (about 1 in 10 needed some fix), then locked • All constituencies locked by early afternoon Saturday 9th April • Still some errors, found on subsequent passes by volunteers
15. Technology PopIt YourNextMP uses PopIt as its datastore for people, organizations, posts and memberships: http://popit.poplus.org/ Provides an HTTP-based API and web interface for storing data about people and organizations in the data model specified by the Popolo project: http://www.popoloproject.com/ Like all the Poplus components, PopIt is designed to make it easier for people to build civic tech sites and services. (PopIt is Node / MongoDB, the YourNextMP interface is Django-based.) yournextmp-popit (Django) PostgreSQL Memcached PopIt
16. How did we do? Data reusers Google The Guardian The Daily Telegraph Trinity Mirror (Ampp3d) Wikimedia http://twittelection.co.uk/ https://www.electionmentions.com/ http://voteswap.org/ Votr http://www.yourcandidates.org.uk/ http://meetyournextmp.com/ http://cv.democracyclub.org.uk/ http://wheredotheystand.org.uk/ http://voteforpolicies.org.uk 38 Degreees http://www.edulect.co.uk/ https://www.voteforbob.co.uk/ http://www.nhsbill2015.org/ http://www.election2015.org.uk/ http://democraticdashboard.com/ http://www.getsurrey.co.uk/ http://election.awedience.com/ http://www.theyworkforyou.com Verto ElectionLeaflets.org http://www.votesmart.org.uk/ Position Dial
17. How did we do? Data reusers Google The Guardian The Daily Telegraph Trinity Mirror (Ampp3d) Wikimedia http://twittelection.co.uk/ https://www.electionmentions.com/ http://voteswap.org/ Votr http://www.yourcandidates.org.uk/ http://meetyournextmp.com/ http://cv.democracyclub.org.uk/ http://wheredotheystand.org.uk/ http://voteforpolicies.org.uk 38 Degreees http://www.edulect.co.uk/ https://www.voteforbob.co.uk/ http://www.nhsbill2015.org/ http://www.election2015.org.uk/ http://democraticdashboard.com/ http://www.getsurrey.co.uk/ http://election.awedience.com/ http://www.theyworkforyou.com Verto ElectionLeaflets.org http://www.votesmart.org.uk/
18. How did we do? Data reusers Google The Guardian The Daily Telegraph Trinity Mirror (Ampp3d) Wikimedia http://twittelection.co.uk/ https://www.electionmentions.com/ http://voteswap.org/ Votr http://www.yourcandidates.org.uk/ http://meetyournextmp.com/ http://cv.democracyclub.org.uk/ http://wheredotheystand.org.uk/ http://voteforpolicies.org.uk 38 Degreees http://www.edulect.co.uk/ https://www.voteforbob.co.uk/ http://www.nhsbill2015.org/ http://www.election2015.org.uk/ http://democraticdashboard.com/ http://www.getsurrey.co.uk/ http://election.awedience.com/ http://www.theyworkforyou.com Verto ElectionLeaflets.org http://www.votesmart.org.uk/
19. How did we do? Data reusers Google The Guardian The Daily Telegraph Trinity Mirror (Ampp3d) Wikimedia http://twittelection.co.uk/ https://www.electionmentions.com/ http://voteswap.org/ Votr http://www.yourcandidates.org.uk/ http://meetyournextmp.com/ http://cv.democracyclub.org.uk/ http://wheredotheystand.org.uk/ http://voteforpolicies.org.uk 38 Degreees http://www.edulect.co.uk/ https://www.voteforbob.co.uk/ http://www.nhsbill2015.org/ http://www.election2015.org.uk/ http://democraticdashboard.com/ http://www.getsurrey.co.uk/ http://election.awedience.com/ http://www.theyworkforyou.com Verto ElectionLeaflets.org http://www.votesmart.org.uk/ Position Dial
20. How did we do? Providing information directly
21. Malicious or incorrect edits Changed policy over time • Initial emphasis on crowd-sourcing and bringing in volunteers: • Confirmed email address required for each user • Post-moderation (volunteers checking “Recent Changes”) • Except for photos: uploaded to a moderation queue • Only some users allowed to merge candidates • Edits restricted as we got closer to the election: • Locked candidates & parties after Statements of Persons Nominated were announced • No candidate renames allowed • No new accounts allowed • No non-staff edits allowed Less than 10 problematic users identified
22. Election night “We're up all night to get MPs” (https://www.mysociety.org/2015/05/05/were-up-all-night-to-get-mps/) We attempted to record the winner of each constituency (though not vote counts) in real time from the BBC coverage: • TheyWorkForYou was updated in real time from that • @democlub tweeted each new MP • Atom feed of results These might have been the only free and structured election results data
23. Election night
24. Lessons: Copyright of uploaded photos Broadly: no one knows anything about copyright of photos. This is what we asked:
25. Lessons: User support We got a lot of angry emails: • Most commonly from people who didn't realise their personal information was easily findable online anyway. • We ended up explaining Facebook privacy options quite frequently • Problems with delays in new party registrations appearing in the Electoral Commission What worked best: • Responding as quickly as possible • Being unreservedly apologetic (even when we thought people were being unreasonable)
26. Lessons: Copyright and licensing of data Talk to a lawyer with lots of experience of intellectual property questions before starting a crowd-sourcing project like this. • Needed a legal entity to own the data • Understanding database rights • Problems with building on 2010 data • Needed the flexibility to relicense for some people This took a lot of time to sort out. Francis Davey gave us expert advice on this (http://www.francisdavey.co.uk/)
27. Lessons: Electoral geekery Only descriptions (not necessarily party names) are listed on the SOPNs; also the chosen party emblem isn't shown. For example:
28. Lessons: Electoral geekery Something I didn't know: you can stand jointly for two parties • e.g. the best known example: “Labour and Co-operative Party” Joint parties don't have pages on the Electoral Commission website; you have to infer them from descriptions, e.g. Left Unity Registered descriptions include: Left Unity - Trade Unionists and Socialists (Joint Descriptions with Trade Unionists and Socialists Coalition) Trade Unionist and Socialist Coalition Registered descriptions include: Left Unity - Trade Unionists and Socialists (Joint Description with Left Unity) We found 6 such joint parties.
29. The future... What would I most want to change about the current system? • To not have to do this – it's ridiculous • Nominations should require an email address (or phone number) • It's impractical to verify candidate information without this • Councils should publish nominations as soon as they receive them • Nominations should be published as structured data • Nominations should be collected centrally, e.g. by the Electoral Commission Future for the code-base • Argentina, Chicago, Minnesota • UK elections in 2016: • Northern Ireland Assembly • National Assembly for Wales • Scottish Parliament
30. Acknowledgements Other developers and designers: • Sym Roe, Tim Green, Andy Lulham, Zarino Zappia, Liz Conlan, Martin Wright • Matthew Somerville, Chris Mytton, James Baster PopIt developers: • Chris Mytton and Struan Donald Amazing volunteers: • andylolz, tfgg, JPCarrington, edent, RichCBury1, TimPatmore, sjorford, symroe, mattl, and many more Democracy Club: Joe Mitchell, Emily Randall Full Fact: Will Moy, Mevan Babakar For more information: • Democracy Club: https://democracyclub.org.uk/ Mark Longair <mark@mysociety.org> Twitter: @mhl20