Save 37% off PRO during our Black Friday Sale! »

PyConZA 2014 - Writing Python Code to Decide an Election

PyConZA 2014 - Writing Python Code to Decide an Election

Earlier this year Ona was given three weeks to write the software that will tally votes in the Libyan elections and decide who wins and who loses. This is not something we could get wrong. We combined agile development with best practices in testing and QA to build an open source tally system that was well tested, accurate, and easy to use. We will describe a success story of iterative behavior/test-driven-development under extreme conditions. Did the structure of the data change the day before the election? Yes. Did we have the tests to ensure that our implementation changes would not compromise the system's integrity? Yes, and they didn't.

This talk provides a narrative to both Software Engineers and Tech/Product Managers describing why best practices are essential for any organization and any project of any size. We will provide the audience with:

Real world examples they can implement in their own workflow and organizations,
Insight into what succeeded (quick iteration with prioritization) and what was challenging (nothing being static),
Anecdotes and coherent arguments they can take back to their organization to advocate for best practices.

0e5157ffe5fabf6c6eae051dff43c85c?s=128

Peter Lubell-Doughtie

October 02, 2014
Tweet

Transcript

  1. Writing Python Code to Decide an Election Peter Lubell-Doughtie

  2. http://www.ly.undp.org/etc/designs/ UNDPIndiaDesign/img/home/logo- undp.png

  3. None
  4. None
  5. None
  6. the problem • tally many votes from many districts in

    many elections !
  7. the problem • tally many votes from many districts in

    many elections • elections may span multiple district
  8. the problem • tally many votes from many districts in

    many elections • elections may span multiple district • each ballot has a 7-stage verification process
  9. Intake

  10. Intake Data Entry

  11. Intake Data Entry Corrections

  12. Intake Data Entry Corrections Quality Control

  13. Intake Data Entry Corrections Quality Control Archiving

  14. …the other problem • we had 3 weeks

  15. …the other problem • we had 3 weeks

  16. only the essentials

  17. only the essentials • what functionality must exist? !

  18. only the essentials • what functionality must exist? • what

    tools already do some of that?
  19. only the essentials • what functionality must exist? • what

    tools already do some of that? • what if we miss something?
  20. a schema as a tool

  21. a schema as a tool • decide how data is

    organized ! !
  22. a schema as a tool • decide how data is

    organized • shared reference !
  23. a schema as a tool • decide how data is

    organized • shared reference • scopes possible relationships
  24. a schema as a tool • decide how data is

    organized • shared reference • scopes possible relationships • it’s updatable
  25. issues and milestones

  26. issues and milestones • convert discussion to issues !

  27. issues and milestones • convert discussion to issues • decide

    prioritization through milestones
  28. issues and milestones • convert discussion to issues • decide

    prioritization through milestones • communicate to what will get done when
  29. do it

  30. do it 1. assign an issue ! ! ! !

    !
  31. do it 1. assign an issue 2. write tests for

    the issue ! ! ! !
  32. do it 1. assign an issue 2. write tests for

    the issue 3. write code for the tests ! ! !
  33. do it 1. assign an issue 2. write tests for

    the issue 3. write code for the tests 4. issue a pull request ! !
  34. do it 1. assign an issue 2. write tests for

    the issue 3. write code for the tests 4. issue a pull request 5. review the code !
  35. do it 1. assign an issue 2. write tests for

    the issue 3. write code for the tests 4. issue a pull request 5. review the code 6. pass the CI
  36. do it 1. assign an issue 2. write tests for

    the issue 3. write code for the tests 4. open a pull request 5. review the code 6. pass the CI 7. merge and go to step (1.)
  37. automate speed

  38. automate speed • anything that is done twice gets a

    script
  39. automate speed • anything that is done twice gets a

    script • anything that breaks once gets a test
  40. we used south

  41. we used postgres

  42. we used postgres

  43. nose, flake8, and coverage

  44. travis for ci

  45. do not write code

  46. do not write code • unless you really really have

    to ! ! !
  47. guardian for permissions

  48. enumfield state machines

  49. enumfield state machines

  50. reversion for data purity

  51. fabric for remote execution

  52. plan to handle what you have not planned for

  53. plan to handle what you have not planned for •

    elections can span multiple districts
  54. plan to handle what you have not planned for •

    elections can span multiple districts • did not know until the first day of live testing
  55. plan to handle what you have not planned for

  56. plan to handle what you have not planned for •

    we changed the data model => south ! !
  57. plan to handle what you have not planned for •

    we changed the data model => south • we wrote tests => django-nose !
  58. plan to handle what you have not planned for •

    we changed the data model => south • we wrote tests => django-nose • we changed the code => travis-ci
  59. plan to handle what you have not planned for •

    we changed the data model => south • we wrote tests => django-nose • we changed the code => travis-ci • we deployed => fabric
  60. sage advice • “simplicity is prerequisite for reliability” - Dijkstra

  61. references • http://south.readthedocs.org/ • https://github.com/5monkeys/django-enumfield • http://www.fabfile.org/ • https://flake8.readthedocs.org/ •

    https://bitbucket.org/ned/coveragepy
  62. thank you! • https://github.com/onaio/tally-ho • http://company.ona.io/ • http://peet.ldee.org/

  63. peter@ona.io