Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Katie Bell - The computer science of marking computer science assignments

Katie Bell - The computer science of marking computer science assignments

When writing systems to test if beginner programmers' code was correct, I didn't expect to need numpy, scipy a custom C module and a whole lot of cool geometry algorithms. Giving actionable feedback on tasks (in this case logo/turtle vector drawings), is necessary for the learning process and goes some fun places. Take this as a case study of writing efficient geometry number crunching in Python.


PyCon 2016

May 29, 2016

More Decks by PyCon 2016

Other Decks in Programming


  1. Example from turtle import * left(60) forward(100) right(120) forward(100) right(120)

    forward(100) left(90) forward(100) left(90) forward(100) left(90) forward(100)
  2. ◦ Simple, graphical, engaging ◦ Built-in module in Python ◦

    Teachers were asking for it ◦ Ties in well with maths teaching Why Turtle?
  3. True to built-in Python Turtle in the browser Frontend Sandboxed

    Python Browser Code and stdin Realtime animation events SVG animation Modified turtle.py
  4. “You’re missing this line: {diagram}” “The angle between these two

    lines should be {angle} but it’s {angle}°.” “This line is the wrong colour. It was {colour1} but it should be {colour2}.” “There’s an extra line here {diagram}, that shouldn’t be there.” Meaningful feedback goals
  5. Visually distinguishable differences between drawings should matter. ◦ Additional lines,

    Missing lines ◦ Lines of different lengths/positions So we have two drawings: Expected and Actual
  6. ◦ Fuzzy floating point matching ◦ Drawing order should not

    matter. ◦ Overlapping lines should not matter. ◦ Translation of the whole drawing should not matter Additional Requirements:
  7. from turtle import * fillcolor('red') begin_fill() for i in range(3):

    forward(100) left(120) end_fill() Fills with turtle
  8. “Resolving overlapping polygons is a solved problem right? There should

    be a ready made algorithm and library for this.” - Me
  9. 1. Resolve shapes to non-overlapping triangles 2. Work out what

    colour each triangle is 3. Stitch the triangles together to form an outline of the filled area 4. Compare the vectors of filled area outlines for each colour Here’s the plan:
  10. Take the centroid of each triangle. Determine the highest (drawing

    order) shape which contains that point. Colour each triangle
  11. $ nosetests test_checker_logo_diff.py --with-profile ................................................ 13327123 function calls (13321330 primitive

    calls) in 22.131 seconds Ordered by: cumulative time ncalls cumtime percall filename:lineno(function) 9/1 22.131 22.131 site-packages/nose/suite.py:176(__call__) 9/1 22.131 22.131 site-packages/nose/suite.py:197(run) 48 22.129 0.461 site-packages/nose/case.py:44(__call__) 48 22.129 0.461 site-packages/nose/case.py:115(run) 48 22.124 0.461 site-packages/nose/case.py:142(runTest) 48 22.124 0.461 unittest/case.py:394(__call__) 48 22.124 0.461 unittest/case.py:297(run) 38 22.091 0.581 checker_logo_diff.py:1094(check) 146 21.554 0.148 checker_logo_diff.py:925(resolve_fill_triangles) 90280 20.810 0.000 checker_logo_diff.py:140(is_intersection) 538466 14.157 0.000 site-packages/numpy/core/numeric.py:2276(isclose) 467962 13.619 0.000 site-packages/numpy/core/numeric.py:2212(allclose) 76 11.475 0.151 checker_logo_diff.py:773(collapse_shapes_to_lines) Profiling
  12. Conditions: 1. No two line segment endpoints or crossings have

    the same x-coordinate 2. No line segment endpoint lies upon another line segment 3. No three line segments intersect at a single point. Attempt 1: Bentley–Ottmann algorithm
  13. ◦ 2 days of development time ◦ 84% of test

    cases passing ◦ Speed roughly the same as before Conclusion: Not working Attempt 1: Bentley–Ottmann algorithm
  14. ◦ 305 lines of horrific SWIG and C++ typedefs ◦

    ~1 day of development time ◦ Test suite even slower Conclusion: Not working Attempt 2: CGAL
  15. Attempt 3: Write a custom C Module def is_intersection(seg1, seg2,

    check_dir=False, ends=True): return fastlogo.fast_is_intersection( tuple(seg1[0]), tuple(seg1[1]), tuple(seg2[0]), tuple(seg2[1]), check_dir, ends)
  16. Lessons Learned: 1. Geometry is fun! 2. Profiling is the

    best. 3. If you want more speed, forget algorithms and rewrite it in C.
  17. ◦ True-to-builtin Python Turtle in the browser ◦ Good automated

    feedback to students Todo: ◦ Add yet more detailed feedback ◦ More algorithmic/speed improvements Where we are now: