The Computer Science of Marking
Computer Science Assignments
Katie Bell
Software Engineer - Grok Learning
Slide 2
Slide 2 text
I’ve never heard of this ‘Grok Learning’
Image: CC0 Public Domain
Slide 3
Slide 3 text
Background
Slide 4
Slide 4 text
Let me tell you a story...
Image: The Princess Bride (1987)
Slide 5
Slide 5 text
Turtles
“It’s turtles all the way down!”
1
Slide 6
Slide 6 text
Example
from turtle import *
left(60)
forward(100)
right(120)
forward(100)
right(120)
forward(100)
left(90)
forward(100)
left(90)
forward(100)
left(90)
forward(100)
Slide 7
Slide 7 text
Example challenges
Slide 8
Slide 8 text
○ Simple, graphical, engaging
○ Built-in module in Python
○ Teachers were asking for it
○ Ties in well with maths teaching
Why Turtle?
Slide 9
Slide 9 text
True to built-in Python Turtle in the browser
Frontend
Sandboxed
Python
Browser
Code and
stdin Realtime
animation events
SVG animation
Modified
turtle.py
Slide 10
Slide 10 text
The marking system
Because feedback is important.
2
Slide 11
Slide 11 text
“You’re missing this line: {diagram}”
“The angle between these two lines should
be {angle} but it’s {angle}°.”
“This line is the wrong colour. It was
{colour1} but it should be {colour2}.”
“There’s an extra line here {diagram}, that
shouldn’t be there.”
Meaningful feedback goals
Comparing pixel by pixel
angle = 360/7
angle = int(360/7)
Slide 14
Slide 14 text
Vector Comparison
Down the rabbit hole...
3
Slide 15
Slide 15 text
Visually distinguishable differences between
drawings should matter.
○ Additional lines, Missing lines
○ Lines of different lengths/positions
So we have two drawings: Expected and Actual
Slide 16
Slide 16 text
○ Fuzzy floating point matching
○ Drawing order should not matter.
○ Overlapping lines should not matter.
○ Translation of the whole drawing should
not matter
Additional Requirements:
Slide 17
Slide 17 text
Connected & Overlapping Lines
Slide 18
Slide 18 text
Simplifying
Raw line segments
from turtle drawing
Minimised line
segments
Slide 19
Slide 19 text
Where we’re at:
Slide 20
Slide 20 text
Filled areas!
“It’s complicated.” - Facebook
4
Slide 21
Slide 21 text
from turtle import *
fillcolor('red')
begin_fill()
for i in range(3):
forward(100)
left(120)
end_fill()
Fills with turtle
Slide 22
Slide 22 text
The fun of fills
Slide 23
Slide 23 text
The fun of fills
Slide 24
Slide 24 text
The fun of fills
White rectangle drawn
over the top
Slide 25
Slide 25 text
The fun of fills
Slide 26
Slide 26 text
“Resolving overlapping polygons is a solved
problem right? There should be a ready made
algorithm and library for this.”
- Me
from scipy.spatial import Delaunay
Conveniently in Scipy
Slide 31
Slide 31 text
1. Resolve shapes to non-overlapping triangles
2. Work out what colour each triangle is
3. Stitch the triangles together to form an
outline of the filled area
4. Compare the vectors of filled area outlines
for each colour
Here’s the plan:
Slide 32
Slide 32 text
Applying triangulation
Slide 33
Slide 33 text
Applying triangulation
Slide 34
Slide 34 text
Applying triangulation
Slide 35
Slide 35 text
Take the centroid of each triangle.
Determine the highest (drawing order)
shape which contains that point.
Colour each triangle
Slide 36
Slide 36 text
Testing if a point is in a complex polygon
Slide 37
Slide 37 text
Testing if a point is in a complex polygon
Slide 38
Slide 38 text
Testing if a point is in a complex polygon
Slide 39
Slide 39 text
Testing if a point is in a complex polygon
Slide 40
Slide 40 text
INTERSECTING LINES
Trickier than originally thought.
Slide 41
Slide 41 text
In order for the triangulation to work
Slide 42
Slide 42 text
In order for the triangulation to work
Slide 43
Slide 43 text
Resolve all the shapes in one triangulation
Slide 44
Slide 44 text
Resolve all the shapes in one triangulation
Slide 45
Slide 45 text
Stitching together
Slide 46
Slide 46 text
Stitching together
Slide 47
Slide 47 text
Stitching together
Slide 48
Slide 48 text
Stitching together
Slide 49
Slide 49 text
Stitching together
Slide 50
Slide 50 text
Stitching together
Slide 51
Slide 51 text
Stitching together
Slide 52
Slide 52 text
Simplify the vectors and we're done!
Slide 53
Slide 53 text
Sweet, it works!
But there's one small problem...
5
Conditions:
1. No two line segment endpoints or
crossings have the same x-coordinate
2. No line segment endpoint lies upon
another line segment
3. No three line segments intersect at a
single point.
Attempt 1: Bentley–Ottmann algorithm
Slide 60
Slide 60 text
○ 2 days of development time
○ 84% of test cases passing
○ Speed roughly the same as before
Conclusion: Not working
Attempt 1: Bentley–Ottmann algorithm
Slide 61
Slide 61 text
Attempt 2: CGAL
The Computational Geometry Algorithms Library (C++)
Slide 62
Slide 62 text
github.com/CGAL/cgal-swig-bindings
“This project is still experimental and more
packages will be added.”
So uh… Python bindings?
Slide 63
Slide 63 text
But it was too precise...
Slide 64
Slide 64 text
○ 305 lines of horrific SWIG and C++ typedefs
○ ~1 day of development time
○ Test suite even slower
Conclusion: Not working
Attempt 2: CGAL
Slide 65
Slide 65 text
is_intersection
Slide 66
Slide 66 text
Attempt 3: Write a custom C Module
def is_intersection(seg1, seg2, check_dir=False, ends=True):
return fastlogo.fast_is_intersection(
tuple(seg1[0]),
tuple(seg1[1]),
tuple(seg2[0]),
tuple(seg2[1]),
check_dir,
ends)
Slide 67
Slide 67 text
~10x speed increase!
For both birthday banner and the test suite.
Slide 68
Slide 68 text
Lessons Learned:
1. Geometry is fun!
2. Profiling is the best.
3. If you want more speed, forget algorithms
and rewrite it in C.
Slide 69
Slide 69 text
○ True-to-builtin Python Turtle in the browser
○ Good automated feedback to students
Todo:
○ Add yet more detailed feedback
○ More algorithmic/speed improvements
Where we are now:
Slide 70
Slide 70 text
Questions?
@notsolonecoder, @groklearning
katie@groklearning.com
Try a turtle activity (Hour of Code):
groklearning.com/hoc-2015