$30 off During Our Annual Pro Sale. View Details »

Teaching Programming Online at Khan Academy

Pamela Fox
October 23, 2013

Teaching Programming Online at Khan Academy

A talk given at USC about what Khan Academy is, where our technical challenges lie, featuring a deep dive on how we teach the programming curriculum.

Pamela Fox

October 23, 2013
Tweet

More Decks by Pamela Fox

Other Decks in Technology

Transcript

  1. Teaching Programming Online
    pamela fox
    @pamelafox
    Wednesday, October 23, 13

    View Slide

  2. My Background
    Computer Science Linguistics 3d Animation
    + +
    Developer Relations
    2002
    2007
    2007
    2011
    2012
    2013
    Frontend Engineer
    Wednesday, October 23, 13

    View Slide

  3. August 2013 - ??????
    Wednesday, October 23, 13

    View Slide

  4. Wednesday, October 23, 13

    View Slide

  5. Wednesday, October 23, 13

    View Slide

  6. K-12 Subject Tutorials
    Wednesday, October 23, 13

    View Slide

  7. Exercises & Videos
    4,800+ English videos
    100,000+ practice problems
    Wednesday, October 23, 13

    View Slide

  8. Personalized Learning
    Wednesday, October 23, 13

    View Slide

  9. Teacher-facing Analytics
    Wednesday, October 23, 13

    View Slide

  10. Students Everywhere
    Wednesday, October 23, 13

    View Slide

  11. Rapid Growth
    8M+ monthly unique visitors
    300M+ video views
    1.3B+ problems done
    Wednesday, October 23, 13

    View Slide

  12. The Tech Stack
    Backend
    Frontend
    Backbone FB React
    jQuery Handlebars LESS
    Wednesday, October 23, 13

    View Slide

  13. Heavy on Open Source
    Using Contributing
    https://github.com/Khan
    Blogging
    http://bjk5.com/
    http://mattfaus.com/
    Wednesday, October 23, 13

    View Slide

  14. Wednesday, October 23, 13

    View Slide

  15. What should we teach, exactly?
    Wednesday, October 23, 13

    View Slide

  16. So many options
    Languages
    Uses
    Java Python
    C++
    Haskell
    JavaScript
    Ruby
    Scheme
    Games
    Animation
    Hardware
    Robotics
    Data Science
    Simulation
    Mobile
    Websites
    Lua
    Wednesday, October 23, 13

    View Slide

  17. Our goals
    No Installation Needed
    Shareable
    Fun for Anyone
    Gateway Drug
    Wednesday, October 23, 13

    View Slide

  18. So many options
    Languages
    Uses
    Java Python
    C++
    Haskell
    JavaScript
    Ruby
    Scheme
    Games
    Animation
    Hardware
    Robotics
    Data Science
    Simulation
    Mobile
    Websites
    Lua
    Wednesday, October 23, 13

    View Slide

  19. How do students program?
    Wednesday, October 23, 13

    View Slide

  20. ACE editor
    JSHint ! BabyHint ! Loop Checker
    ProcessingJS
    Wednesday, October 23, 13

    View Slide

  21. ACE Editor
    number scrubber color picker
    Wednesday, October 23, 13

    View Slide

  22. ACE Editor
    number scrubber color picker
    Wednesday, October 23, 13

    View Slide

  23. ACE Editor
    number scrubber color picker
    Wednesday, October 23, 13

    View Slide

  24. ProcessingJS
    http://processingjs.org/reference
    Wednesday, October 23, 13

    View Slide

  25. JSHint
    var myName = “spaghetti
    if (i == 0) {
    }
    var i = 2;
    if (i == 0) {
    }
    errors
    warnings
    best
    practices
    Wednesday, October 23, 13

    View Slide

  26. BabyHint
    elipse(10, 10, 20, 30);
    spelling
    wrong
    arguments
    ellipse(1, 1, 20, 30, 5);
    Wednesday, October 23, 13

    View Slide

  27. Infinite Loop Checker
    var i = 0;
    while(i < 10) {
    ellipse(i, i, 30, 30);
    }
    Web Worker
    Wednesday, October 23, 13

    View Slide

  28. Now, how do we teach?
    Wednesday, October 23, 13

    View Slide

  29. Usual way to teach: Videos
    https://www.khanacademy.org/science/computer-science/v/python-lists
    Wednesday, October 23, 13

    View Slide

  30. Our approach: “talk-throughs”
    Uses same environment they program in
    Making passive instruction interactive!
    https://www.khanacademy.org/cs/programming/drawing-basics/p/intro-to-drawing
    Wednesday, October 23, 13

    View Slide

  31. Playing talk-throughs
    SoundManager2.js
    commands = [
    {"key": "\n", "time": 14124},
    {"key": "\n", "time": 14260},
    {"key": "r", "time": 14676},
    {"key": "e", "time": 14764},
    {"key": "c", "time": 15036},
    {"key": "t", "time": 15548},...]
    or
    var player = soundManager.createSound({
    url: revision.getMp3Url(),
    whileplaying: function() {
    updateTimeLeft(Record.currentTime());
    Record.trigger("playUpdate");
    }
    });
    Wednesday, October 23, 13

    View Slide

  32. Creating talk-throughs
    canvas controls
    recording controls
    Wednesday, October 23, 13

    View Slide

  33. Recording audio
    var multirecorder = new MultiRecorder();
    multirecorder.startRecording();
    https://github.com/Khan/MultiRecorderJS/blob/master/multirecorder.js
    rightBuffer.push(stream[0]);
    leftBuffer.push(stream[1]);
    getUserMedia()
    new Worker()
    multirecorder.stopRecording();
    getInterleaved();
    encodeWAV();
    Wednesday, October 23, 13

    View Slide

  34. How can we assess learning?
    Wednesday, October 23, 13

    View Slide

  35. Usual way to assess: Exercises
    Repeated multiple times with variants
    Wednesday, October 23, 13

    View Slide

  36. Our approach: coding challenges
    Structured yet flexible.
    More than one way to code the solution.
    Wednesday, October 23, 13

    View Slide

  37. ...and they’re fun!
    Wednesday, October 23, 13

    View Slide

  38. How do we “grade” challenges?
    staticTest StructuredJS Esprima
    Wednesday, October 23, 13

    View Slide

  39. Esprima
    http://esprima.org/demo/parse.html#
    var theNumber = 50;
    if (theNumber > 0) {
    }
    AST
    JavaScript
    Wednesday, October 23, 13

    View Slide

  40. StructuredJS
    http://khan.github.io/structuredjs/index.html
    var $numVar = $numVal;
    if ($numVar > 0) {
    rect($x, $y, $w, $h);
    }
    var theNumber = 10;
    fill(255, 255, 255);
    if (theNumber > 0) {
    rect(10, 10, 30, 40);
    }
    if (theNumber < 0) {
    rect(10, 50, 30, 40);
    }
    structure: user code:
    it’s a match!
    Wednesday, October 23, 13

    View Slide

  41. staticTest
    staticTest(“Add the ifs!”, function() {
    var descrip = “Now add an if to check if the number is positive.”;
    var pattern = function() {
    var $numVar = $numVal;
    if ($numVar > 0) {
    rect($x, $y, $w, $h);
    }
    };
    result = match(pattern);
    if (passes(result)) {
    var goodX = structure(pattern, inRange(“$x”, 10, 20));
    if (!matches(goodX)) {
    result = fail(“Hm, does your rect start on the side?”);
    }
    }
    assertMatch(result, descrip, displayP);
    });
    Wednesday, October 23, 13

    View Slide

  42. ...Not quite that simple, though!
    Most challenge tests are hundreds of lines long.
    Most steps have 10-20 helpful messages.
    https://www.khanacademy.org/cs/challenge-exploding-sun/2050946856
    Wednesday, October 23, 13

    View Slide

  43. How do we get feedback on challenges?
    d/1OmFRH5NoBusswiSaSYkoDiUPayycXLnpQh8IX60tJbM/formResponse”
    method="post" target="hidden_iframe">
    Wednesday, October 23, 13

    View Slide

  44. Spreadsheet of user feedback
    pivot table!
    Wednesday, October 23, 13

    View Slide

  45. ...plus automated statistics
    Wednesday, October 23, 13

    View Slide

  46. How can we create a community?
    Wednesday, October 23, 13

    View Slide

  47. Questions & Answers
    Wednesday, October 23, 13

    View Slide

  48. Wilson Voting Algorithm, GAE’d
    def wilson_confidence(upvotes_name, downvotes_name, score):
    """Lower bound of Wilson score 90% confidence interval.
    This is the algorithm Reddit uses to sort comments.
    You should not use this if downvotes are disallowed - it is only useful in
    the presence of both upvotes and downvotes because its ranking is based on
    an estimate of the ratio of upvotes to downvotes.
    See http://www.evanmiller.org/how-not-to-sort-by-average-rating.html
    """
    upvotes = getattr(score, upvotes_name)
    downvotes = getattr(score, downvotes_name)
    if upvotes == 0:
    return -downvotes
    elif upvotes == downvotes:
    return 0
    n = upvotes + downvotes
    z = 1.64485 # 90% confidence z-score
    phat = float(upvotes) / n # p-hat
    return ((phat + z * z / (2 * n) - z *
    math.sqrt((phat * (1 - phat) + z * z / (4 * n)) / n))
    / (1 + z * z / n))
    class TimeIndependentScoreProperty(ndb.ComputedProperty):
    def __init__(self, upvotes_name="upvotes", downvotes_name="downvotes",
    **kwargs):
    super(TimeIndependentScoreProperty, self).__init__(
    functools.partial(wilson_confidence, upvotes_name, downvotes_name),
    **kwargs)
    Wednesday, October 23, 13

    View Slide

  49. Spin-offs!
    Wednesday, October 23, 13

    View Slide

  50. The Hot Programs
    Wednesday, October 23, 13

    View Slide

  51. Reddit Voting Algorithm, GAE’d
    def time_dependent(decay_seconds, upvotes_name, downvotes_name, created_name, score):
    """Ranking based on both age and quality.
    This is the algorithm Reddit uses to sort stories. We want there to be
    churn, a constant stream of new programs hitting the hot page,
    so this algorithm takes into account both the score of the scratchpad and the age.
    See http://amix.dk/blog/post/19588
    """
    s = getattr(score, upvotes_name) - getattr(score, downvotes_name)
    # Weight votes logarithmically - initial votes are worth a ton
    order = math.log(max(abs(s), 1), 10)
    sign = 1 if s > 0 else -1 if s < 0 else 0
    # Seconds since this algorithm's start date
    date = getattr(score, created_name) or datetime.datetime.now()
    seconds = epoch_seconds(date) - 1349108492
    return round(order + sign * seconds / decay_seconds, 7)
    class TimeDependentScoreProperty(ndb.ComputedProperty):
    def __init__(self, decay_seconds, upvotes_name="upvotes",
    downvotes_name="downvotes", created_name="created", **kwargs):
    super(TimeDependentScoreProperty, self).__init__(functools.partial(
    time_dependent, decay_seconds, upvotes_name, downvotes_name,
    created_name), **kwargs)
    Wednesday, October 23, 13

    View Slide

  52. Teaching Programming Online
    Learn Practice Create Share Help
    Wednesday, October 23, 13

    View Slide

  53. Scaling the Website
    Wednesday, October 23, 13

    View Slide

  54. Optimizing Conversion & Learning
    Wednesday, October 23, 13

    View Slide

  55. Improving Performance
    Wednesday, October 23, 13

    View Slide

  56. Join the team!
    Wednesday, October 23, 13

    View Slide

  57. Wednesday, October 23, 13

    View Slide

  58. Software Developer
    Mobile Developer
    Data Scientist
    Product Designer
    Full-time Roles
    Wednesday, October 23, 13

    View Slide

  59. Software Developer Intern
    Mobile Developer Intern
    Data Scientist Intern
    Product Designer Intern
    Intern Roles
    Wednesday, October 23, 13

    View Slide

  60. khanacademy.org/careers
    Wednesday, October 23, 13

    View Slide