Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Debugging Algorithms with Visualizations

Debugging Algorithms with Visualizations

This is a talk I gave recently at Data Science MD. The abstract: "We all build complicated applications that do amazing things. They juggle state for thousands of objects, perform complicated algorithms across multiple threads, and stay up for weeks or months at a time. We need to understand what's happening in our applications and in the terabytes of data our applications produce. Until now the primary means of communication of internal application state to developers has been symbolic; in gigabytes of logs, alert statements, stack traces, and emails. We can do better. Jason will argue the case that we should be building visualizations to debug our code."

This was inspired by the work of Bret Victor, Chris Granger, and others.

Jason Gilman

April 08, 2013
Tweet

More Decks by Jason Gilman

Other Decks in Programming

Transcript

  1. Debugging Algorithms
    with
    Visualizations
    Jason Gilman - Element 84 - 4/8/2013
    @jasongilman
    [email protected]

    View full-size slide

  2. We've got a problem.

    View full-size slide

  3. "
    What we've got here is
    failure to communicate
    "
    -Cool Hand Luke

    View full-size slide

  4. Communication
    Developer Computer
    Code
    Output
    Focus of this talk
    Source: Heinz Nixdorf
    MuseumsForum

    View full-size slide

  5. Developer
    Output
    Computer
    • API Responses
    • GUI Output
    • External Systems
    Changes
    • ie. rows inserted
    into tables
    Externally Visible
    • Logs
    • Stacktraces
    • Alerts
    • Console Messages
    • Debugger
    • REPL
    Internal State

    View full-size slide

  6. Lightweight
    System.out.println("x = " + x);
    alert("I'm here");
    puts "uh oh"

    View full-size slide

  7. Deficiencies:
    Computer
    Complicated State
    Logs,
    stack traces,
    alerts
    Developer

    View full-size slide

  8. Which of these is bigger?
    source: commons.wikimedia.org/wiki/File:Baseball_(crop).jpg

    View full-size slide

  9. Which of these is bigger?
    550 3

    View full-size slide

  10. Which of these is bigger?
    550mm 3m

    View full-size slide

  11. 0
    750
    1500
    2250
    3000
    550mm 3m
    3000mm
    550mm

    View full-size slide

  12. 2012-09-08 20:08:07.430 Endless Space[863:fe0b] AUHAL::AUIOProc: mono buffer too small (1024 > 512)
    Steam_SetMinidumpSteamID: Caching Steam ID: 76561198025064504 [API loaded no]
    Endless Space(863,0xac65aa28) malloc: *** error for object 0x20afa84c: pointer being freed was not allocated
    *** set a breakpoint in malloc_error_break to debug
    Endless Space(863,0xac65aa28) malloc: *** error for object 0x20afa94b: pointer being freed was not allocated
    *** set a breakpoint in malloc_error_break to debug
    Endless Space(863,0xac65aa28) malloc: *** error for object 0x92d3552: pointer being freed was not allocated
    *** set a breakpoint in malloc_error_break to debug
    Endless Space(863,0xac65aa28) malloc: *** error for object 0x20afa94b: pointer being freed was not allocated
    *** set a breakpoint in malloc_error_break to debug
    Endless Space(863,0xac65aa28) malloc: *** error for object 0x9169d52: pointer being freed was not allocated
    *** set a breakpoint in malloc_error_break to debug
    Endless Space(863,0xac65aa28) malloc: *** error for object 0x20afa94b: pointer being freed was not allocated
    *** set a breakpoint in malloc_error_break to debug
    Lib.dll into Unity Child Domain
    Exception in thread "main" java.io.FileNotFoundException: Could not locate clojure/tools/nrepl/ack__init.class or clojure/tools/nrepl/ack.clj on classpath:
    (lein2.clj:1)
    at clojure.lang.Compiler.eval(Compiler.java:5440)
    at clojure.lang.Compiler.eval(Compiler.java:5415)
    at clojure.lang.Compiler.load(Compiler.java:5857)
    at clojure.lang.RT.loadResourceScript(RT.java:340)
    at clojure.lang.RT.loadResourceScript(RT.java:331)
    at clojure.lang.RT.load(RT.java:409)
    at clojure.lang.RT.load(RT.java:381)
    at clojure.core$load$fn__4519.invoke(core.clj:4915)
    at clojure.core$load.doInvoke(core.clj:4914)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core$load_one.invoke(core.clj:4729)
    at clojure.core$load_lib.doInvoke(core.clj:4766)
    at clojure.lang.RestFn.applyTo(RestFn.java:142)
    at clojure.core$apply.invoke(core.clj:542)
    at clojure.core$load_libs.doInvoke(core.clj:4800)
    at clojure.lang.RestFn.applyTo(RestFn.java:137)
    at clojure.core$apply.invoke(core.clj:542)
    at clojure.core$require.doInvoke(core.clj:4881)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at leinjacker.utils$try_resolve$fn__383.invoke(utils.clj:12)
    at leinjacker.utils$try_resolve.invoke(utils.clj:12)
    at clojure.lang.AFn.applyToHelper(AFn.java:161)
    at clojure.lang.AFn.applyTo(AFn.java:151)
    at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2906)
    at clojure.lang.Compiler$MapExpr.eval(Compiler.java:2472)
    at clojure.lang.Compiler$DefExpr.eval(Compiler.java:361)
    at clojure.lang.Compiler.eval(Compiler.java:5429)
    at clojure.lang.Compiler.load(Compiler.java:5857)
    at clojure.lang.RT.loadResourceScript(RT.java:340)
    at clojure.lang.RT.loadResourceScript(RT.java:331)
    at clojure.lang.RT.load(RT.java:409)
    at clojure.lang.RT.load(RT.java:381)
    at clojure.core$load$fn__4519.invoke(core.clj:4915)
    - Completed reload, in 0.075 seconds
    Switching to 800x600 fullscreen
    Platform assembly: /Users/mark/Library/Application Support/Steam/SteamApps/common/Endless Space/Endless Space.app/Contents/Data/Managed/System.dll (this message
    is harmless)
    Platform assembly: /Users/mark/Library/Application Support/Steam/SteamApps/common/Endless Space/Endless Space.app/Contents/Data/Managed/System.Xml.dll (this
    message is harmless)
    Platform assembly: /Users/mark/Library/Application Support/Steam/SteamApps/common/Endless Space/Endless Space.app/Contents/Data/Managed/UnityScript.Lang.dll (this
    message is harmless)
    Platform assembly: /Users/mark/Library/Application Support/Steam/SteamApps/common/Endless Space/Endless Space.app/Contents/Data/Managed/Boo.Lang.dll (this message
    is harmless)
    [Bootloader] version: 1.00.18 - 02
    (Filename: /Applications/buildAgent/work/b0bcff80449a48aa/Runtime/ExportGenerated/MacStandalonePlayer/UnityEngineDebug.cpp Line: 43)
    [Bootloader] command line: /Users/mark/Library/Application Support/Steam/SteamApps/common/Endless Space/Endless Space.app/Contents/MacOS/Endless Space -single-
    instance -windowed -w 2560 -h 1440
    (Filename: /Applications/buildAgent/work/b0bcff80449a48aa/Runtime/ExportGenerated/MacStandalonePlayer/UnityEngineDebug.cpp Line: 43)
    [Bootloader] GetCurrentGameLanguage: available game languages are '', current game languages is 'english'.
    What's the problem?

    View full-size slide

  13. Concrete examples are much more
    useful in the real world

    View full-size slide

  14. You're planning a wedding
    source: imdb.com

    View full-size slide

  15. http://www.flickr.com/photos/gavinbell/81011618
    We need some music

    View full-size slide

  16. Is this a good choice?

    View full-size slide

  17. A good cake for the reception?
    source: allrecipes.com/recipe/zuccotto-cupcakes

    View full-size slide

  18. source: flickr.com/photos/louisephotography/8100744225/

    View full-size slide

  19. source: google.com/trends/explore#q=ruby
    Data Visualization

    View full-size slide

  20. But there's a piece missing
    source: commons.wikimedia.org/wiki/File:Wikt_puzzle_favicon.svg

    View full-size slide

  21. 0
    75
    150
    225
    300
    10 20 30 40 50 60 70 80 90 100
    Run 1 Run 2
    A Static Graph

    View full-size slide

  22. Interactivity
    • Ask Questions
    • Explore Application State
    • Step Forward and Backward through
    algorithm
    • Show metadata

    View full-size slide

  23. These aren't new ideas

    View full-size slide

  24. forward 50
    right 90
    forward 50
    right 90
    forward 50
    ...
    LOGO
    source: http://el.media.mit.edu/logo-foundation/logo/turtle.html

    View full-size slide

  25. source: http://cyberneticzoo.com/?p=1711

    View full-size slide

  26. source: http://cyberneticzoo.com/?p=1711
    Not Just for Kids

    View full-size slide

  27. "object-to-think-with"

    View full-size slide

  28. Developer
    Logo Interpreter
    Surrogate

    View full-size slide

  29. Drawing a House
    square
    triangle
    square
    right 30
    triangle
    A Plan
    right 90
    square
    right 30
    triangle
    Success!

    View full-size slide

  30. Example:
    Spherical Polygon Validation
    Valid Invalid

    View full-size slide

  31. Spatial-calc Ruby Gem
    • Models
    • Point, Ring, Polygon, Arc, etc
    • polygon.valid?
    • arc1.intersects?(arc2)

    View full-size slide

  32. Spatial-viz App
    • Includes spatial-calc gem
    • Sinatra
    • Slim, Sass, CoffeeScript
    • Google Earth Plugin
    • JS Events and interaction

    View full-size slide

  33. Visualization #1
    Arc Intersection Algorithm

    View full-size slide

  34. Concrete
    Representation
    of spherical
    arcs

    View full-size slide

  35. Draggable
    Points
    Visually
    confirm
    algorithm is
    working

    View full-size slide

  36. Show Metadata
    Bounding
    Rectangle
    Great Circle
    Optional

    View full-size slide

  37. 84) SpatialCalc::Arc.intersect Near North Pole2 should intersect Endpoint on North Pole [:endpoint_on_np, :near_np2]
    Failure/Error: arc2.send(should_method, intersect(arc1))
    expected arc (-42.05,88.78 -> 128.32,86.04) to intersect arc (0.0,85.0 -> 0.0,90.0).
    Spatial Viz Link: http://localhost:4567/arc_intersection?arc1_ordinates=-42.05,88.78,128.32,86.04&arc2_ordinates=0.0,85.0,0.0,90.0
    (cmd-double click to view)
    # ./spec/models/arc_spec.rb:33:in `block in test_arc_intersection'
    RSpec
    Failed spec
    Visualize
    Problem
    Link

    View full-size slide

  38. Visualize
    Problem
    Then debug it

    View full-size slide

  39. Visualization
    Driven
    Development

    View full-size slide

  40. 1. Think
    • What's the problem?
    • How can I visualize it?

    View full-size slide

  41. 2. Create a visualization

    View full-size slide

  42. 3. Write some code

    View full-size slide

  43. 4. Rinse and Repeat
    Viz
    Test
    Code

    View full-size slide

  44. Visualizing an Algorithm

    View full-size slide

  45. Polygon
    Problem: Does the polygon contain the North Pole?

    View full-size slide

  46. • Loop over the arcs
    • Calculate heading
    • Find delta from previous
    heading
    • Sum the deltas
    My Algorithm

    View full-size slide

  47. Looping
    over Arcs
    Visual Representation
    of Numeric Value

    View full-size slide

  48. Visualizing Chaos

    View full-size slide

  49. Visualizing Chaos

    View full-size slide

  50. Show the Data
    source: http://hint.fm/wind/gallery/oct-30.js.html

    View full-size slide

  51. Web App Search App Search Index

    View full-size slide

  52. source: flickr.com/photos/epsos/6018530849
    Woodworking
    Carpentry
    &
    Developing
    software is like

    View full-size slide

  53. Jig
    source: flickr.com/photos/crown_molding/5079441077
    wood to cut

    View full-size slide

  54. Jig Benefits
    • Makes work easier (faster)
    • Higher quality results

    View full-size slide

  55. Visualization Benefits
    • Makes work easier (faster)
    • Higher quality results

    View full-size slide

  56. Build your own tools.

    View full-size slide

  57. Interactive Visualizations

    View full-size slide

  58. Abstract Becomes Concrete
    {
    lat1:89,
    lon1:88,
    lat2:5,
    lon2:37.3
    }

    View full-size slide

  59. 84) SpatialCalc::Arc.intersect Near North Pole2 should intersect Endpoint on North Pole [:endpoint_on_np, :near_np2]
    Failure/Error: arc2.send(should_method, intersect(arc1))
    expected arc (-42.05,88.78 -> 128.32,86.04) to intersect arc (0.0,85.0 -> 0.0,90.0).
    Spatial Viz Link: http://localhost:4567/arc_intersection?arc1_ordinates=-42.05,88.78,128.32,86.04&arc2_ordinates=0.0,85.0,0.0,90.0
    (cmd-double click to view)
    # ./spec/models/arc_spec.rb:33:in `block in test_arc_intersection'
    Visualize your problems

    View full-size slide

  60. Will Reveal Themselves
    The Data Patterns

    View full-size slide

  61. Questions
    Visualizations available at
    http://testbed.echo.nasa.gov/spatial-viz
    @jasongilman [email protected]

    View full-size slide