Debugging Algorithms with Visualizations

This is a talk I gave recently at Data Science MD. The abstract: "We all build complicated applications that do amazing things. They juggle state for thousands of objects, perform complicated algorithms across multiple threads, and stay up for weeks or months at a time. We need to understand what's happening in our applications and in the terabytes of data our applications produce. Until now the primary means of communication of internal application state to developers has been symbolic; in gigabytes of logs, alert statements, stack traces, and emails. We can do better. Jason will argue the case that we should be building visualizations to debug our code."

This was inspired by the work of Bret Victor, Chris Granger, and others.

Transcript

1. Debugging Algorithms with Visualizations Jason Gilman - Element 84 - 4/8/2013 @jasongilman jason@element84.com
2. We've got a problem.
3. " What we've got here is failure to communicate " -Cool Hand Luke
4. Communication Developer Computer Code Output Focus of this talk Source: Heinz Nixdorf MuseumsForum
5. Developer Output Computer • API Responses • GUI Output • External Systems Changes • ie. rows inserted into tables Externally Visible • Logs • Stacktraces • Alerts • Console Messages • Debugger • REPL Internal State
6. Lightweight System.out.println("x = " + x); alert("I'm here"); puts "uh oh"
7. Deficiencies: Computer Complicated State Logs, stack traces, alerts Developer
8. Which of these is bigger? source: commons.wikimedia.org/wiki/File:Baseball_(crop).jpg
9. Which of these is bigger? 550 3
10. Which of these is bigger? 550mm 3m
11. 0 750 1500 2250 3000 550mm 3m 3000mm 550mm
12. 2012-09-08 20:08:07.430 Endless Space[863:fe0b] AUHAL::AUIOProc: mono buffer too small (1024 > 512) Steam_SetMinidumpSteamID: Caching Steam ID: 76561198025064504 [API loaded no] Endless Space(863,0xac65aa28) malloc: *** error for object 0x20afa84c: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug Endless Space(863,0xac65aa28) malloc: *** error for object 0x20afa94b: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug Endless Space(863,0xac65aa28) malloc: *** error for object 0x92d3552: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug Endless Space(863,0xac65aa28) malloc: *** error for object 0x20afa94b: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug Endless Space(863,0xac65aa28) malloc: *** error for object 0x9169d52: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug Endless Space(863,0xac65aa28) malloc: *** error for object 0x20afa94b: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug Lib.dll into Unity Child Domain Exception in thread "main" java.io.FileNotFoundException: Could not locate clojure/tools/nrepl/ack__init.class or clojure/tools/nrepl/ack.clj on classpath: (lein2.clj:1) at clojure.lang.Compiler.eval(Compiler.java:5440) at clojure.lang.Compiler.eval(Compiler.java:5415) at clojure.lang.Compiler.load(Compiler.java:5857) at clojure.lang.RT.loadResourceScript(RT.java:340) at clojure.lang.RT.loadResourceScript(RT.java:331) at clojure.lang.RT.load(RT.java:409) at clojure.lang.RT.load(RT.java:381) at clojure.core$load$fn__4519.invoke(core.clj:4915) at clojure.core$load.doInvoke(core.clj:4914) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.core$load_one.invoke(core.clj:4729) at clojure.core$load_lib.doInvoke(core.clj:4766) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invoke(core.clj:542) at clojure.core$load_libs.doInvoke(core.clj:4800) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invoke(core.clj:542) at clojure.core$require.doInvoke(core.clj:4881) at clojure.lang.RestFn.invoke(RestFn.java:408) at leinjacker.utils$try_resolve$fn__383.invoke(utils.clj:12) at leinjacker.utils$try_resolve.invoke(utils.clj:12) at clojure.lang.AFn.applyToHelper(AFn.java:161) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2906) at clojure.lang.Compiler$MapExpr.eval(Compiler.java:2472) at clojure.lang.Compiler$DefExpr.eval(Compiler.java:361) at clojure.lang.Compiler.eval(Compiler.java:5429) at clojure.lang.Compiler.load(Compiler.java:5857) at clojure.lang.RT.loadResourceScript(RT.java:340) at clojure.lang.RT.loadResourceScript(RT.java:331) at clojure.lang.RT.load(RT.java:409) at clojure.lang.RT.load(RT.java:381) at clojure.core$load$fn__4519.invoke(core.clj:4915) - Completed reload, in 0.075 seconds Switching to 800x600 fullscreen Platform assembly: /Users/mark/Library/Application Support/Steam/SteamApps/common/Endless Space/Endless Space.app/Contents/Data/Managed/System.dll (this message is harmless) Platform assembly: /Users/mark/Library/Application Support/Steam/SteamApps/common/Endless Space/Endless Space.app/Contents/Data/Managed/System.Xml.dll (this message is harmless) Platform assembly: /Users/mark/Library/Application Support/Steam/SteamApps/common/Endless Space/Endless Space.app/Contents/Data/Managed/UnityScript.Lang.dll (this message is harmless) Platform assembly: /Users/mark/Library/Application Support/Steam/SteamApps/common/Endless Space/Endless Space.app/Contents/Data/Managed/Boo.Lang.dll (this message is harmless) [Bootloader] version: 1.00.18 - 02 (Filename: /Applications/buildAgent/work/b0bcff80449a48aa/Runtime/ExportGenerated/MacStandalonePlayer/UnityEngineDebug.cpp Line: 43) [Bootloader] command line: /Users/mark/Library/Application Support/Steam/SteamApps/common/Endless Space/Endless Space.app/Contents/MacOS/Endless Space -single- instance -windowed -w 2560 -h 1440 (Filename: /Applications/buildAgent/work/b0bcff80449a48aa/Runtime/ExportGenerated/MacStandalonePlayer/UnityEngineDebug.cpp Line: 43) [Bootloader] GetCurrentGameLanguage: available game languages are '', current game languages is 'english'. What's the problem?
13. Concrete examples are much more useful in the real world
14. You're planning a wedding source: imdb.com
15. http://www.flickr.com/photos/gavinbell/81011618 We need some music
16. Is this a good choice?
17. A good cake for the reception? source: allrecipes.com/recipe/zuccotto-cupcakes
18. source: flickr.com/photos/louisephotography/8100744225/
19. source: google.com/trends/explore#q=ruby Data Visualization
20. But there's a piece missing source: commons.wikimedia.org/wiki/File:Wikt_puzzle_favicon.svg
21. 0 75 150 225 300 10 20 30 40 50 60 70 80 90 100 Run 1 Run 2 A Static Graph
22. Interactivity • Ask Questions • Explore Application State • Step Forward and Backward through algorithm • Show metadata
23. These aren't new ideas
24. forward 50 right 90 forward 50 right 90 forward 50 ... LOGO source: http://el.media.mit.edu/logo-foundation/logo/turtle.html
25. source: http://cyberneticzoo.com/?p=1711
26. source: http://cyberneticzoo.com/?p=1711 Not Just for Kids
27. "object-to-think-with"
28. Developer Logo Interpreter Surrogate
29. Drawing a House square triangle square right 30 triangle A Plan right 90 square right 30 triangle Success!
30. Example: Spherical Polygon Validation Valid Invalid
31. Spatial-calc Ruby Gem • Models • Point, Ring, Polygon, Arc, etc • polygon.valid? • arc1.intersects?(arc2)
32. Spatial-viz App • Includes spatial-calc gem • Sinatra • Slim, Sass, CoffeeScript • Google Earth Plugin • JS Events and interaction
33. Visualization #1 Arc Intersection Algorithm
36. Concrete Representation of spherical arcs
37. Draggable Points Visually confirm algorithm is working
38. Show Metadata Bounding Rectangle Great Circle Optional
39. 84) SpatialCalc::Arc.intersect Near North Pole2 should intersect Endpoint on North Pole [:endpoint_on_np, :near_np2] Failure/Error: arc2.send(should_method, intersect(arc1)) expected arc (-42.05,88.78 -> 128.32,86.04) to intersect arc (0.0,85.0 -> 0.0,90.0). Spatial Viz Link: http://localhost:4567/arc_intersection?arc1_ordinates=-42.05,88.78,128.32,86.04&arc2_ordinates=0.0,85.0,0.0,90.0 (cmd-double click to view) # ./spec/models/arc_spec.rb:33:in `block in test_arc_intersection' RSpec Failed spec Visualize Problem Link
40. Visualize Problem Then debug it
41. Visualization Driven Development
42. 1. Think • What's the problem? • How can I visualize it?
43. 2. Create a visualization
44. 3. Write some code
45. 4. Rinse and Repeat Viz Test Code
46. Visualizing an Algorithm
47. Polygon Problem: Does the polygon contain the North Pole?
48. • Loop over the arcs • Calculate heading • Find delta from previous heading • Sum the deltas My Algorithm
49. Looping over Arcs Visual Representation of Numeric Value
51. Visualizing Chaos
52. Visualizing Chaos
53. Show the Data source: http://hint.fm/wind/gallery/oct-30.js.html
54. Web App Search App Search Index
57. Before After
58. Skepticism
59. source: flickr.com/photos/epsos/6018530849 Woodworking Carpentry & Developing software is like
61. Jig source: flickr.com/photos/crown_molding/5079441077 wood to cut
62. Jig Benefits • Makes work easier (faster) • Higher quality results
63. Visualization Benefits • Makes work easier (faster) • Higher quality results
64. Build your own tools.
65. Interactive Visualizations
66. Abstract Becomes Concrete { lat1:89, lon1:88, lat2:5, lon2:37.3 }
67. 84) SpatialCalc::Arc.intersect Near North Pole2 should intersect Endpoint on North Pole [:endpoint_on_np, :near_np2] Failure/Error: arc2.send(should_method, intersect(arc1)) expected arc (-42.05,88.78 -> 128.32,86.04) to intersect arc (0.0,85.0 -> 0.0,90.0). Spatial Viz Link: http://localhost:4567/arc_intersection?arc1_ordinates=-42.05,88.78,128.32,86.04&arc2_ordinates=0.0,85.0,0.0,90.0 (cmd-double click to view) # ./spec/models/arc_spec.rb:33:in `block in test_arc_intersection' Visualize your problems
68. Will Reveal Themselves The Data Patterns
69. Questions Visualizations available at http://testbed.echo.nasa.gov/spatial-viz @jasongilman jason@element84.com