Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to think like a system: Puzzles, Problems, and Programs

Chris
September 28, 2018

How to think like a system: Puzzles, Problems, and Programs

Strange Loop 2018 talk.

Chris

September 28, 2018
Tweet

More Decks by Chris

Other Decks in Technology

Transcript

  1. How to think like a system:
    Puzzles, Problems, Programs
    Chris Martens
    North Carolina State University
    @ Strange Loop 2018
    1

    View Slide

  2. Bongard Problems
    2

    View Slide

  3. Bongard Problems
    3
    Triangles vs. Quadrilaterals

    View Slide

  4. Bongard Problems
    4

    View Slide

  5. Bongard Problems
    5
    Unfilled shape overlaps filled vs. vice versa

    View Slide

  6. Bongard Problems
    6

    View Slide

  7. Bongard Problems
    7
    Ask a neighbor!

    View Slide

  8. Bongard Problems
    8
    Harry Foundalis - index and disseration
    http://www.foundalis.com/res/bps/bpidx.htm

    View Slide

  9. 9
    About Me
    •Assistant Professor, Computer Science @ NC State
    •Director, Principles of Expressive Machines (POEM) Lab
    •Strange Loop #3
    •I <3 functional programming, logic, proofs, etc.
    •Research: tools & languages for interactive narrative,
    game AI, procedural generation, system modeling

    View Slide

  10. 10
    Lately I’ve been thinking about
    how things like this relate to things like this
    (Experiential characteristics) (Formal descriptions)

    View Slide

  11. 11
    human-system interaction

    View Slide

  12. Normal problems: apply rules to new cases
    Bongard problems: study cases to learn the rule

    View Slide

  13. Using Examples to Learn Rules
    Things humans do:
    ● Hypothesis-driven science
    ● Test-driven development
    Things computers do:
    ● Example-driven program synthesis
    ● Data-trained classifiers

    View Slide

  14. How should we design the tools (game levels, code
    editors, interfaces, programming languages) necessary
    for humans and computers to do these things together?

    View Slide

  15. Outline
    •Mental model alignment theory
    •Case Studies
    •Puzzles
    •Problems
    •Programs
    •Tools for Thought
    15

    View Slide

  16. Outline
    16
    •Mental model alignment theory
    •Case Studies
    •Puzzles
    •Problems
    •Programs
    •Tools for Thought

    View Slide

  17. Mental Models
    17
    Predict
    Explain
    1 - P.N. Johnson-Laird. Mental Models. Cambridge University Press, Cambridge, 1983.
    1

    View Slide

  18. Mental Model Alignment Theory
    18
    Press right, Mario goes right

    Press A, Mario jumps
    YouTube “Extra Credits,” episode “Design Club - Super Mario
    Bros: Level 1-1 - How Super Mario Mastered Level Design.”
    https://www.youtube.com/watch?v=ZH2wGpEZVgE

    View Slide

  19. Mental Model Alignment Theory
    19
    Press right, Mario goes right

    Press A, Mario jumps

    Collide w/this thing ???

    View Slide

  20. Mental Model Alignment Theory
    20
    Press right, Mario goes right

    Press A, Mario jumps

    Collide w/this thing, die

    View Slide

  21. Generalization
    21
    Press right, Mario goes right

    Press A, Mario jumps

    Collide w/mushroom shape, die

    View Slide

  22. Application
    22
    Press right, Mario goes right

    Press A, Mario jumps

    Collide w/mushroom, die?

    View Slide

  23. Application
    23
    Press right, Mario goes right

    Press A, Mario jumps

    Collide w/mushroom, die?

    View Slide

  24. Revision
    24
    Press right, Mario goes right

    Press A, Mario jumps

    Collide w/angry mushroom, die
    Collide w/mushroom, get big

    View Slide

  25. Mental Model Alignment Theory
    In sum:
    *Mental models simulate an external reality
    *They make predictions and explanations
    *They generalize from examples
    *They revise upon failure

    View Slide

  26. Failing Means Learning
    26
    From the game Celeste released in 2018

    View Slide

  27. Outline
    27
    •Mental model alignment theory
    •Case Studies
    •Puzzles
    •Problems
    •Programs
    •Tools for Thought

    View Slide

  28. What makes a good puzzle?
    28
    Mark Brown (YouTube video of same title):
    0. Clever mechanics

    View Slide

  29. Cosmic Express
    29
    Draknek, 2017

    View Slide

  30. 0. Clever Mechanics
    30
    “Iron-clad rules & limitations that never change”
    In Cosmic Express:
    * Tracks can’t cross over one another
    * Each passenger must be taken to a matching box
    * Each train car can hold 1 passenger
    * Passengers will try to hop into any empty car that passes
    them

    View Slide

  31. 31
    …with interesting, gradually-revealed consequences

    View Slide

  32. What makes a good puzzle?
    32
    1. Assumption
    2. Catch
    3. Revelation
    Building on a solid foundation of mechanics:

    View Slide

  33. 1. Assumption
    33
    Each passenger should be delivered to the box nearest to it

    View Slide

  34. 2. Catch
    34
    Drawing routes to pick up passengers block access to boxes
    (and vice versa)

    View Slide

  35. 3. Revelation
    35
    Visit passengers in a different order from the boxes

    View Slide

  36. Research Question
    36
    Does the assumption-catch-revelation pattern support faster or
    more accurate mental model alignment?

    View Slide

  37. Laserverse
    37
    A POEM Lab original
    (Summer 2018)

    View Slide

  38. “Feedback”
    38

    View Slide

  39. Conceptual Dependencies
    39

    View Slide

  40. “Beamlock” (assumption/catch)
    40

    View Slide

  41. “Beamlock” (revelation)
    41

    View Slide

  42. Studying Mental Model Formation
    42
    Predict Explain
    Play

    View Slide

  43. Research Goal
    43
    Use what we learn about mental model formation to
    procedurally generate puzzle sequences that
    support deep mastery

    View Slide

  44. Outline
    44
    •Mental model alignment theory
    •Case Studies
    •Puzzles
    •Problems
    •Programs
    •Tools for Thought

    View Slide

  45. A certain class of puzzles?
    45
    Infinifactory (Zachtronics 2015), Factorio (Wube 2012)

    View Slide

  46. (a personal favorite)
    46
    The Incredible Machine (Dynamix, 1993)

    View Slide

  47. Problem Solving Games
    47
    Mark Brown, YouTube: “Puzzle Solving… or Problem Solving?”:
    DISCOVER THE SOLUTION INVENT A SOLUTION
    vs

    View Slide

  48. 48
    Redstone in Minecraft (Mojang 2011)

    View Slide

  49. 49
    Literally writing code in TIS-100 (Zachtronics 2015)

    View Slide

  50. 50
    Literally teaching code in BOTS (Game2Learn lab, 2015)

    View Slide

  51. 51
    Teaching functional idioms in Cube Composer (David Peter)

    View Slide

  52. Outline
    52
    •Mental model alignment theory
    •Case Studies
    •Puzzles
    •Problems
    •Programs
    •Tools for Thought

    View Slide

  53. Puzzles: small, intentionally-crafted solution space
    Designed Problems: bigger solution space, may be
    co-designed with problem
    Programming: big solution space, problem
    precedes solution
    53

    View Slide

  54. Mechanics
    54
    Programming Model/Language

    View Slide

  55. Mechanics
    Puzzles
    55
    Programming Model/Language
    Programming Problems

    View Slide

  56. Mechanics
    Puzzles
    Puzzle Solutions
    56
    Programming Model/Language
    Programming Problems
    Programs

    View Slide

  57. Learning rules from examples
    57
    Designing abstractions

    View Slide

  58. Learning rules from examples
    58
    Designing abstractions

    View Slide

  59. How learners develop mental models of the
    programming model/language:
    lots of prior research
    59

    View Slide

  60. Most of us are coding with an incomplete mental
    model of our programming language
    (see: Gary Bernhardt’s WAT talk)
    60
    https://www.destroyallsoftware.com/talks/wat

    View Slide

  61. Most of us are coding with an incomplete mental
    model of our programming language

    but most of our bugs come from poor mental
    models of our domains
    61

    View Slide

  62. Outline
    62
    •Mental model alignment theory
    •Case Studies
    •Puzzles
    •Problems
    •Programs
    •Tools for Thought

    View Slide

  63. Tools for communicating our
    mental models to a system
    63

    View Slide

  64. Writing Tests
    count_os(“strangeloooop”) = 4
    count_os(“object oriented”) = 2
    count_os(“functional”) = 1
    64
    Each test is a communication of our mental
    model to the program

    View Slide

  65. count_os(“strangeloooop”) = 4
    count_os(“object oriented”) = 2
    count_os(“functional”) = 1
    count_os(“”) = 1337
    65
    We write tests because we want failure to
    happen early!

    View Slide

  66. Type annotations: another way to express our
    mental models
    count_os : string -> int
    66

    View Slide

  67. Checkable specs (or fancier types) = more precise
    communication of mental models
    count_os :
    s:string -> n:int | 0 <= n <= len(s)
    67

    View Slide

  68. 68
    Solver-aided tools:
    directly run our mental models as programs?

    View Slide

  69. Program Synthesis
    69

    View Slide

  70. Procedural Generation
    70

    View Slide

  71. Solver-aided tools:
    directly run our mental models as programs?
    71
    …but how do we form correct mental models of the solution
    space, navigate it, and change it?

    View Slide

  72. Cognitive Artifacts
    72
    David Krakauer
    Competitive: enhance our abilities when we have the
    artifact, but leave us just as powerless as we were
    before using it when we stop using it.
    Complementary: give us new abilities that
    we can internalize and use when the artifact
    is discarded.

    View Slide

  73. Cognitive Artifacts
    73
    GPS navigation: as a driver vs. as a passenger

    View Slide

  74. Cognitive Artifacts
    74
    Changing our relationships with the systems we’re a part of

    View Slide

  75. Takeaways
    75
    1. Automated systems that develop mental models are
    more powerful than ones that make them obsolete
    2. More research is needed to understand how!

    View Slide

  76. Collaborate!
    76
    https://go.ncsu.edu/poem
    * Apply for a Ph.D. in Computer Science at NC
    State! Deadline December 15
    * Faculty/industry collaborators welcome!

    View Slide

  77. Chris Martens
    go.ncsu.edu/martens
    [email protected]
    Twitter: @chrisamaphone
    77

    View Slide

  78. Extra Slides
    78

    View Slide

  79. Neural Networks
    79
    Pile of annotated
    data
    New example
    New annotation!
    MODEL

    View Slide

  80. Failing is Learning
    80
    Type Errors

    View Slide

  81. Zendo
    81
    (Kory Heath, 1999)
    Follows the rule Doesn’t follow the rule

    View Slide

  82. Zendo
    82
    (Kory Heath, 1999)
    Guesser: Does this example follow the rule?

    View Slide

  83. Zendo
    83
    (Kory Heath, 1999)
    Rulemaker: no.

    View Slide

  84. Zendo
    84
    (Kory Heath, 1999)
    Guesser: is the rule that there must not be a pentagon?
    Follows the rule Doesn’t follow the rule

    View Slide

  85. Zendo
    85
    (Kory Heath, 1999)
    Rulemaker: no. Counterexample:
    Doesn’t follow the proposed rule, follows my rule

    View Slide

  86. Zendo
    86
    (Kory Heath, 1999)

    View Slide