Principles of Expressive Machines

An introduction to my new research lab, presented to 1st year Ph.D. students.

Transcript

1. Dr. Chris Martens - Principles of Expressive Machines
   Principles of
   Expressive Machines
   Dr. Chris Martens
   CSC 600
   October 21, 2016
   1
   

   View Slide

2. Dr. Chris Martens - Principles of Expressive Machines
   Brief Autobio
   2
   NCSU Assistant Professor as of Fall 2016
   in the Digital Games Research Center
   Ph.D. Carnegie Mellon University Fall 2015
   Postdoc UC Santa Cruz, 2015-2016
   

   View Slide

3. Dr. Chris Martens - Principles of Expressive Machines
   3
   Digital Games
   Programming Languages
   Artificial Intelligence
   Human-computer interaction
   

   View Slide

4. Dr. Chris Martens - Principles of Expressive Machines
   Ultimate goal:
   4
   Bridge gaps between theory
   and practice
   for dynamic system modeling
   

   View Slide

5. Dr. Chris Martens - Principles of Expressive Machines
   5
   Secret weapon: declarative
   representations based on
   logic
   

   View Slide

6. Dr. Chris Martens - Principles of Expressive Machines
   6
   Talk Outline
   Intro
   Agenda 1: Narrative KR & generation
   Agenda 2: Game/Simulation Design Tools
   Agenda 3: Social Multi-Agent Systems
   Long-term Impact
   Working with Me
   

   View Slide

7. Dr. Chris Martens - Principles of Expressive Machines
   7
   Narrative KR & Generation
   

   View Slide

8. Dr. Chris Martens - Principles of Expressive Machines
   Leading Theory
   8
   Plot:
   sequence of events
   

   View Slide

9. Dr. Chris Martens - Principles of Expressive Machines
   9
   The Ant and the Dove
   

   View Slide

10. Dr. Chris Martens - Principles of Expressive Machines
    10
    The Ant and the Dove
    

    View Slide

11. Dr. Chris Martens - Principles of Expressive Machines
    Leading Theory
    11
    Plot:
    sequence causally-ordered DAG
    of events
    

    View Slide

12. Dr. Chris Martens - Principles of Expressive Machines
    The Ant and the Dove
    12
    …
    

    View Slide

13. Dr. Chris Martens - Principles of Expressive Machines
    The Ant and the Dove
    13
    …
    

    View Slide

14. Dr. Chris Martens - Principles of Expressive Machines
    14
    in(ant, water)
    in(leaf, water)
    sees(dove, ant)
    ant falls in water
    ant climbs onto leaf
    dove drops leaf
    

    View Slide

15. Dr. Chris Martens - Principles of Expressive Machines
    15
    in(ant, water)
    in(leaf, water)
    sees(dove, ant)
    STATES
    ant falls in water
    ant climbs onto leaf
    dove drops leaf
    

    View Slide

16. Dr. Chris Martens - Principles of Expressive Machines
    16
    in(ant, water)
    in(leaf, water)
    sees(dove, ant)
    ant falls in water
    ant climbs onto leaf
    EVENTS dove drops leaf
    

    View Slide

17. Dr. Chris Martens - Principles of Expressive Machines
    17
    Event: ant climbs onto leaf
    in(ant, water) in(leaf, water)
    on(ant, leaf)
    

    View Slide

18. Dr. Chris Martens - Principles of Expressive Machines
    18
    Generic Action: climbs onto
    in(Climber, Location) in(Support, Location)
    on(Climber, Support)
    climb_onto(Climber, Support)
    

    View Slide

19. Dr. Chris Martens - Principles of Expressive Machines
    19
    Generic Action: X climbs onto Y
    harms
    (Location, Climber)
    supports
    (Support, Climber)
    in(Climber, Location) in(Support, Location)
    on(Climber, Support)
    climb_onto(Climber, Support)
    NOT
    on(Climber, Support)
    

    View Slide

20. Dr. Chris Martens - Principles of Expressive Machines
    Generating Stories:
    The Simulationist Approach
    20
    1. Author a story-world as collection of
    operators like climb_onto
    2. Provide a starting configuration of
    states
    3. Run an automatic procedure to select
    operators
    

    View Slide

21. Dr. Chris Martens - Principles of Expressive Machines
    Selecting Operators
    21
    1. Based on Author Goals
    e.g.:
    safe(ant)
    safe(dove)
    

    View Slide

22. Dr. Chris Martens - Principles of Expressive Machines
    Selecting Operators
    22
    1. Based on Author Goals
    e.g.:
    safe(ant)
    on(ant, right_bank)
    safe(dove)
    

    View Slide

23. Dr. Chris Martens - Principles of Expressive Machines
    Selecting Operators
    23
    1. Based on Author Goals
    e.g.:
    safe(ant)
    on(ant, right_bank)
    safe(dove)
    bitten(birdcatcher)
    

    View Slide

24. Dr. Chris Martens - Principles of Expressive Machines
    Selecting Operators
    24
    2. Based on controlled randomness
    at(Agent, Place) near_water(Place)
    drinking(Agent, Place)
    

    View Slide

25. Dr. Chris Martens - Principles of Expressive Machines
    Selecting Operators
    25
    2. Based on controlled randomness
    at(Agent, Place) near_water(Place)
    drinking(Agent, Place)
    thirsty(Agent)
    not thirsty(Agent)
    

    View Slide

26. Dr. Chris Martens - Principles of Expressive Machines
    Generating Stories
    26
    Forward-chaining
    (character-directed)
    Backward-chaining
    (author goal-directed)
    Initial Goal
    …
    Initial
    

    View Slide

27. Dr. Chris Martens - Principles of Expressive Machines
    27
    Computer science tools:
    theorem proving
    planning
    linear logic
    multiset rewriting
    

    View Slide

28. Dr. Chris Martens - Principles of Expressive Machines
    28
    Codifying Actions in Linear Logic
    supports
    (Support, Climber)
    in(Climber, Location) in(Support, Location)
    on(Climber, Support)
    climb_onto(Climber, Support)
    climb_onto :
    in(C,L) * $in(S,L) * $supports(S, C)
    -o
    on(C,S).
    

    View Slide

29. Dr. Chris Martens - Principles of Expressive Machines
    29
    climb_onto :
    in(C,L) * $in(S,L) * $supports(S, C)
    -o
    on(C,S).
    C,S,L.
    ∀
    Theorem prover answers:
    ? Δ ⊢ G
    G goal
    Δ initial state
    Codifying Actions in Linear Logic
    

    View Slide

30. Dr. Chris Martens - Principles of Expressive Machines
    Ceptre
    30
    Linear logic-based action description language
    Currently only incorporates forward-chaining
    simulation
    Future work: integrate backward chaining
    

    View Slide

31. Dr. Chris Martens - Principles of Expressive Machines
    31
    Talk Outline
    Intro
    Agenda 1: Narrative KR & generation
    Agenda 2: Game/Simulation Design Tools
    Agenda 3: Social Multi-Agent Systems
    Long-term Impact
    Working with Me
    

    View Slide

32. Dr. Chris Martens - Principles of Expressive Machines
    32
    Game/Simulation
    DesignTools
    

    View Slide

33. Dr. Chris Martens - Principles of Expressive Machines
    33
    A nice thing about forward-chaining simulation models
    is that they offer a good starting point for interactivity.
    {at(dove, tree),
    at(ant, water),
    near(water, tree),
    at(leaf, tree),
    portable(leaf) }
    move : at(X, L) *
    near(L, L’)
    -o at(X, L’)
    take : at(X, L) *
    at(Thing, L) *
    portable(Thing)
    -o
    has(X, Thing)
    drop : at(X, L) *
    has(X, Thing)
    -o
    at(Thing, L)
    

    View Slide

34. Dr. Chris Martens - Principles of Expressive Machines
    34
    A nice thing about forward-chaining simulation models
    is that they offer a good starting point for interactivity.
    You are a dove in a tree near
    the water. There is a leaf
    here.
    > take leaf
    Taken.
    > go water
    You are at the water. There
    is an ant here.
    > drop leaf
    You drop the leaf in the
    water. The ant climbs on.
    

    View Slide

35. Dr. Chris Martens - Principles of Expressive Machines
    35
    Interactive Fiction
    

    View Slide

36. Dr. Chris Martens - Principles of Expressive Machines
    36
    
    “take lamp”
    “read book”
    
    “put chalice on pedestal”
    “connect magnet to string”
    

    View Slide

37. Dr. Chris Martens - Principles of Expressive Machines
    37
    Inventory systems in adventure games
    

    View Slide

38. Dr. Chris Martens - Principles of Expressive Machines
    38
    Combinatorial Interfaces
    

    View Slide

39. Dr. Chris Martens - Principles of Expressive Machines
    39
    Inform 7
    

    View Slide

40. Dr. Chris Martens - Principles of Expressive Machines
    40
    Authoring Open Worlds
    - Interface: typed text? selected options?
    - Responses to commands:
    - agency vs. author burden
    

    View Slide

41. Dr. Chris Martens - Principles of Expressive Machines
    41
    Property-based simulation
    If something (X) hard hits something (Y) brittle,
    then Y breaks
    When something brittle breaks, the pieces are sharp
    Sharp things can break soft things
    Rope is soft, Glass is brittle
    ==>
    Player with a rock and rope in a room with a window can
    formulate a plan to break the glass and cut the rope.
    

    View Slide

42. Dr. Chris Martens - Principles of Expressive Machines
    Generative Methods
    42
    Processes for recombining elements of an input corpus to
    create new content without the aid of human designers.
    e.g.: Procedural Content Generation (PCG)
    

    View Slide

43. Dr. Chris Martens - Principles of Expressive Machines
    Generative Methods
    43
    Procedurally generated game levels (Spelunky)
    

    View Slide

44. Dr. Chris Martens - Principles of Expressive Machines
    Generative Methods
    44
    Autonomous NPC behaviors
    

    View Slide

45. Dr. Chris Martens - Principles of Expressive Machines
    Generative Methods
    45
    Generated game rules and mechanics
    

    View Slide

46. Dr. Chris Martens - Principles of Expressive Machines
    ASP (Answer Set Programming)
    46
    1. Specify possible ways of constructing an artifact
    2. Write rules that designate certain constructions as
    forbidden
    3. Run an answer set solver to get satisfying examples
    

    View Slide

47. Dr. Chris Martens - Principles of Expressive Machines
    Future Projects
    47
    Develop tools that combine the simulation capabilities of
    Ceptre with the generative and constraint-driven
    expressiveness of ASP
    

    View Slide

48. Dr. Chris Martens - Principles of Expressive Machines
    Future Projects
    48
    Develop tools that combine the simulation capabilities of
    Ceptre with the generative and constraint-driven
    expressiveness of ASP
    Create better ways for generative processes and (multiple)
    humans to collaborate with one another
    

    View Slide

49. Dr. Chris Martens - Principles of Expressive Machines
    49
    Talk Outline
    Intro
    Agenda 1: Narrative KR & generation
    Agenda 2: Game/Simulation Design Tools
    Agenda 3: Social Multi-Agent Systems
    Long-term Impact
    Working with Me
    

    View Slide

50. Dr. Chris Martens - Principles of Expressive Machines
    50
    Social Multi-Agent Systems
    

    View Slide

51. Dr. Chris Martens - Principles of Expressive Machines
    Example: Pass a handout to
    everyone in a classroom
    51
    Activity: enumerate some algorithms.
    

    View Slide

52. Dr. Chris Martens - Principles of Expressive Machines
    Example: Pass a handout to
    everyone in a classroom
    52
    1. Instructor gives each student a handout
    individually.
    2. Instructor hands stack to one student and
    says to “take one and pass it down”
    3. Instructor divides the stack to hand to
    each row of seats
    

    View Slide

53. Dr. Chris Martens - Principles of Expressive Machines
    53
    handout :
    stack(P::Ps) * no_paper(Student)
    -o
    has(Student, P).
    t1pd :
    stack_at(Student, P::Ps) *
    neighbor(Student, Student’) *
    no_paper(Student’)
    -o
    has(Student, P) *
    stack_at(Student’, Ps).
    

    View Slide

54. Dr. Chris Martens - Principles of Expressive Machines
    54
    Correctness property:
    At quiescence, all students should have one paper.
    Δ ⊢
    has(student1, P1) * … * has(studentn, Pn)
    Research Problem:
    automatically derive constraints on initial conditions that
    ensure this property.
    e.g.: stack size >= number of students
    

    View Slide

55. Dr. Chris Martens - Principles of Expressive Machines
    55
    Correctness property:
    At quiescence, all students should have one paper.
    Δ ⊢
    has(student1, P1) * … * has(studentn, Pn)
    Research Problem:
    verify that the centralized and distributed versions of this
    algorithm are equivalent in behavior, while the distributed
    version has more locality.
    

    View Slide

56. Dr. Chris Martens - Principles of Expressive Machines
    Stories as dynamic systems
    56
    Independent plans per character
    

    View Slide

57. Dr. Chris Martens - Principles of Expressive Machines
    Selecting Operators
    57
    3. Based on character goals
    e.g.:
    ant: safe(ant)
    ant: not_thirsty(ant)
    dove: safe(dove)
    birdwatcher: caught(dove)
    

    View Slide

58. Dr. Chris Martens - Principles of Expressive Machines
    Other Multi-Agent Systems:
    58
    n-way conversation
    Particle systems
    Self-organizing robot swarms
    Multi-player games
    Cities
    Voting protocols
    Political processes
    

    View Slide

59. Dr. Chris Martens - Principles of Expressive Machines
    Ongoing project
    (with Markus Eger):
    representing asymmetric
    information in games
    with epistemic modal logic
    59
    Three logicians walk into a bar…
    

    View Slide

60. Dr. Chris Martens - Principles of Expressive Machines
    Ongoing project
    (with Hannah Morrison):
    modeling n agents in deliberative
    conversation with each other
    how emotion, relationships, and
    personality influence the course of
    conversation, e.g. belief change
    60
    

    View Slide

61. Dr. Chris Martens - Principles of Expressive Machines
    61
    Talk Outline
    Intro
    Agenda 1: Narrative KR & generation
    Agenda 2: Game/Simulation Design Tools
    Agenda 3: Social Multi-Agent Systems
    Long-term Impact
    Working with Me
    

    View Slide

62. Dr. Chris Martens - Principles of Expressive Machines
    62
    Long-term Impact
    

    View Slide

63. Dr. Chris Martens - Principles of Expressive Machines
    63
    Interdisciplinary Connections
    Example: logic and functional programming
    

    View Slide

64. Dr. Chris Martens - Principles of Expressive Machines
    64
    Interdisciplinary Connections
    Structure in music and narrative
    

    View Slide

65. Dr. Chris Martens - Principles of Expressive Machines
    65
    common languages as extensible targets
    for numerous compilation sources
    Ceptre
    Rules
    Mapping from
    predicates to
    rendering
    instructions
    Extensible
    front-end
    extn
    extn
    extn
    

    View Slide

66. Dr. Chris Martens - Principles of Expressive Machines
    66
    Accessible Computing
    

    View Slide

67. Dr. Chris Martens - Principles of Expressive Machines
    67
    Accessible Computing
    

    View Slide

68. Dr. Chris Martens - Principles of Expressive Machines
    68
    Radically transform what programming
    looks like
    and who is doing it
    

    View Slide

69. Dr. Chris Martens - Principles of Expressive Machines
    69
    Secret weapon: declarative
    representations based on
    logic
    

    View Slide

70. Dr. Chris Martens - Principles of Expressive Machines
    Ultimate goal:
    70
    Bridge gaps between theory
    and practice
    for dynamic system modeling
    

    View Slide

71. Dr. Chris Martens - Principles of Expressive Machines
    71
    Talk Outline
    Intro
    Agenda 1: Game/IF Design Tools
    Agenda 2: Narrative KR & generation
    Agenda 3: Social Multi-Agent Systems
    Mentorship approach
    

    View Slide

72. Dr. Chris Martens - Principles of Expressive Machines
    Working with Me
    72
    Games & Visual Narrative Suite in Venture II
    Neighbors:
    Dave Roberts
    Arnav Jhala
    The Visual Narrative Cluster
    

    View Slide

73. Dr. Chris Martens - Principles of Expressive Machines
    Working with Me
    73
    Day-to-day work (possibilities):
    Building systems (prefer Haskell, OCaml, JavaScript)
    Building models (Ceptre, ASP, Prolog)
    Designing languages and logics
    Writing proofs (paper, Coq, Agda, Twelf)
    Building games (browser, Unity, analog, live-action)
    

    View Slide

74. Dr. Chris Martens - Principles of Expressive Machines
    Working with Me
    74
    Venues I like:
    POPL - Principles of Programming Languages
    AIIDE - AI in Games
    FDG - Foundations of Digital Games
    ICIDS - Interactive Digital Storytelling
    

    View Slide

75. Dr. Chris Martens - Principles of Expressive Machines
    Advising Style
    75
    1st half of Ph.D.:
    Weekly meetings
    Structured semester planning and check-ins
    Task assignments if needed
    Always Be Writing
    

    View Slide

76. Dr. Chris Martens - Principles of Expressive Machines
    Advising Style
    76
    2nd half of Ph.D.:
    Weekly meetings, or as-needed
    Devise your own plans
    Teach me things!
    Always Be Writing
    

    View Slide

77. Dr. Chris Martens - Principles of Expressive Machines
    Get in touch
    77
    Dr. Chris Martens
    martens@csc.ncsu.edu
    http://go.ncsu.edu/poem
    

    View Slide