Principles of Expressive Machines

An introduction to my new research lab, presented to 1st year Ph.D. students.

Transcript

1. Dr. Chris Martens - Principles of Expressive Machines Principles of Expressive Machines Dr. Chris Martens CSC 600 October 21, 2016 1
2. Dr. Chris Martens - Principles of Expressive Machines Brief Autobio 2 NCSU Assistant Professor as of Fall 2016 in the Digital Games Research Center Ph.D. Carnegie Mellon University Fall 2015 Postdoc UC Santa Cruz, 2015-2016
3. Dr. Chris Martens - Principles of Expressive Machines 3 Digital Games Programming Languages Artificial Intelligence Human-computer interaction
4. Dr. Chris Martens - Principles of Expressive Machines Ultimate goal: 4 Bridge gaps between theory and practice for dynamic system modeling
5. Dr. Chris Martens - Principles of Expressive Machines 5 Secret weapon: declarative representations based on logic
6. Dr. Chris Martens - Principles of Expressive Machines 6 Talk Outline Intro Agenda 1: Narrative KR & generation Agenda 2: Game/Simulation Design Tools Agenda 3: Social Multi-Agent Systems Long-term Impact Working with Me
7. Dr. Chris Martens - Principles of Expressive Machines 7 Narrative KR & Generation
8. Dr. Chris Martens - Principles of Expressive Machines Leading Theory 8 Plot: sequence of events
9. Dr. Chris Martens - Principles of Expressive Machines 9 The Ant and the Dove
10. Dr. Chris Martens - Principles of Expressive Machines 10 The Ant and the Dove
11. Dr. Chris Martens - Principles of Expressive Machines Leading Theory 11 Plot: sequence causally-ordered DAG of events
12. Dr. Chris Martens - Principles of Expressive Machines The Ant and the Dove 12 …
13. Dr. Chris Martens - Principles of Expressive Machines The Ant and the Dove 13 …
14. Dr. Chris Martens - Principles of Expressive Machines 14 in(ant, water) in(leaf, water) sees(dove, ant) ant falls in water ant climbs onto leaf dove drops leaf
15. Dr. Chris Martens - Principles of Expressive Machines 15 in(ant, water) in(leaf, water) sees(dove, ant) STATES ant falls in water ant climbs onto leaf dove drops leaf
16. Dr. Chris Martens - Principles of Expressive Machines 16 in(ant, water) in(leaf, water) sees(dove, ant) ant falls in water ant climbs onto leaf EVENTS dove drops leaf
17. Dr. Chris Martens - Principles of Expressive Machines 17 Event: ant climbs onto leaf in(ant, water) in(leaf, water) on(ant, leaf)
18. Dr. Chris Martens - Principles of Expressive Machines 18 Generic Action: <climber> climbs onto <support> in(Climber, Location) in(Support, Location) on(Climber, Support) climb_onto(Climber, Support)
19. Dr. Chris Martens - Principles of Expressive Machines 19 Generic Action: X climbs onto Y harms (Location, Climber) supports (Support, Climber) in(Climber, Location) in(Support, Location) on(Climber, Support) climb_onto(Climber, Support) NOT on(Climber, Support)
20. Dr. Chris Martens - Principles of Expressive Machines Generating Stories: The Simulationist Approach 20 1. Author a story-world as collection of operators like climb_onto 2. Provide a starting configuration of states 3. Run an automatic procedure to select operators
21. Dr. Chris Martens - Principles of Expressive Machines Selecting Operators 21 1. Based on Author Goals e.g.: safe(ant) safe(dove)
22. Dr. Chris Martens - Principles of Expressive Machines Selecting Operators 22 1. Based on Author Goals e.g.: safe(ant) on(ant, right_bank) safe(dove)
23. Dr. Chris Martens - Principles of Expressive Machines Selecting Operators 23 1. Based on Author Goals e.g.: safe(ant) on(ant, right_bank) safe(dove) bitten(birdcatcher)
24. Dr. Chris Martens - Principles of Expressive Machines Selecting Operators 24 2. Based on controlled randomness at(Agent, Place) near_water(Place) drinking(Agent, Place)
25. Dr. Chris Martens - Principles of Expressive Machines Selecting Operators 25 2. Based on controlled randomness at(Agent, Place) near_water(Place) drinking(Agent, Place) thirsty(Agent) not thirsty(Agent)
26. Dr. Chris Martens - Principles of Expressive Machines Generating Stories 26 Forward-chaining (character-directed) Backward-chaining (author goal-directed) Initial Goal … Initial
27. Dr. Chris Martens - Principles of Expressive Machines 27 Computer science tools: theorem proving planning linear logic multiset rewriting
28. Dr. Chris Martens - Principles of Expressive Machines 28 Codifying Actions in Linear Logic supports (Support, Climber) in(Climber, Location) in(Support, Location) on(Climber, Support) climb_onto(Climber, Support) climb_onto : in(C,L) * $in(S,L) * $supports(S, C) -o on(C,S).
29. Dr. Chris Martens - Principles of Expressive Machines 29 climb_onto : in(C,L) * $in(S,L) * $supports(S, C) -o on(C,S). C,S,L. ∀ Theorem prover answers: ? Δ ⊢ G G goal Δ initial state Codifying Actions in Linear Logic
30. Dr. Chris Martens - Principles of Expressive Machines Ceptre 30 Linear logic-based action description language Currently only incorporates forward-chaining simulation Future work: integrate backward chaining
31. Dr. Chris Martens - Principles of Expressive Machines 31 Talk Outline Intro Agenda 1: Narrative KR & generation Agenda 2: Game/Simulation Design Tools Agenda 3: Social Multi-Agent Systems Long-term Impact Working with Me
32. Dr. Chris Martens - Principles of Expressive Machines 32 Game/Simulation DesignTools
33. Dr. Chris Martens - Principles of Expressive Machines 33 A nice thing about forward-chaining simulation models is that they offer a good starting point for interactivity. {at(dove, tree), at(ant, water), near(water, tree), at(leaf, tree), portable(leaf) } move : at(X, L) * near(L, L’) -o at(X, L’) take : at(X, L) * at(Thing, L) * portable(Thing) -o has(X, Thing) drop : at(X, L) * has(X, Thing) -o at(Thing, L)
34. Dr. Chris Martens - Principles of Expressive Machines 34 A nice thing about forward-chaining simulation models is that they offer a good starting point for interactivity. You are a dove in a tree near the water. There is a leaf here. > take leaf Taken. > go water You are at the water. There is an ant here. > drop leaf You drop the leaf in the water. The ant climbs on.
35. Dr. Chris Martens - Principles of Expressive Machines 35 Interactive Fiction
36. Dr. Chris Martens - Principles of Expressive Machines 36 <verb> <noun> “take lamp” “read book” <verb> <noun> <preposition> <noun> “put chalice on pedestal” “connect magnet to string”
37. Dr. Chris Martens - Principles of Expressive Machines 37 Inventory systems in adventure games
38. Dr. Chris Martens - Principles of Expressive Machines 38 Combinatorial Interfaces
39. Dr. Chris Martens - Principles of Expressive Machines 39 Inform 7
40. Dr. Chris Martens - Principles of Expressive Machines 40 Authoring Open Worlds - Interface: typed text? selected options? - Responses to commands: - agency vs. author burden
41. Dr. Chris Martens - Principles of Expressive Machines 41 Property-based simulation If something (X) hard hits something (Y) brittle, then Y breaks When something brittle breaks, the pieces are sharp Sharp things can break soft things Rope is soft, Glass is brittle ==> Player with a rock and rope in a room with a window can formulate a plan to break the glass and cut the rope.
42. Dr. Chris Martens - Principles of Expressive Machines Generative Methods 42 Processes for recombining elements of an input corpus to create new content without the aid of human designers. e.g.: Procedural Content Generation (PCG)
43. Dr. Chris Martens - Principles of Expressive Machines Generative Methods 43 Procedurally generated game levels (Spelunky)
44. Dr. Chris Martens - Principles of Expressive Machines Generative Methods 44 Autonomous NPC behaviors
45. Dr. Chris Martens - Principles of Expressive Machines Generative Methods 45 Generated game rules and mechanics
46. Dr. Chris Martens - Principles of Expressive Machines ASP (Answer Set Programming) 46 1. Specify possible ways of constructing an artifact 2. Write rules that designate certain constructions as forbidden 3. Run an answer set solver to get satisfying examples
47. Dr. Chris Martens - Principles of Expressive Machines Future Projects 47 Develop tools that combine the simulation capabilities of Ceptre with the generative and constraint-driven expressiveness of ASP
48. Dr. Chris Martens - Principles of Expressive Machines Future Projects 48 Develop tools that combine the simulation capabilities of Ceptre with the generative and constraint-driven expressiveness of ASP Create better ways for generative processes and (multiple) humans to collaborate with one another
49. Dr. Chris Martens - Principles of Expressive Machines 49 Talk Outline Intro Agenda 1: Narrative KR & generation Agenda 2: Game/Simulation Design Tools Agenda 3: Social Multi-Agent Systems Long-term Impact Working with Me
50. Dr. Chris Martens - Principles of Expressive Machines 50 Social Multi-Agent Systems
51. Dr. Chris Martens - Principles of Expressive Machines Example: Pass a handout to everyone in a classroom 51 Activity: enumerate some algorithms.
52. Dr. Chris Martens - Principles of Expressive Machines Example: Pass a handout to everyone in a classroom 52 1. Instructor gives each student a handout individually. 2. Instructor hands stack to one student and says to “take one and pass it down” 3. Instructor divides the stack to hand to each row of seats
53. Dr. Chris Martens - Principles of Expressive Machines 53 handout : stack(P::Ps) * no_paper(Student) -o has(Student, P). t1pd : stack_at(Student, P::Ps) * neighbor(Student, Student’) * no_paper(Student’) -o has(Student, P) * stack_at(Student’, Ps).
54. Dr. Chris Martens - Principles of Expressive Machines 54 Correctness property: At quiescence, all students should have one paper. Δ ⊢ has(student1, P1) * … * has(studentn, Pn) Research Problem: automatically derive constraints on initial conditions that ensure this property. e.g.: stack size >= number of students
55. Dr. Chris Martens - Principles of Expressive Machines 55 Correctness property: At quiescence, all students should have one paper. Δ ⊢ has(student1, P1) * … * has(studentn, Pn) Research Problem: verify that the centralized and distributed versions of this algorithm are equivalent in behavior, while the distributed version has more locality.
56. Dr. Chris Martens - Principles of Expressive Machines Stories as dynamic systems 56 Independent plans per character
57. Dr. Chris Martens - Principles of Expressive Machines Selecting Operators 57 3. Based on character goals e.g.: ant: safe(ant) ant: not_thirsty(ant) dove: safe(dove) birdwatcher: caught(dove)
58. Dr. Chris Martens - Principles of Expressive Machines Other Multi-Agent Systems: 58 n-way conversation Particle systems Self-organizing robot swarms Multi-player games Cities Voting protocols Political processes
59. Dr. Chris Martens - Principles of Expressive Machines Ongoing project (with Markus Eger): representing asymmetric information in games with epistemic modal logic 59 Three logicians walk into a bar…
60. Dr. Chris Martens - Principles of Expressive Machines Ongoing project (with Hannah Morrison): modeling n agents in deliberative conversation with each other how emotion, relationships, and personality influence the course of conversation, e.g. belief change 60
61. Dr. Chris Martens - Principles of Expressive Machines 61 Talk Outline Intro Agenda 1: Narrative KR & generation Agenda 2: Game/Simulation Design Tools Agenda 3: Social Multi-Agent Systems Long-term Impact Working with Me
62. Dr. Chris Martens - Principles of Expressive Machines 62 Long-term Impact
63. Dr. Chris Martens - Principles of Expressive Machines 63 Interdisciplinary Connections Example: logic and functional programming
64. Dr. Chris Martens - Principles of Expressive Machines 64 Interdisciplinary Connections Structure in music and narrative
65. Dr. Chris Martens - Principles of Expressive Machines 65 common languages as extensible targets for numerous compilation sources Ceptre Rules Mapping from predicates to rendering instructions Extensible front-end extn extn extn
66. Dr. Chris Martens - Principles of Expressive Machines 66 Accessible Computing
67. Dr. Chris Martens - Principles of Expressive Machines 67 Accessible Computing
68. Dr. Chris Martens - Principles of Expressive Machines 68 Radically transform what programming looks like and who is doing it
69. Dr. Chris Martens - Principles of Expressive Machines 69 Secret weapon: declarative representations based on logic
70. Dr. Chris Martens - Principles of Expressive Machines Ultimate goal: 70 Bridge gaps between theory and practice for dynamic system modeling
71. Dr. Chris Martens - Principles of Expressive Machines 71 Talk Outline Intro Agenda 1: Game/IF Design Tools Agenda 2: Narrative KR & generation Agenda 3: Social Multi-Agent Systems Mentorship approach
72. Dr. Chris Martens - Principles of Expressive Machines Working with Me 72 Games & Visual Narrative Suite in Venture II Neighbors: Dave Roberts Arnav Jhala The Visual Narrative Cluster
73. Dr. Chris Martens - Principles of Expressive Machines Working with Me 73 Day-to-day work (possibilities): Building systems (prefer Haskell, OCaml, JavaScript) Building models (Ceptre, ASP, Prolog) Designing languages and logics Writing proofs (paper, Coq, Agda, Twelf) Building games (browser, Unity, analog, live-action)
74. Dr. Chris Martens - Principles of Expressive Machines Working with Me 74 Venues I like: POPL - Principles of Programming Languages AIIDE - AI in Games FDG - Foundations of Digital Games ICIDS - Interactive Digital Storytelling
75. Dr. Chris Martens - Principles of Expressive Machines Advising Style 75 1st half of Ph.D.: Weekly meetings Structured semester planning and check-ins Task assignments if needed Always Be Writing
76. Dr. Chris Martens - Principles of Expressive Machines Advising Style 76 2nd half of Ph.D.: Weekly meetings, or as-needed Devise your own plans Teach me things! Always Be Writing
77. Dr. Chris Martens - Principles of Expressive Machines Get in touch 77 Dr. Chris Martens martens@csc.ncsu.edu http://go.ncsu.edu/poem