Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Haskell SpriteKit — A Purely Functional API for a Stateful Animation System & Physics Engine

Haskell SpriteKit — A Purely Functional API for a Stateful Animation System & Physics Engine

VIDEOS:
* https://www.youtube.com/watch?v=kd8mlbN0Mws (Functional Conf 2017, best choice right now)
* https://www.youtube.com/watch?v=H_z4NKvxf1U (Curry On 2017, more detail than YOW! at the end, but slides out of sync for first 1.5min)
* https://www.youtube.com/watch?v=GaorHAlUkVs (YOW! Lambda Jam 2017, slightly shorter)

Paper with additional details: http://www.cse.unsw.edu.au/%7Echak/papers/CK17.html

Graphics, animation, and games programming in Haskell faces a dilemma. We can either use existing frameworks with their highly imperative APIs (such as OpenGL, Cocos2D, or SpriteKit) or we waste a lot of energy trying to re-engineer those rather complex systems from scratch. Or, maybe, we can escape the dilemma. Instead of a Haskell program directly manipulating the mutable object-graph of existing high-level frameworks, we provide an API for purely functional transformations of a Haskell data structure, together with an adaptation layer that transcribes those transformations into edits of the mutable object-graph.

I this talk, I will explain how I used this approach to architect a Haskell binding to the animation system and physics engine of Apple’s SpriteKit framework. I will discuss both how the binding is structured internally and how it achieves the translation of Haskell side changes to SpriteKit and vice versa, such that it is sufficiently efficient. Moreover, I will explain how to use the Haskell library to implement animations and games.

This talk is aimed at an intermediate level. It will use Haskell and SpriteKit as concrete technology in all examples, but it requires neither previous experience with Haskell nor with SpriteKit. In a similar vain, the talk is based on Apple’s SpriteKit framework, as there is no reasonable (in terms of effort) way around platform-specific technology if you want something that is simultaneously state-of-the-art, practically useful, and convenient to use. Nevertheless, the concepts explained in the talk transcend the specific technology and are generally applicable. Moreover, the entire code base of the Haskell binding is open source and available from https://github.com/mchakravarty/HaskellSpriteKit

Presented at
* YOW! Lambda Jam, Sydney, May 2017: http://lambdajam.yowconference.com.au/speakers/manuel-chakravarty-4/
* IFIP WG2.8, Edinburgh, June 2017
* Curry On, Barcelona, June 2017: http://www.curry-on.org/2017/sessions/haskell-spritekit-a-case-study-in-turning-a-stateful-into-a-purely-functional-api.html
* Functional Conf, Bangalore, November 2017: https://functionalconf.com/proposal.html?id=3939

2296a6bdc7779fe4017a23d268c8a79d?s=128

Manuel Chakravarty
PRO

June 19, 2017
Tweet

Transcript

  1. Manuel M T Chakravarty & Gabriele Keller Applicative & UNSW

    Sydney Haskell SpriteKit A Purely Functional API for a Stateful Animation System & Physics Engine mchakravarty TacticalGrace justtesting.org haskellformac.com
  2. The Dilemma

  3. The Dilemma

  4. The Dilemma SKShapeNode *shipOverlayShape = [[SKShapeNode alloc] init]; shipOverlayShape.path =

    boundingPath; shipOverlayShape.strokeColor = [SKColor clearColor]; shipOverlayShape.fillColor = [SKColor colorWithRed:0.0 green:1.0 blue:0.0 alpha:0.5]; [ship addChild:shipOverlayShape];
  5. The Dilemma SKShapeNode *shipOverlayShape = [[SKShapeNode alloc] init]; shipOverlayShape.path =

    boundingPath; shipOverlayShape.strokeColor = [SKColor clearColor]; shipOverlayShape.fillColor = [SKColor colorWithRed:0.0 green:1.0 blue:0.0 alpha:0.5]; [ship addChild:shipOverlayShape];
  6. The Dilemma SKShapeNode *shipOverlayShape = [[SKShapeNode alloc] init]; shipOverlayShape.path =

    boundingPath; shipOverlayShape.strokeColor = [SKColor clearColor]; shipOverlayShape.fillColor = [SKColor colorWithRed:0.0 green:1.0 blue:0.0 alpha:0.5]; [ship addChild:shipOverlayShape]; data GameState = GameState { gsSize :: (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] }
  7. The Compromise Purely Functional API data GameState = GameState {

    gsSize :: (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] }
  8. The Compromise Purely Functional API data GameState = GameState {

    gsSize :: (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented
  9. The Compromise Purely Functional API represent scenes with algebraic data

    types data GameState = GameState { gsSize :: (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented
  10. The Compromise Purely Functional API translate transformations into state changes

    represent scenes with algebraic data types data GameState = GameState { gsSize :: (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented
  11. Purely Functional API data GameState = GameState { gsSize ::

    (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented
  12. Purely Functional API data GameState = GameState { gsSize ::

    (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented Step ❶ — state of the art
  13. Purely Functional API data GameState = GameState { gsSize ::

    (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented Step ❶ — state of the art Step ❷ — purification
  14. Purely Functional API data GameState = GameState { gsSize ::

    (Double, Double) , time :: Millisecond , level :: Int , lifes :: Int , towers :: [Tower] , creeps :: [Creep] , shots :: [Shot] } functional imperative object-oriented Step ❶ — state of the art Step ❷ — purification Step ❸ — efficiency
  15. SpriteKit Physics, fields & particles Animation & lighting Joints &

    constraints
  16. SpriteKit Physics, fields & particles Animation & lighting Joints &

    constraints Lazy Lambda Simple side scroller
  17. Step ❶ The State of the Art

  18. None
  19. None
  20. None
  21. None
  22. 0 pipes moving nodes pipe pair l ground physics score

    contact l l
  23. — node without visuals 0 pipes moving nodes pipe pair

    l ground physics score contact l l
  24. — node without visuals — attached actions 0 pipes moving

    nodes pipe pair l ground physics score contact l l
  25. — node without visuals — attached actions — attached physics

    body 0 pipes moving nodes pipe pair l ground physics score contact l l
  26. — node without visuals — attached actions — attached physics

    body 0 pipes moving nodes pipe pair l ground physics score contact l l
  27. 0 pipes moving nodes pipe pair l ground physics score

    contact l l SKScene SKLabelNode SKNode SKNode SKSpriteNode SKNode SKNode SKNode SKSpriteNode SKSpriteNode SKSpriteNode SKSpriteNode
  28. SKLabelNode SKScene SKNode SKSpriteNode SKEffectNode OOism #1: Subclassing

  29. SKLabelNode SKScene SKNode SKSpriteNode SKEffectNode SKShapeNode SKLightNode ộ OOism #1:

    Subclassing
  30. SKLabelNode SKScene SKNode SKSpriteNode SKEffectNode SKShapeNode SKLightNode ộ OOism #1:

    Subclassing class SKNode: … { var frame: CGRect func calculatedFrame() -> CGRect var position: CGPoint var zRotation: CGFloat var xScale: CGFloat ⋮ }
  31. SKLabelNode SKScene SKNode SKSpriteNode SKEffectNode SKShapeNode SKLightNode ộ OOism #1:

    Subclassing this one is odd class SKNode: … { var frame: CGRect func calculatedFrame() -> CGRect var position: CGPoint var zRotation: CGFloat var xScale: CGFloat ⋮ }
  32. SKLabelNode SKScene SKNode SKSpriteNode SKEffectNode SKShapeNode SKLightNode ộ NSResponder LambdaScene

  33. SKLabelNode SKScene SKNode SKSpriteNode SKEffectNode SKShapeNode SKLightNode ộ NSResponder LambdaScene

    class NSResponder: NSObject { func mouseDown(with: NSEvent) func mouseUp(with: NSEvent) func keyDown(with: NSEvent) func keyUp(with: NSEvent) ⋮ }
  34. SKLabelNode SKScene SKNode SKSpriteNode SKEffectNode SKShapeNode SKLightNode ộ NSResponder LambdaScene

    class NSResponder: NSObject { func mouseDown(with: NSEvent) func mouseUp(with: NSEvent) func keyDown(with: NSEvent) func keyUp(with: NSEvent) ⋮ } user-defined subclass to override event handlers
  35. OOism #2: Mutable Properties

  36. OOism #2: Mutable Properties Lambda tilts (rotation) depending on vertical

    velocity
  37. OOism #2: Mutable Properties class SKNode: … { var frame:

    CGRect var position: CGPoint var zRotation: CGFloat var xScale: CGFloat ⋮ } Lambda tilts (rotation) depending on vertical velocity mutation
  38. None
  39. 0 pipes moving nodes pipe pair l ground physics score

    contact l l
  40. 0 pipes moving nodes pipe pair l ground physics score

    contact l l class SKScene: SKEffectNode { var backgroundColor: NSColor func update(_ currentTime: TimeInterval) ⋮ }
  41. 0 pipes moving nodes pipe pair l ground physics score

    contact l l class SKScene: SKEffectNode { var backgroundColor: NSColor func update(_ currentTime: TimeInterval) ⋮ } called once per frame
  42. OOism #3: In-place Graph Edits

  43. OOism #3: In-place Graph Edits Pipe pairs appear and disappear

    at regular intervals
  44. OOism #3: In-place Graph Edits Pipe pairs appear and disappear

    at regular intervals class SKNode: … { var children: [SKNode] { get } func addChild(_ node: SKNode) ⋮ } mutation
  45. 0 pipes moving nodes ground physics l l pipe pair

    score contact
  46. 0 pipes moving nodes ground physics l l pipe pair

    score contact pipe pair score contact
  47. 0 pipes moving nodes ground physics l l pipe pair

    score contact pipe pair score contact pipe pair score contact
  48. 0 pipes moving nodes ground physics l l pipe pair

    score contact pipe pair score contact
  49. The Three Issues

  50. The Three Issues Subclassing Inherited properties & event handlers

  51. The Three Issues Subclassing Inherited properties & event handlers Mutable

    properties Inplace updates by callbacks
  52. The Three Issues Subclassing Inherited properties & event handlers Mutable

    properties Inplace updates by callbacks Inplace graph edits Add, delete & rearrange nodes
  53. Step ❷ Pure Functions & Datatypes

  54. Subclassing Inherited properties & event handlers

  55. SKLabelNode SKScene SKNode SKSpriteNode SKEffectNode SKShapeNode SKLightNode ộ

  56. SKLabelNode SKScene SKNode SKSpriteNode SKEffectNode SKShapeNode SKLightNode ộ

  57. SKNode SKLabelNode SKSpriteNode SKShapeNode

  58. data Node u = Node { nodeName :: Maybe String

    , nodePosition :: Point , nodeZRotation :: GFloat , nodeUserData :: u ⋮ } | Label { nodeName :: Maybe String , nodePosition :: Point , nodeZRotation :: GFloat , nodeUserData :: u ⋮ , labelText :: String ⋮ } | Sprite { … } | Shape { … } ⋮
  59. data Node u = Node { nodeName :: Maybe String

    , nodePosition :: Point , nodeZRotation :: GFloat , nodeUserData :: u ⋮ } | Label { nodeName :: Maybe String , nodePosition :: Point , nodeZRotation :: GFloat , nodeUserData :: u ⋮ , labelText :: String ⋮ } | Sprite { … } | Shape { … } ⋮ shared fields of ”superclass” node included in each variant
  60. data Node u = Node { nodeName :: Maybe String

    , nodePosition :: Point , nodeZRotation :: GFloat , nodeUserData :: u ⋮ } | Label { nodeName :: Maybe String , nodePosition :: Point , nodeZRotation :: GFloat , nodeUserData :: u ⋮ , labelText :: String ⋮ } | Sprite { … } | Shape { … } ⋮ shared fields of ”superclass” node included in each variant each variant (except Node) has specific fields
  61. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneData :: sceneData , sceneBackgroundColor :: Color ⋮ }
  62. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneData :: sceneData , sceneBackgroundColor :: Color ⋮ } children of a Scene are all Nodes
  63. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneData :: sceneData , sceneBackgroundColor :: Color ⋮ } children of a Scene are all Nodes the Scene and the Nodes may contain extra user-defined data
  64. None
  65. Mutable properties Inplace updates by callbacks

  66. data Node u = Node { … } | Sprite

    { nodeName :: Maybe String , nodeZRotation :: GFloat ⋮ } ⋮ data Scene sceneData nodeData = Scene { sceneName :: Maybe String , sceneChildren :: [Node nodeData] ⋮ }
  67. data Node u = Node { … } | Sprite

    { nodeName :: Maybe String , nodeZRotation :: GFloat ⋮ } ⋮ data Scene sceneData nodeData = Scene { sceneName :: Maybe String , sceneChildren :: [Node nodeData] ⋮ } immutable, but must change to tilt the bird
  68. data Node u = Node { … } | Sprite

    { nodeName :: Maybe String , nodeZRotation :: GFloat ⋮ } ⋮ data Scene sceneData nodeData = Scene { sceneName :: Maybe String , sceneChildren :: [Node nodeData] , sceneUpdate :: Maybe (SceneUpdate sceneData nodeData) ⋮ } immutable, but must change to tilt the bird
  69. data Node u = Node { … } | Sprite

    { nodeName :: Maybe String , nodeZRotation :: GFloat ⋮ } ⋮ data Scene sceneData nodeData = Scene { sceneName :: Maybe String , sceneChildren :: [Node nodeData] , sceneUpdate :: Maybe (SceneUpdate sceneData nodeData) ⋮ } immutable, but must change to tilt the bird callback called once per frame
  70. data Node u = Node { … } | Sprite

    { nodeName :: Maybe String , nodeZRotation :: GFloat ⋮ } ⋮ data Scene sceneData nodeData = Scene { sceneName :: Maybe String , sceneChildren :: [Node nodeData] , sceneUpdate :: Maybe (SceneUpdate sceneData nodeData) ⋮ } immutable, but must change to tilt the bird type SceneUpdate sd nd = Scene sd nd -> TimeInterval -> Scene sd nd callback called once per frame pure scene transformer
  71. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneUpdate :: Maybe (SceneUpdate sceneData nodeData) ⋮ } type SceneUpdate sd nd = Scene sd nd -> TimeInterval -> Scene sd nd
  72. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneUpdate :: Maybe (SceneUpdate sceneData nodeData) , sceneHandleEvent :: Maybe (EventHandler sceneData) ⋮ } type SceneUpdate sd nd = Scene sd nd -> TimeInterval -> Scene sd nd type EventHandler sceneData = Event -> sceneData -> Maybe sceneData
  73. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneUpdate :: Maybe (SceneUpdate sceneData nodeData) , sceneHandleEvent :: Maybe (EventHandler sceneData) ⋮ } type SceneUpdate sd nd = Scene sd nd -> TimeInterval -> Scene sd nd type EventHandler sceneData = Event -> sceneData -> Maybe sceneData transforms game state in response to input events
  74. None
  75. Inplace graph edits Add, delete & rearrange nodes

  76. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneData :: sceneData ⋮ } data Node u = Node { … , nodeChildren :: [Node u] ⋮ } | Label { … , nodeChildren :: [Node u] ⋮ } ⋮
  77. data Scene sceneData nodeData = Scene { sceneName :: Maybe

    String , sceneChildren :: [Node nodeData] , sceneData :: sceneData ⋮ } data Node u = Node { … , nodeChildren :: [Node u] ⋮ } | Label { … , nodeChildren :: [Node u] ⋮ } ⋮ these fields describe the scene tree
  78. None
  79. Step ❸ Hidden State & Efficient Marshalling

  80. 0 pip movin groun l l pip score

  81. 0 pip movin groun l l pip score ObjC ->

    Haskell
  82. 0 pip movin groun l l pip score spawnPipePair ObjC

    -> Haskell
  83. 0 pip movin groun l l pip score 0 pi

    movin groun l l pip score pip score spawnPipePair new subtree ObjC -> Haskell Haskell -> ObjC
  84. 0 pip movin groun l l pip score 0 pi

    movin groun l l pip score pip score spawnPipePair new subtree old subtree ObjC -> Haskell Haskell -> ObjC
  85. Eager Marshalling Is Infeasible

  86. Eager Marshalling Is Infeasible Hidden State Nodes have private state

    that would get lost
  87. Eager Marshalling Is Infeasible Hidden State Nodes have private state

    that would get lost Efficiency Marshalling a whole tree to change a few properties
  88. data Scene sc nd = Scene { … } —

    Haskell record type SceneUpdate sc nd = Scene sc nd -> TimeInterval -> Scene sc nd once per frame 0 moving nodes ground physics l l l
  89. data Scene sc nd = Scene { … } —

    Haskell record type SceneUpdate sc nd = Scene sc nd -> TimeInterval -> Scene sc nd once per frame Scene sc nd lazy marshalling 0 moving nodes ground physics l l l
  90. data Scene sc nd = Scene { … } —

    Haskell record type SceneUpdate sc nd = Scene sc nd -> TimeInterval -> Scene sc nd once per frame Scene sc nd lazy marshalling SceneUpdate sc nd Scene sc nd 0 moving nodes ground physics l l l
  91. data Scene sc nd = Scene { … } —

    Haskell record type SceneUpdate sc nd = Scene sc nd -> TimeInterval -> Scene sc nd once per frame Scene sc nd lazy marshalling SceneUpdate sc nd Scene sc nd compute diff 0 moving nodes ground physics l l l
  92. data Scene sc nd = Scene { … } —

    Haskell record type SceneUpdate sc nd = Scene sc nd -> TimeInterval -> Scene sc nd once per frame Scene sc nd lazy marshalling SceneUpdate sc nd Scene sc nd compute diff apply changes 0 moving nodes ground physics l l l
  93. Lazy Marshalling 0 moving nodes ground physics l l l

  94. Lazy Marshalling 0 moving nodes ground physics l l l

    lazy marshalling Scene { sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ }
  95. Lazy Marshalling 0 moving nodes ground physics l l l

    lazy marshalling Scene { sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } thunks (suspended computations)
  96. Lazy Marshalling 0 moving nodes ground physics l l l

    lazy marshalling Scene { sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } thunks (suspended computations)
  97. Lazy Marshalling marshalSKScene :: SKScene -> Scene sceneData nodeData marshalSKScene

    skScene = Scene { sceneName = unsafePerformIO $(objc … ) , sceneChildren = unsafePerformIO $ do { nodes <- $(objc … ) ; unsafeInterleaveNSArrayTolistOfNode nodes } ⋮ }
  98. Lazy Marshalling marshalSKScene :: SKScene -> Scene sceneData nodeData marshalSKScene

    skScene = Scene { sceneName = unsafePerformIO $(objc … ) , sceneChildren = unsafePerformIO $ do { nodes <- $(objc … ) ; unsafeInterleaveNSArrayTolistOfNode nodes } ⋮ } produces a thunk (unevaluated expression)
  99. Lazy Marshalling marshalSKScene :: SKScene -> Scene sceneData nodeData marshalSKScene

    skScene = Scene { sceneName = unsafePerformIO $(objc … ) , sceneChildren = unsafePerformIO $ do { nodes <- $(objc … ) ; unsafeInterleaveNSArrayTolistOfNode nodes } ⋮ } produces a thunk (unevaluated expression) inline Objective-C
  100. 0 moving nodes ground physics l l l Scene {

    sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } Compute Diff & Update
  101. 0 moving nodes ground physics l l l Scene {

    sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } Compute Diff & Update Scene { sceneName = ”New Name” , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } SceneUpdate sc nd
  102. 0 moving nodes ground physics l l l Scene {

    sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } Compute Diff & Update Scene { sceneName = ”New Name” , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } SceneUpdate sc nd compute diff
  103. 0 moving nodes ground physics l l l Scene {

    sceneName = , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } Compute Diff & Update Scene { sceneName = ”New Name” , sceneChildren = , sceneData = , sceneBackgroundColor = ⋮ } SceneUpdate sc nd compute diff
  104. Compute Diff & Update marshalScene :: Scene sceneData nodeData —

    original scene -> Scene sceneData nodeData — updated scene -> IO SKScene marshalScene originalScene Scene{..} = do { … ; case reallyUnsafePtrEquality# originalName sceneName of 1# -> return () _ -> $(objc …) ; updateChildren skNode originalChildren sceneChildren ⋮ }
  105. Compute Diff & Update marshalScene :: Scene sceneData nodeData —

    original scene -> Scene sceneData nodeData — updated scene -> IO SKScene marshalScene originalScene Scene{..} = do { … ; case reallyUnsafePtrEquality# originalName sceneName of 1# -> return () _ -> $(objc …) ; updateChildren skNode originalChildren sceneChildren ⋮ }
  106. Object Caching & Identity

  107. Object Caching & Identity data Node u = Node {

    nodeName :: Maybe String , nodePosition :: Point ⋮ , nodeForeign :: Maybe SKNode } | Label { nodeName :: Maybe String , nodePosition :: Point ⋮ , nodeForeign :: Maybe SKNode , labelText :: String ⋮ } ⋮ if available, basis for marshalling this node to ObjC land
  108. Haskell SpriteKit is open source! https://github.com/mchakravarty/HaskellSpriteKit mchakravarty TacticalGrace justtesting.org haskellformac.com

    >< state types Want to see the gory details?
  109. Thank you!

  110. Image Attribution https://pixabay.com/photo-51675/ https://pixabay.com/photo-509871/ https://pixabay.com/photo-1294844/