Building a Pixel Art Editor with Elm Shuhei Kagawa http://shuheikagawa.com/pixelm 1

About Me Shuhei Kagawa !" Frontend Engineer @ Zalando Writing JavaScript by day and Elm by night Consuming IPA ! as well as API " Twitter: @shuheikagawa GitHub: @shuhei http://shuheikagawa.com/pixelm 2

Agenda 1. Pixel Art 2. Pixel Grid - Data Structure 3. Pixel Grid - Event Handling 4. Animation - Data Structure 5. Animation - JavaScript Interop 6. Animation - Event Handling http://shuheikagawa.com/pixelm 3

1. Pixel Art http://shuheikagawa.com/pixelm 4

Check out great works by: • @1041uuu • @waneella • @timswast http://shuheikagawa.com/pixelm 5

This is awesome! http://shuheikagawa.com/pixelm 6

Let's draw! http://shuheikagawa.com/pixelm 7

Let's draw! Let's make a tool to draw! http://shuheikagawa.com/pixelm 8

Pixelm • Web-based drawing tool (16x16) • Elm: 1,700 LOC, JavaScript: 150 LOC • Mobile-compatible • Already many tools for desktop, but not many for mobile • Animation support • Export to SVG/GIF/Animated GIF http://shuheikagawa.com/pixelm https://github.com/shuhei/pixelm http://shuheikagawa.com/pixelm 9

2. Pixel Grid - Data Structure http://shuheikagawa.com/pixelm 10

Pixel Grid with SVG • elm-lang/svg • s for pixels • s for borders between pixels http://shuheikagawa.com/pixelm 11

Data Structure for Pixels import Array.Hamt as Array exposing (Array) type alias PixelGrid = Array (Array Color) -- ! Straightforward -- " Filled with transparent color at first -- # A bit tedious to get/set a pixel type alias PixelGrid = Dict (Int, Int) Color -- ! getting & setting a pixel are super easy -- ! No need to keep transparent pixels -- # A transparent pixel can be no entry or a transparent color -- # `Nothing` can be a transparent pixel or out of range http://shuheikagawa.com/pixelm 12

Bucket Tool http://shuheikagawa.com/pixelm 13

fill : Int -> Int -> a -> Array2 a -> Array2 a fill x y to arr2 = let neighbors x y = [ ( x - 1, y ), ( x, y - 1), ( x + 1, y ), ( x, y + 1 ) ] fillRegion a ( x, y ) ( visited, arr2 ) = case get x y arr2 of Nothing -> ( visited, arr2 ) Just c -> if Set.member ( x, y ) visited then ( visited, arr2 ) else if c == a then List.foldl (fillRegion a) ( Set.insert ( x, y ) visited, set x y to arr2 ) (neighbors x y) else ( Set.insert ( x, y ) visited, arr2 ) start a = Tuple.second <| fillRegion a ( x, y ) ( Set.empty, arr2 ) in get x y arr2 |> Maybe.map start |> Maybe.withDefault arr2 http://shuheikagawa.com/pixelm 14

3. Pixel Grid - Event Handling http://shuheikagawa.com/pixelm 15

Drawing on Pixels viewPixel : Int -> Int -> Color viewPixel col row color = Svg.rect [ Svg.Attributes.width "1" , Svg.Attributes.height "1" , Svg.Attributes.x <| toString (toFloat col) , Svg.Attributes.y <| toString (toFloat row) , Svg.Attributes.fill <| ColorUtil.toColorString color , Svg.Events.onMouseDown <| MouseDown col row , Svg.Events.onMouseMove <| MouseMove col row , Svg.Events.onMouseUp <| MouseUp ] [] This works fine on desktop. http://shuheikagawa.com/pixelm 16

Drawing on Pixels viewPixel : Int -> Int -> Color viewPixel col row color = Svg.rect [ Svg.Attributes.width "1" , Svg.Attributes.height "1" , Svg.Attributes.x <| toString (toFloat col) , Svg.Attributes.y <| toString (toFloat row) , Svg.Attributes.fill <| ColorUtil.toColorString color , Svg.Events.onMouseDown <| MouseDown col row , Svg.Events.onMouseMove <| MouseMove col row , Svg.Events.onMouseUp <| MouseUp , Svg.Events.on "touchstart" <| MouseDown col row -- !ɹtouchmove doesn't work... , Svg.Events.on "touchmove" <| MouseMove col row , Svg.Events.on "touchend" <| MouseUp ] [] http://shuheikagawa.com/pixelm 17

http://shuheikagawa.com/pixelm 18

http://shuheikagawa.com/pixelm 19

Decode event as JSON -- Cannot get data from a event Html.Events.onClick msg -- Can get data from a event with JSON decoder! Html.Events.on "click" decoder Html.Events.onWithOptions "click" options decoder http://shuheikagawa.com/pixelm 20

Relative Touch Position decodeMouseEvent : (( Float, Float ) -> msg) -> Json.Decoder msg decodeMouseEvent tagger = let decodeXY xField yField = Json.map2 (,) (Json.field xField Json.float) (Json.field yField Json.float) decodeTouch = Json.at [ "touches", "0" ] <| decodeXY "clientX" "clientY" decodeTarget = Json.field "currentTarget" <| decodeXY "offsetLeft" "offsetTop" in Json.map2 minusPos decodeTouch decodeTarget |> Json.map tagger minusPos : (Float, Float) -> (Float, Float) -> (Float, Float) minusPos ( x0, y0 ) ( x1, y1 ) = ( x0 - x1, y0 - y1 ) http://shuheikagawa.com/pixelm 21

viewGrid grid = Html.div [ Html.Events.on "mousedown" <| decodeMouseEvent MouseDown , Html.Events.on "mousemove" <| decodeMouseEvent MouseMove , Html.Events.on "mouseup" <| Json.succeed MouseUp , Html.Events.on "touchstart" <| decodeTouchEvent MouseDown , Html.Events.on "touchmove" <| decodeTouchEvent MouseMove , Html.Events.on "touchend" <| Json.succeed MouseUp ] [ viewSvg grid ] update msg = case msg of MouseDown screenPos -> ( { model | frames = updateCurrentFrame (toPixelPos screenPos) model } , Cmd.none) -- ... http://shuheikagawa.com/pixelm 22

http://shuheikagawa.com/pixelm 23

http://shuheikagawa.com/pixelm 24

4. Animation - Data Structure http://shuheikagawa.com/pixelm 25

http://shuheikagawa.com/pixelm 26

Animation Frames • Always have a selected item (cannot be empty) • Get the selected item without Maybe http://shuheikagawa.com/pixelm 27

Array with a selected item -- ! Duplicated frame data type SelectionArray a = SelectionArray a (Array a) -- ! Cannot get the current frame without `Maybe` type SelectionArray a = SelectionArray Int (Array a) -- " type alias SelectionArray a = { previous : Array a , current : a , next : Array a } http://shuheikagawa.com/pixelm 28

Changing Selection http://shuheikagawa.com/pixelm 29

5. Animation - JavaScript Interop http://shuheikagawa.com/pixelm 30

Exporting Images port download : DownloadData -> Cmd msg -- Skinney/elm-array-exploration/Array.Hamt needs to be converted type alias DownloadData = { grids : List (List (List RGBA)) , format : String } -- elm-lang/core/Color needs to be converted type alias RGBA = { red : Int, green : Int, blue: Int, alpha: Int } http://shuheikagawa.com/pixelm 31

Exporting Images with JavaScript • SVG: String concatenation • GIF: Canvas + blueimp-canvas-to-blob (a polyfill for Canvas.toBlob) • Animated GIF: Canvas + gif.js • File download: FileSaver.js http://shuheikagawa.com/pixelm 32

http://shuheikagawa.com/pixelm 33

6. Animation - Event Handling http://shuheikagawa.com/pixelm 34

Drag & Drop for Swapping Frames • ! Easy with HTML5 Drag & Drop API • On mobile devices: ios-html5-drag-drop-shim (also works for Android...) • For passing data, used Model instead of event.dataTransfer http://shuheikagawa.com/pixelm 35

Drag & Drop with Elm preventDefault : String -> msg -> Html.Attribute msg preventDefault eventName msg = Html.Events.onWithOptions eventName { preventDefault = True, stopPropagation = False } (Json.succeed msg) viewFrame : Frame -> Html Msg viewFrame frame = Html.div [ Html.Attributes.draggable "true" , Html.Events.on "dragstart" <| Json.succeed (SelectFrame frame) , preventDefault "ondragenter" NoOp , preventDefault "ondragover" NoOp , preventDefault "ontouchmove" NoOp , preventDefault "drop" <| DropOnFrame frame ] [ viewGrid frame ] http://shuheikagawa.com/pixelm 36

Double Click Double click/tap shows a modal to: • duplicate a frame • delete a frame Easy with dblclick event. But it's not supported on iOS Safari ! http://shuheikagawa.com/pixelm 37

Can we do this in Elm? element.addEventListener('click', function (event) { var el = event.currentTarget; if (el.dataset.clicked === 'true') { // double click!! } else { // single click! } if (el.dataset.timer) { clearTimeout(parseInt(el.dataset.timer, 10)); } el.dataset.timer = setTimeout(function () { delete el.dataset.clicked; }, 500); el.dataset.clicked = 'true'; }); http://shuheikagawa.com/pixelm 38

http://shuheikagawa.com/pixelm 39

Let's do it in Elm! supportDoubleClick : Html.Attribute msg supportDoubleClick = Html.Attributes.attribute "onclick" """ var el = event.currentTarget; setTimeout(function () { if (el.dataset.timer) { clearTimeout(parseInt(el.dataset.timer, 10)); } el.dataset.timer = setTimeout(function () { delete el.dataset.clicked; }, 500); el.dataset.clicked = 'true'; }); """ http://shuheikagawa.com/pixelm 40

onSingleOrDoubleClick : msg -> msg -> Html.Attribute msg onSingleOrDoubleClick singleMessage doubleMessage = let chooseMessage isDoubleClick = if isDoubleClick then doubleMessage else singleMessage decodeData = Json.at [ "currentTarget", "dataset", "clicked" ] Json.string |> Json.map ((==) "true") decodeClicked = Json.oneOf [ decodeData, Json.succeed False ] in HE.onWithOptions "click" { preventDefault = True, stopPropagation = False } (Json.map chooseMessage decodeClicked) http://shuheikagawa.com/pixelm 41

button : Html Msg button = Html.button [ supportDoubleClick , onSingleOrDoubleClick OnSingleClick OnDoubleClick ] [ Html.text "Double click me!" ] http://shuheikagawa.com/pixelm 42

http://shuheikagawa.com/pixelm 43

A Bug in elm-lang/core Changes on Json.Decode.oneOf are not reflected to DOM event handlers. There was a bug on quality check of Json.Decode.oneOf and oneOf decoder was not updated. Probably not many people are not abusing JSON decoders... Html.button [ HE.on "click" <| -- OK Json.succeed (Foo model.something) , HE.on "click" <| -- Not updated when something changes Json.oneOf [ Json.succeed (Foo model.something) ] ] [ Html.text "click me" ] http://shuheikagawa.com/pixelm 44

Thoughts • It's been super easy to add features with Elm • Implementing interaction can be a bit tedious, but totally doable with Elm • With JSON decoders! • Now I can start drawing... http://shuheikagawa.com/pixelm 45

Thanks! http://shuheikagawa.com/pixelm 46