Building a Pixel Art Editor with Elm

A Talk about my tiny pixel art editor at Elm Berlin Meetup #4. https://www.meetup.com/Elm-Berlin/

- The app: http://shuheikagawa.com/pixelm/
- Source code: https://github.com/shuhei/pixelm

Transcript

1. Building a Pixel Art Editor with Elm
   Shuhei Kagawa
   http://shuheikagawa.com/pixelm 1
   

   View Slide

2. About Me
   Shuhei Kagawa
   !"
   Frontend Engineer @ Zalando
   Writing JavaScript by day and Elm by
   night
   Consuming IPA ! as well as API "
   Twitter: @shuheikagawa
   GitHub: @shuhei
   http://shuheikagawa.com/pixelm 2
   

   View Slide

3. Agenda
   1. Pixel Art
   2. Pixel Grid - Data Structure
   3. Pixel Grid - Event Handling
   4. Animation - Data Structure
   5. Animation - JavaScript Interop
   6. Animation - Event Handling
   http://shuheikagawa.com/pixelm 3
   

   View Slide

4. 1. Pixel Art
   http://shuheikagawa.com/pixelm 4
   

   View Slide

5. Check out great works by:
   • @1041uuu
   • @waneella
   • @timswast
   http://shuheikagawa.com/pixelm 5
   

   View Slide

6. This is awesome!
   http://shuheikagawa.com/pixelm 6
   

   View Slide

7. Let's draw!
   http://shuheikagawa.com/pixelm 7
   

   View Slide

8. Let's draw!
   Let's make a tool to draw!
   http://shuheikagawa.com/pixelm 8
   

   View Slide

9. Pixelm
   • Web-based drawing tool (16x16)
   • Elm: 1,700 LOC, JavaScript: 150 LOC
   • Mobile-compatible
   • Already many tools for desktop, but
   not many for mobile
   • Animation support
   • Export to SVG/GIF/Animated GIF
   http://shuheikagawa.com/pixelm
   https://github.com/shuhei/pixelm
   http://shuheikagawa.com/pixelm 9
   

   View Slide

10. 2. Pixel Grid - Data Structure
    http://shuheikagawa.com/pixelm 10
    

    View Slide

11. Pixel Grid with SVG
    • elm-lang/svg
    • s for pixels
    • s for borders between pixels
    http://shuheikagawa.com/pixelm 11
    

    View Slide

12. Data Structure for Pixels
    import Array.Hamt as Array exposing (Array)
    type alias PixelGrid =
    Array (Array Color)
    --
    !
    Straightforward
    --
    "
    Filled with transparent color at first
    --
    #
    A bit tedious to get/set a pixel
    type alias PixelGrid =
    Dict (Int, Int) Color
    --
    !
    getting & setting a pixel are super easy
    --
    !
    No need to keep transparent pixels
    --
    #
    A transparent pixel can be no entry or a transparent color
    --
    #
    `Nothing` can be a transparent pixel or out of range
    http://shuheikagawa.com/pixelm 12
    

    View Slide

13. Bucket Tool
    http://shuheikagawa.com/pixelm 13
    

    View Slide

14. fill : Int -> Int -> a -> Array2 a -> Array2 a
    fill x y to arr2 =
    let
    neighbors x y =
    [ ( x - 1, y ), ( x, y - 1), ( x + 1, y ), ( x, y + 1 ) ]
    fillRegion a ( x, y ) ( visited, arr2 ) =
    case get x y arr2 of
    Nothing ->
    ( visited, arr2 )
    Just c ->
    if Set.member ( x, y ) visited then
    ( visited, arr2 )
    else if c == a then
    List.foldl
    (fillRegion a)
    ( Set.insert ( x, y ) visited, set x y to arr2 )
    (neighbors x y)
    else
    ( Set.insert ( x, y ) visited, arr2 )
    start a =
    Tuple.second <| fillRegion a ( x, y ) ( Set.empty, arr2 )
    in
    get x y arr2
    |> Maybe.map start
    |> Maybe.withDefault arr2
    http://shuheikagawa.com/pixelm 14
    

    View Slide

15. 3. Pixel Grid - Event Handling
    http://shuheikagawa.com/pixelm 15
    

    View Slide

16. Drawing on Pixels
    viewPixel : Int -> Int -> Color
    viewPixel col row color =
    Svg.rect
    [ Svg.Attributes.width "1"
    , Svg.Attributes.height "1"
    , Svg.Attributes.x <| toString (toFloat col)
    , Svg.Attributes.y <| toString (toFloat row)
    , Svg.Attributes.fill <| ColorUtil.toColorString color
    , Svg.Events.onMouseDown <| MouseDown col row
    , Svg.Events.onMouseMove <| MouseMove col row
    , Svg.Events.onMouseUp <| MouseUp
    ]
    []
    This works fine on desktop.
    http://shuheikagawa.com/pixelm 16
    

    View Slide

17. Drawing on Pixels
    viewPixel : Int -> Int -> Color
    viewPixel col row color =
    Svg.rect
    [ Svg.Attributes.width "1"
    , Svg.Attributes.height "1"
    , Svg.Attributes.x <| toString (toFloat col)
    , Svg.Attributes.y <| toString (toFloat row)
    , Svg.Attributes.fill <| ColorUtil.toColorString color
    , Svg.Events.onMouseDown <| MouseDown col row
    , Svg.Events.onMouseMove <| MouseMove col row
    , Svg.Events.onMouseUp <| MouseUp
    , Svg.Events.on "touchstart" <| MouseDown col row
    -- !ɹtouchmove doesn't work...
    , Svg.Events.on "touchmove" <| MouseMove col row
    , Svg.Events.on "touchend" <| MouseUp
    ]
    []
    http://shuheikagawa.com/pixelm 17
    

    View Slide

18. http://shuheikagawa.com/pixelm 18
    

    View Slide

19. http://shuheikagawa.com/pixelm 19
    

    View Slide

20. Decode event as JSON
    -- Cannot get data from a event
    Html.Events.onClick msg
    -- Can get data from a event with JSON decoder!
    Html.Events.on "click" decoder
    Html.Events.onWithOptions "click" options decoder
    http://shuheikagawa.com/pixelm 20
    

    View Slide

21. Relative Touch Position
    decodeMouseEvent : (( Float, Float ) -> msg) -> Json.Decoder msg
    decodeMouseEvent tagger =
    let
    decodeXY xField yField =
    Json.map2 (,)
    (Json.field xField Json.float)
    (Json.field yField Json.float)
    decodeTouch =
    Json.at [ "touches", "0" ] <|
    decodeXY "clientX" "clientY"
    decodeTarget =
    Json.field "currentTarget" <|
    decodeXY "offsetLeft" "offsetTop"
    in
    Json.map2 minusPos decodeTouch decodeTarget
    |> Json.map tagger
    minusPos : (Float, Float) -> (Float, Float) -> (Float, Float)
    minusPos ( x0, y0 ) ( x1, y1 ) =
    ( x0 - x1, y0 - y1 )
    http://shuheikagawa.com/pixelm 21
    

    View Slide

22. viewGrid grid =
    Html.div
    [ Html.Events.on "mousedown" <| decodeMouseEvent MouseDown
    , Html.Events.on "mousemove" <| decodeMouseEvent MouseMove
    , Html.Events.on "mouseup" <| Json.succeed MouseUp
    , Html.Events.on "touchstart" <| decodeTouchEvent MouseDown
    , Html.Events.on "touchmove" <| decodeTouchEvent MouseMove
    , Html.Events.on "touchend" <| Json.succeed MouseUp
    ]
    [ viewSvg grid ]
    update msg =
    case msg of
    MouseDown screenPos ->
    ( { model | frames = updateCurrentFrame (toPixelPos screenPos) model }
    , Cmd.none)
    -- ...
    http://shuheikagawa.com/pixelm 22
    

    View Slide

23. http://shuheikagawa.com/pixelm 23
    

    View Slide

24. http://shuheikagawa.com/pixelm 24
    

    View Slide

25. 4. Animation - Data Structure
    http://shuheikagawa.com/pixelm 25
    

    View Slide

26. http://shuheikagawa.com/pixelm 26
    

    View Slide

27. Animation Frames
    • Always have a selected item (cannot be empty)
    • Get the selected item without Maybe
    http://shuheikagawa.com/pixelm 27
    

    View Slide

28. Array with a selected item
    -- ! Duplicated frame data
    type SelectionArray a =
    SelectionArray a (Array a)
    -- ! Cannot get the current frame without `Maybe`
    type SelectionArray a =
    SelectionArray Int (Array a)
    -- "
    type alias SelectionArray a =
    { previous : Array a , current : a , next : Array a }
    http://shuheikagawa.com/pixelm 28
    

    View Slide

29. Changing Selection
    http://shuheikagawa.com/pixelm 29
    

    View Slide

30. 5. Animation - JavaScript Interop
    http://shuheikagawa.com/pixelm 30
    

    View Slide

31. Exporting Images
    port download : DownloadData -> Cmd msg
    -- Skinney/elm-array-exploration/Array.Hamt needs to be converted
    type alias DownloadData =
    { grids : List (List (List RGBA)) , format : String }
    -- elm-lang/core/Color needs to be converted
    type alias RGBA =
    { red : Int, green : Int, blue: Int, alpha: Int }
    http://shuheikagawa.com/pixelm 31
    

    View Slide

32. Exporting Images with JavaScript
    • SVG: String concatenation
    • GIF: Canvas + blueimp-canvas-to-blob (a polyfill for
    Canvas.toBlob)
    • Animated GIF: Canvas + gif.js
    • File download: FileSaver.js
    http://shuheikagawa.com/pixelm 32
    

    View Slide

33. http://shuheikagawa.com/pixelm 33
    

    View Slide

34. 6. Animation - Event Handling
    http://shuheikagawa.com/pixelm 34
    

    View Slide

35. Drag & Drop for Swapping Frames
    • ! Easy with HTML5 Drag & Drop API
    • On mobile devices: ios-html5-drag-drop-shim (also works for
    Android...)
    • For passing data, used Model instead of event.dataTransfer
    http://shuheikagawa.com/pixelm 35
    

    View Slide

36. Drag & Drop with Elm
    preventDefault : String -> msg -> Html.Attribute msg
    preventDefault eventName msg =
    Html.Events.onWithOptions eventName
    { preventDefault = True, stopPropagation = False }
    (Json.succeed msg)
    viewFrame : Frame -> Html Msg
    viewFrame frame =
    Html.div
    [ Html.Attributes.draggable "true"
    , Html.Events.on "dragstart" <| Json.succeed (SelectFrame frame)
    , preventDefault "ondragenter" NoOp
    , preventDefault "ondragover" NoOp
    , preventDefault "ontouchmove" NoOp
    , preventDefault "drop" <| DropOnFrame frame
    ]
    [ viewGrid frame ]
    http://shuheikagawa.com/pixelm 36
    

    View Slide

37. Double Click
    Double click/tap shows a modal to:
    • duplicate a frame
    • delete a frame
    Easy with dblclick event. But it's not
    supported on iOS Safari !
    http://shuheikagawa.com/pixelm 37
    

    View Slide

38. Can we do this in Elm?
    element.addEventListener('click', function (event) {
    var el = event.currentTarget;
    if (el.dataset.clicked === 'true') {
    // double click!!
    } else {
    // single click!
    }
    if (el.dataset.timer) {
    clearTimeout(parseInt(el.dataset.timer, 10));
    }
    el.dataset.timer = setTimeout(function () {
    delete el.dataset.clicked;
    }, 500);
    el.dataset.clicked = 'true';
    });
    http://shuheikagawa.com/pixelm 38
    

    View Slide

39. http://shuheikagawa.com/pixelm 39
    

    View Slide

40. Let's do it in Elm!
    supportDoubleClick : Html.Attribute msg
    supportDoubleClick =
    Html.Attributes.attribute "onclick"
    """
    var el = event.currentTarget;
    setTimeout(function () {
    if (el.dataset.timer) {
    clearTimeout(parseInt(el.dataset.timer, 10));
    }
    el.dataset.timer = setTimeout(function () {
    delete el.dataset.clicked;
    }, 500);
    el.dataset.clicked = 'true';
    });
    """
    ...
    http://shuheikagawa.com/pixelm 40
    

    View Slide

41. onSingleOrDoubleClick : msg -> msg -> Html.Attribute msg
    onSingleOrDoubleClick singleMessage doubleMessage =
    let
    chooseMessage isDoubleClick =
    if isDoubleClick then
    doubleMessage
    else
    singleMessage
    decodeData =
    Json.at [ "currentTarget", "dataset", "clicked" ] Json.string
    |> Json.map ((==) "true")
    decodeClicked =
    Json.oneOf [ decodeData, Json.succeed False ]
    in
    HE.onWithOptions "click"
    { preventDefault = True, stopPropagation = False }
    (Json.map chooseMessage decodeClicked)
    http://shuheikagawa.com/pixelm 41
    

    View Slide

42. button : Html Msg
    button =
    Html.button
    [ supportDoubleClick
    , onSingleOrDoubleClick OnSingleClick OnDoubleClick
    ]
    [ Html.text "Double click me!" ]
    http://shuheikagawa.com/pixelm 42
    

    View Slide

43. http://shuheikagawa.com/pixelm 43
    

    View Slide

44. A Bug in elm-lang/core
    Changes on
    Json.Decode.oneOf
    are not
    reflected to DOM event handlers.
    There was a bug on quality check of
    Json.Decode.oneOf
    and
    oneOf
    decoder
    was not updated. Probably not many
    people are not abusing JSON decoders...
    Html.button
    [ HE.on "click" <| -- OK
    Json.succeed (Foo model.something)
    , HE.on "click" <| -- Not updated when something changes
    Json.oneOf [ Json.succeed (Foo model.something) ]
    ]
    [ Html.text "click me" ]
    http://shuheikagawa.com/pixelm 44
    

    View Slide

45. Thoughts
    • It's been super easy to add features with Elm
    • Implementing interaction can be a bit tedious, but totally doable
    with Elm
    • With JSON decoders!
    • Now I can start drawing...
    http://shuheikagawa.com/pixelm 45
    

    View Slide

46. Thanks!
    http://shuheikagawa.com/pixelm 46
    

    View Slide