Upgrade to Pro — share decks privately, control downloads, hide ads and more …

사진편집기! 이것DOM 적용해볼GL?

kakao
PRO
December 08, 2022

사진편집기! 이것DOM 적용해볼GL?

#WebGL

새로운 사진편집기에 WebGL을 적용하고, 공용 컴포넌트를 개발하면서 마주했던 선택과 문제 해결과정을 공유합니다.

발표자 : sky.blue
더 멋진 카카오의 에디터를 위해 노력합니다. 카카오 FE플랫폼팀 스카이입니다.

kakao
PRO

December 08, 2022
Tweet

More Decks by kakao

Other Decks in Programming

Transcript

  1. Copyright 2022. Kakao Corp. All rights reserved. Redistribution or public display is not permitted without written permission from Kakao.
    ! 사진편집기! 이것DOM 적용해볼GL?
    황선준 Sky.blue
    카카오
    if(kakao)2022

    View Slide

  2. 1. ! 사진편집기에 WebGL 부어보기
    2. " DOM딱 DOM딱 아이템 붙이기

    View Slide

  3. ! 사진편집기에 WebGL 부어보기

    View Slide

  4. Who?
    “누구십니까?”

    View Slide

  5. Brunch
    Daum cafe Kakao story

    View Slide

  6. View Slide

  7. ! 사진편집기!
    - 글쓰기에서 제공하는 사진 편집 기능
    - 자르기 / 크기조절
    - 보정 / 필터
    - 텍스트 / 이미지 추가

    View Slide

  8. ! 사진편집기!

    View Slide

  9. ! 사진편집기!

    View Slide

  10. Re: ! 사진편집기에 WebGL 부어보기

    View Slide

  11. Why?
    “왜 붓게 되었을까?”

    View Slide

  12. API
    # 옛날 옛날엔..

    View Slide

  13. (축) API 종료
    # 옛날 옛날엔..
    $

    View Slide

  14. 능력자 절찬 모집중
    # 옛날 옛날엔..
    %

    View Slide

  15. 여기가 맞나요?
    # 옛날 옛날엔..

    View Slide

  16. &' 새친구후보 WebGL
    - Web Graphics Library
    - OpenGL ES 2.0 기반
    - GPU 사용
    - JS 개발 지원
    - 대부분의 모던 브라우저 지원 ( IE 11 지원 )

    View Slide

  17. (
    “알아보자”

    View Slide

  18. How to?
    “어떻게 사용하는 걸까??”

    View Slide

  19. ❓ WebGL
    F
    Image
    V
    User Canvas WebGL
    create
    input
    create
    create
    input
    render
    T
    Input

    View Slide

  20. ❓ WebGL
    F
    Image
    V
    User Canvas WebGL
    create
    input
    create
    create
    input
    render
    T
    Input

    View Slide

  21. ❓ WebGL
    F
    Image
    V
    User Canvas WebGL
    input
    create
    create
    input
    render
    T
    Input
    create

    View Slide

  22. ❓ WebGL
    F
    Image
    V
    User Canvas WebGL
    input
    create
    create
    input
    render
    T
    Input
    create

    View Slide

  23. ❓ WebGL
    F
    Image
    V
    User Canvas
    create
    input
    create
    create
    input
    render
    T
    Input
    WebGL

    View Slide

  24. F
    V
    Vertex Fragment
    T
    Texture

    View Slide

  25. F
    V
    좌표 Fragment
    T
    Texture

    View Slide

  26. F
    V
    좌표 색칠
    T
    Texture

    View Slide

  27. F
    V
    좌표 색칠
    T
    데이터

    View Slide

  28. ❓ WebGL
    Vertex Shader Fragment Shader
    Original Clipspace
    Vertex Shader
    attribute vec3 a_position;
    uniform mat4 u_matrix;
    void main() {
    gl_Position = u_matrix * a_position
    }

    View Slide

  29. ❓ WebGL
    Vertex Shader Fragment Shader
    Original Clipspace
    Vertex Shader
    attribute vec3 a_position;
    uniform mat4 u_matrix;
    void main() {
    gl_Position = u_matrix * a_position
    }

    View Slide

  30. ❓ WebGL
    Vertex Shader Fragment Shader
    Original Clipspace
    Vertex Shader
    attribute vec3 a_position;
    uniform mat4 u_matrix;
    void main() {
    gl_Position = u_matrix * a_position
    }

    View Slide

  31. ❓ WebGL
    Vertex Shader Fragment Shader
    Original Clipspace
    Vertex Shader
    attribute vec3 a_position;
    uniform mat4 u_matrix;
    void main() {
    gl_Position = u_matrix * a_position
    }

    View Slide

  32. ❓ WebGL
    Image Filter Result

    View Slide

  33. ❓ WebGL
    Image Filter Result

    View Slide

  34. ❓ WebGL
    Image Filter Result

    View Slide

  35. ⏰ 얼마나 빠르지?

    View Slide

  36. ⏰ 속도?

    View Slide

  37. ⏰ 속도?
    정보
    OS macOS Monterey ( 12.5.1 )
    CPU Intel core i9 2.4GHz 8 core
    Graphic ADM Radeon PRO 5300M 4 GB
    Memory 32GB 2667 MHz DDR4

    View Slide

  38. API ( ms ) WebGL ( ms ) 차이 ( % )
    100 1191.0 1216.7 -2.1
    200 1212.2 1230.5 -1.5
    300 1391.9 1203.1 13.6
    400 1455.4 1267.9 12.9
    500 1826.0 1400.2 23.3
    600 1874.9 1310.6 30.1
    700 1684.9 1580.2 6.2
    800 2189.9 1874.1 22.3
    900 2412.7 1714.4 28.9
    1000 2673.8 1899.4 29.0
    1100 2746.3 2007.3 26.9
    1200 2835.3 2279.5 19.6
    1300 3449.1 2072.0 39.9
    1400 3677.4 2169.7 41.0
    1500 3324.3 2839.0 14.6
    1600 3558.4 2430.2 31.7
    1700 3518.8 2913.2 17.2
    1800 5277.7 2940.4 44.3
    1900 5206.2 3096.6 40.5
    2000 5128.1 3784.9 26.2

    View Slide

  39. API ( ms ) WebGL ( ms ) 차이 ( % )
    100 1191.0 1216.7 -2.1
    200 1212.2 1230.5 -1.5
    300 1391.9 1203.1 13.6
    400 1455.4 1267.9 12.9
    500 1826.0 1400.2 23.3
    600 1874.9 1310.6 30.1
    700 1684.9 1580.2 6.2
    800 2189.9 1874.1 22.3
    900 2412.7 1714.4 28.9
    1000 2673.8 1899.4 29.0
    1100 2746.3 2007.3 26.9
    1200 2835.3 2279.5 19.6
    1300 3449.1 2072.0 39.9
    1400 3677.4 2169.7 41.0
    1500 3324.3 2839.0 14.6
    1600 3558.4 2430.2 31.7
    1700 3518.8 2913.2 17.2
    1800 5277.7 2940.4 44.3
    1900 5206.2 3096.6 40.5
    2000 5128.1 3784.9 26.2

    View Slide

  40. 0
    3000
    6000
    100 300 500 700 900 1100 1300 1500 1700 1900
    ⏰ 속도?
    API
    WebGL
    임의 필터에 대한 10회 실행시간 누적합 ( disabled cache, ms )
    개발환경에 따라 결과가 상이할 수 있습니다

    View Slide

  41. +
    “그의 손에 쥐어지는..”

    View Slide

  42. What?
    “무엇이 바뀌었을까요??”

    View Slide

  43. 모자이크
    효과

    View Slide

  44. , WebGL 활약

    View Slide

  45. , WebGL 활약

    View Slide

  46. , WebGL 활약

    View Slide

  47. , WebGL 활약

    View Slide

  48. -
    “짝짝짝”

    View Slide

  49. 1. ! 사진편집기에 WebGL 부어보기
    2. " DOM딱 DOM딱 아이템 붙이기

    View Slide

  50. .
    “어떤걸 선택하지?”

    View Slide

  51. Canvas

    View Slide

  52. Item

    View Slide

  53. DOM Canvas
    편의성 일관성

    View Slide

  54. / 선택지
    DOM Canvas
    디버깅 난이도 낮음 높음
    이벤트 처리 난이도 낮음 높음
    CSS 사이드 이펙트 있음 없음
    History - Canvas 동기화 있음 없음
    공간 제약 없음 있음

    View Slide

  55. 0
    “여정의 시작”

    View Slide


  56. “희망편”

    View Slide

  57. Dev Easy
    빠르고 쉬운 개발 편리한 사용 방법

    View Slide

  58. 1 희노애락
    const Decorator = (props: Props) => {
    // ...states
    // ... events
    // ... react hooks
    return

    {children}






    ;
    };
    export default Decorator;

    View Slide

  59. 1 희노애락
    Components Decorator
    const Decorator = (props: Props) => {
    // ...states
    // ... events
    // ... react hooks
    return

    {children}






    ;
    };
    export default Decorator;
    const ItemComponents = () => {
    const [state, setState] = useState();
    return
    Good

    }

    View Slide

  60. 1 희노애락
    Components Decorator
    interface Props {
    moveable: boolean;
    resizable: boolean;
    removeable: boolean;
    rotatable: boolean;
    size?: Size;
    zoom?: number;
    degree?: number;
    position?: Position;
    fixRatio?: boolean;
    hidden?: boolean;
    onChange?: DecoratorOnChangeCallback;
    removeCallback?: DecoratorRemoveCallback;
    }
    const ItemComponents = () => {
    const [state, setState] = useState();
    return moveable={true}
    resizable={false}
    removeable={false}
    rotatable={true}>
    Good

    }

    View Slide

  61. .
    “너무 과하지 않나?”

    View Slide

  62. history preview
    menu
    1 희노애락
    update
    notify
    update render
    item

    View Slide

  63. history preview
    menu
    1 희노애락
    update
    notify
    update render
    item

    View Slide

  64. history preview
    menu
    1 희노애락
    update
    notify
    update render
    item

    View Slide

  65. ,
    “LGTM”

    View Slide

  66. 2
    “불편해..”

    View Slide

  67. 3
    ”코드 중복”

    View Slide

  68. 1 희노애락
    const Item = (props: Props) => {
    const dispatch = useDispatch();
    const [state, setState] = useState();
    const onMoveCallback = useCallback(() => { /* .. */ }, []);
    const onResizeCallback = useCallback(() => { /* .. */ }, []);
    const onRemoveCallback = useCallback(() => { /* .. */ }, []);
    const onRotateCallback = useCallback(() => { /* .. */ }, []);
    return


    /* .. пઙ ೩ٜ۞ .. */

    ;
    };
    export default Item;

    View Slide

  69. 1 희노애락
    스티커 텍스트 서명
    const Sticker = (props: Props) => {
    const dispatch = useDispatch();
    const [state, setState] = useState();
    const onMoveCallback
    = useCallback(() => { /* .. */ }, []);
    const onResizeCallback
    = useCallback(() => { /* .. */ }, []);
    const onRemoveCallback
    = useCallback(() => { /* .. */ }, []);
    const onRotateCallback
    = useCallback(() => { /* .. */ }, []);
    return







    ;
    };
    export default Sticker;
    const Text = (props: Props) => {
    const dispatch = useDispatch();
    const [state, setState] = useState();
    const onMoveCallback
    = useCallback(() => { /* .. */ }, []);
    const onResizeCallback
    = useCallback(() => { /* .. */ }, []);
    const onRemoveCallback
    = useCallback(() => { /* .. */ }, []);
    const onRotateCallback
    = useCallback(() => { /* .. */ }, []);
    return







    ;
    };
    export default Text;
    const Sign = (props: Props) => {
    const dispatch = useDispatch();
    const [state, setState] = useState();
    const onMoveCallback
    = useCallback(() => { /* .. */ }, []);
    const onResizeCallback
    = useCallback(() => { /* .. */ }, []);
    return





    ;
    };
    export default Sign;

    View Slide

  70. 1 희노애락
    스티커 텍스트 서명
    const Sticker = (props: Props) => {
    const dispatch = useDispatch();
    const [state, setState] = useState();
    const onMoveCallback
    = useCallback(() => { /* .. */ }, []);
    const onResizeCallback
    = useCallback(() => { /* .. */ }, []);
    const onRemoveCallback
    = useCallback(() => { /* .. */ }, []);
    const onRotateCallback
    = useCallback(() => { /* .. */ }, []);
    return







    ;
    };
    export default Sticker;
    const Text = (props: Props) => {
    const dispatch = useDispatch();
    const [state, setState] = useState();
    const onMoveCallback
    = useCallback(() => { /* .. */ }, []);
    const onResizeCallback
    = useCallback(() => { /* .. */ }, []);
    const onRemoveCallback
    = useCallback(() => { /* .. */ }, []);
    const onRotateCallback
    = useCallback(() => { /* .. */ }, []);
    return







    ;
    };
    export default Text;
    const Sign = (props: Props) => {
    const dispatch = useDispatch();
    const [state, setState] = useState();
    const onMoveCallback
    = useCallback(() => { /* .. */ }, []);
    const onResizeCallback
    = useCallback(() => { /* .. */ }, []);
    return





    ;
    };
    export default Sign;

    View Slide

  71. 4
    “으 냄새”

    View Slide

  72. 5
    ”관리 중복”

    View Slide

  73. history preview
    menu
    1 희노애락
    update
    notify
    update render
    item

    View Slide

  74. preview
    item
    menu
    1 희노애락
    notify
    update render
    update
    history

    View Slide

  75. 6
    “어라 왜이렇게 작동하지”

    View Slide

  76. 7
    “᤼ᔥᤍ᪫”

    View Slide

  77. const Decorator = (props: Props) => {
    const [box, setBox] = useState();
    const onMoveCallback = useCallback(() => { /* .. */ }, []);
    const onResizeCallback = useCallback(() => { /* .. */ }, []);
    const onRemoveCallback = useCallback(() => { /* .. */ }, []);
    const onRotateCallback = useCallback(() => { /* .. */ }, []);
    const onMoveStop = useCallback(() => { onChange(nextPosition) }, []);
    const onResizeStop = useCallback(() => { onChange(nextSize) }, []);
    const onRotateStop = useCallback(() => { onChange(nextAngle) }, []);
    return
    { children }

    { props.moveable && }
    { props.resizeable && }
    { props.removable && }
    { props.rotatable && }

    ;
    };
    export default Decorator;
    1 희노애락

    View Slide

  78. const Decorator = (props: Props) => {
    const [box, setBox] = useState();
    const onMoveCallback = useCallback(() => { /* .. */ }, []);
    const onResizeCallback = useCallback(() => { /* .. */ }, []);
    const onRemoveCallback = useCallback(() => { /* .. */ }, []);
    const onRotateCallback = useCallback(() => { /* .. */ }, []);
    const onMoveStop = useCallback(() => { onChange(nextPosition) }, []);
    const onResizeStop = useCallback(() => { onChange(nextSize) }, []);
    const onRotateStop = useCallback(() => { onChange(nextAngle) }, []);
    return
    { children }

    { props.moveable && }
    { props.resizeable && }
    { props.removable && }
    { props.rotatable && }

    ;
    };
    export default Decorator;
    1 희노애락

    View Slide

  79. 1 희노애락
    Component Props
    interface Props {
    moveable: boolean;
    resizable: boolean;
    removeable: boolean;
    rotatable: boolean;
    size?: Size;
    zoom?: number;
    degree?: number;
    position?: Position;
    fixRatio?: boolean;
    hidden?: boolean;
    onChange?: DecoratorOnChangeCallback;
    removeCallback?:
    DecoratorRemoveCallback;
    }
    const Decorator = (props: Props) => {
    const [box, setBox] = useState();
    const onMoveCallback = useCallback(() => { /* .. */ }, []);
    const onResizeCallback = useCallback(() => { /* .. */ }, []);
    const onRemoveCallback = useCallback(() => { /* .. */ }, []);
    const onRotateCallback = useCallback(() => { /* .. */ }, []);
    const onMoveStop = useCallback(() => { onChange(nextPosition) },
    []);
    const onResizeStop = useCallback(() => { onChange(nextSize) }, []);
    const onRotateStop = useCallback(() => { onChange(nextAngle) }, []);
    return
    { children }

    { props.moveable && }
    { props.resizeable && }
    { props.removable && }
    { props.rotatable && }

    ;
    };
    export default Decorator;

    View Slide

  80. history preview
    item
    menu
    1 희노애락
    notify
    update render
    update

    View Slide

  81. history preview
    item
    menu
    1 희노애락
    notify
    update render
    update
    decorator
    update
    update

    View Slide

  82. 1 희노애락

    View Slide

  83. ၌ᛑ
    “절망편”

    View Slide

  84. 8
    “장미와 가시”

    View Slide

  85. / 선택지
    DOM Canvas
    디버깅 난이도 낮음 높음
    이벤트 처리 난이도 낮음 높음
    CSS 사이드 이펙트 있음 없음
    History - Canvas 동기화 있음 없음
    공간 제약 없음 있음

    View Slide

  86. 9
    “Sync”

    View Slide

  87. addon
    attacher
    resize/crop
    processing
    history adapter
    text / image
    addon
    image
    :; 희노애락
    canvas
    processor

    View Slide

  88. addon
    attacher
    resize/crop
    processing
    history adapter
    text / image
    addon
    image
    :; 희노애락
    canvas
    processor

    View Slide

  89. addon
    attacher
    resize/crop
    processing
    history adapter
    text / image
    addon
    image
    :; 희노애락
    canvas
    processor

    View Slide

  90. :; 희노애락
    export const calcAbsPosition = (
    position: Position,
    itemSize: Size,
    imgSize: Size
    ): Coordinate => {
    // ӝળ੼੉ হ׮ === left-top
    if (!position.centerPoint) return {x: position.x, y: position.y};
    // ӝળ੼੉ য়ܲଃੋ҃਋ => left۽ ߸҃
    const x = position.centerPoint[0] === 'l' ? position.x : imgSize.width -
    position.x - itemSize.width;
    // ӝળ੼੉ ইېੋ҃਋ => topਵ۽ ߸҃
    const y = position.centerPoint[1] === 't' ? position.y : imgSize.height -
    position.y - itemSize.height;
    return {x, y};
    };

    View Slide

  91. addon
    attacher
    resize/crop
    processing
    history adapter
    text / image
    addon
    image
    :; 희노애락
    canvas
    processor

    View Slide

  92. :; 희노애락
    const x = (position?.x || 0) + (flip?.flipV ? width : 0);
    const y = (position?.y || 0) + (flip?.flipH ? height : 0);
    const flipWidth = flip?.flipV ? -width : width;
    const flipHeight = flip?.flipH ? -height : height;
    const font = textStyle?.font || '';
    const fontSize = textStyle?.size || DEFAULT_TEXT_SIZE;
    const fontHeight = fontSize * lineHeight;
    const align = textStyle?.align || 'left';
    const textTop = ((fontHeight - fontSize) >> 1);
    const foreColor = color2String(textStyle?.textColor);
    const backColor = color2String(textStyle?.backgroundColor);
    const lines = content.split('\n');

    View Slide

  93. ctx.save();
    ctx.translate(x, y);
    ctx.globalAlpha = (transparent || PERCENT) / PERCENT;
    if (position?.rotate) this.rotateCanvas(ctx, position.rotate, flipWidth, flipHeight);
    if (flip?.flipV || flip?.flipH) ctx.scale(flip.flipV ? -1 : 1, flip.flipH ? -1 : 1);
    // Text Background
    ctx.fillStyle = backColor;
    ctx.fillRect(0, 0, width, height);
    // Text
    ctx.textBaseline = 'top';
    ctx.font = toFontString(fontSize, font, textStyle?.bold, textStyle?.italic);
    ctx.fillStyle = foreColor;
    lines.forEach((line, idx) => {
    const lineY = textTop + (idx * fontHeight);
    const lineWidth = ctx.measureText(line).width;
    const alignPadding = align === 'left' ? 0 : (align === 'center' ? (width - lineWidth) >> 1 : (width -
    lineWidth));
    ctx.fillText(line, alignPadding, lineY);
    // Text Underline
    if (textStyle?.underline) {
    const underlineHeight = Math.round(fontSize / FONT_BASE);
    const underlineY = lineY + fontSize - underlineHeight;
    ctx.beginPath();
    ctx.lineWidth = underlineHeight;
    ctx.strokeStyle = foreColor;
    ctx.moveTo(alignPadding, underlineY);
    ctx.lineTo(alignPadding + lineWidth, underlineY);
    ctx.stroke();
    }
    });

    View Slide

  94. addon
    attacher
    resize/crop
    processing
    history adapter
    text / image
    addon
    image
    :; 희노애락
    canvas
    processor

    View Slide

  95. <
    “EZ”

    View Slide

  96. =
    “나비효과”

    View Slide

  97. :; 희노애락

    View Slide

  98. :; 희노애락
    const Preview = () => {
    return





    }

    View Slide

  99. :; 희노애락
    const Preview = () => {
    return
    <>











    >
    }

    View Slide

  100. :; 희노애락

    View Slide

  101. :; 희노애락

    View Slide

  102. :; 희노애락

    View Slide

  103. :; 희노애락

    View Slide

  104. >
    “?”

    View Slide

  105. :; 희노애락

    View Slide

  106. :; 희노애락

    View Slide

  107. :; 희노애락

    View Slide

  108. export const pos2CropPos = (item: Rectangle & {rotate?: number}, crop: Crop) => {
    // ੹୊ܻ ߂ nullish ୊ܻ
    const cropCtr = {x: crop.centerX, y: crop.centerY};
    const cropAxis = {x: crop.width >> 1, y: crop.height >> 1};
    const cropBoxLt = cropBoxLtPosition(crop);
    const angle = {item: item.rotate ?? 0, crop: crop.angle * ((crop.flipH + crop.flipV) % 2 ? -1 : 1)};
    const resRotate = (FULL_ANGLE + angle.item - (crop.rotate * QUATER_ANGLE + angle.crop)) %
    FULL_ANGLE;
    // பߡझী ࠁৈ૑ח Cropbox੄ Left-top
    const rtCropBoxLt = reverseRotateDeg(cropBoxLt, crop.angle, cropCtr);
    // pos ӝળ item ઺बઝ಴ ୶୹
    const itemCtr = {x: item.x + (item.width >> 1), y: item.y + (item.height >> 1)};
    const {x, y} = flipOnAxis(itemCtr, cropAxis, {flipV: !!crop.flipV, flipH: !!crop.flipH});
    const r = (crop.angle + crop.rotate * QUATER_ANGLE) / HALF_ANGLE * Math.PI;
    const a = x * Math.cos(r) + y * Math.sin(r);
    const b = x * Math.sin(r) - y * Math.cos(r);
    // pos ઝ಴҅ীࢲ CropPos ઝ಴҅۽ ੉ز
    const cropItemCtr = {x: rtCropBoxLt.x + a, y: rtCropBoxLt.y - b};
    const ltCorr = {x: item.width >> 1, y: item.height >> 1};
    const cropLt = {x: cropItemCtr.x - ltCorr.x, y: cropItemCtr.y - ltCorr.y};
    const result = {x: cropLt.x, y: cropLt.y, rotate: resRotate};
    return result;
    };

    View Slide

  109. ?
    “이걸 이렇게 오래?”

    View Slide

  110. :; 희노애락
    export type Item = {
    id: number;
    src: string;
    width: number;
    height: number;
    custom: boolean;
    // for preview and canvas
    position: AnglePosition;
    flip: Flip;
    // for crop menu
    cropPosition: AnglePosition;
    cropFlip: Flip;
    };

    View Slide

  111. @
    “짠”

    View Slide


  112. “즐겨~”

    View Slide

  113. A
    “많 관 부”

    View Slide

  114. 참고 자료 & 이미지 출처
    1 ) Web GL ӝୡ https://webglfundamentals.org/webgl/lessons/ko/webgl-fundamentals.html
    P 14, 15) https://commons.wikimedia.org/wiki/File:WebGL_Logo.svg?uselang=ko

    View Slide

  115. E.O.D

    View Slide