사진편집기! 이것DOM 적용해볼GL?

#WebGL

새로운 사진편집기에 WebGL을 적용하고, 공용 컴포넌트를 개발하면서 마주했던 선택과 문제 해결과정을 공유합니다.

발표자 : sky.blue
더 멋진 카카오의 에디터를 위해 노력합니다. 카카오 FE플랫폼팀 스카이입니다.

Transcript

1. Copyright 2022. Kakao Corp. All rights reserved. Redistribution or public display is not permitted without written permission from Kakao. ! 사진편집기! 이것DOM 적용해볼GL? 황선준 Sky.blue 카카오 if(kakao)2022
2. 1. ! 사진편집기에 WebGL 부어보기 2. " DOM딱 DOM딱 아이템 붙이기
3. ! 사진편집기에 WebGL 부어보기
4. Who? “누구십니까?”
5. Brunch Daum cafe Kakao story
7. ! 사진편집기! - 글쓰기에서 제공하는 사진 편집 기능 - 자르기 / 크기조절 - 보정 / 필터 - 텍스트 / 이미지 추가
8. ! 사진편집기!
9. ! 사진편집기!
10. Re: ! 사진편집기에 WebGL 부어보기
11. Why? “왜 붓게 되었을까?”
12. API # 옛날 옛날엔..
13. (축) API 종료 # 옛날 옛날엔.. $
14. 능력자 절찬 모집중 # 옛날 옛날엔.. %
15. 여기가 맞나요? # 옛날 옛날엔..
16. &' 새친구후보 WebGL - Web Graphics Library - OpenGL ES 2.0 기반 - GPU 사용 - JS 개발 지원 - 대부분의 모던 브라우저 지원 ( IE 11 지원 )
17. ( “알아보자”
18. How to? “어떻게 사용하는 걸까??”
19. ❓ WebGL F Image V User Canvas WebGL create input create create input render T Input
20. ❓ WebGL F Image V User Canvas WebGL create input create create input render T Input
21. ❓ WebGL F Image V User Canvas WebGL input create create input render T Input create
22. ❓ WebGL F Image V User Canvas WebGL input create create input render T Input create
23. ❓ WebGL F Image V User Canvas create input create create input render T Input WebGL
24. F V Vertex Fragment T Texture
25. F V 좌표 Fragment T Texture
26. F V 좌표 색칠 T Texture
27. F V 좌표 색칠 T 데이터
28. ❓ WebGL Vertex Shader Fragment Shader Original Clipspace Vertex Shader attribute vec3 a_position; uniform mat4 u_matrix; void main() { gl_Position = u_matrix * a_position }
29. ❓ WebGL Vertex Shader Fragment Shader Original Clipspace Vertex Shader attribute vec3 a_position; uniform mat4 u_matrix; void main() { gl_Position = u_matrix * a_position }
30. ❓ WebGL Vertex Shader Fragment Shader Original Clipspace Vertex Shader attribute vec3 a_position; uniform mat4 u_matrix; void main() { gl_Position = u_matrix * a_position }
31. ❓ WebGL Vertex Shader Fragment Shader Original Clipspace Vertex Shader attribute vec3 a_position; uniform mat4 u_matrix; void main() { gl_Position = u_matrix * a_position }
32. ❓ WebGL Image Filter Result
33. ❓ WebGL Image Filter Result
34. ❓ WebGL Image Filter Result
35. ⏰ 얼마나 빠르지?
36. ⏰ 속도?
37. ⏰ 속도? 정보 OS macOS Monterey ( 12.5.1 ) CPU Intel core i9 2.4GHz 8 core Graphic ADM Radeon PRO 5300M 4 GB Memory 32GB 2667 MHz DDR4
38. API ( ms ) WebGL ( ms ) 차이 ( % ) 100 1191.0 1216.7 -2.1 200 1212.2 1230.5 -1.5 300 1391.9 1203.1 13.6 400 1455.4 1267.9 12.9 500 1826.0 1400.2 23.3 600 1874.9 1310.6 30.1 700 1684.9 1580.2 6.2 800 2189.9 1874.1 22.3 900 2412.7 1714.4 28.9 1000 2673.8 1899.4 29.0 1100 2746.3 2007.3 26.9 1200 2835.3 2279.5 19.6 1300 3449.1 2072.0 39.9 1400 3677.4 2169.7 41.0 1500 3324.3 2839.0 14.6 1600 3558.4 2430.2 31.7 1700 3518.8 2913.2 17.2 1800 5277.7 2940.4 44.3 1900 5206.2 3096.6 40.5 2000 5128.1 3784.9 26.2
39. API ( ms ) WebGL ( ms ) 차이 ( % ) 100 1191.0 1216.7 -2.1 200 1212.2 1230.5 -1.5 300 1391.9 1203.1 13.6 400 1455.4 1267.9 12.9 500 1826.0 1400.2 23.3 600 1874.9 1310.6 30.1 700 1684.9 1580.2 6.2 800 2189.9 1874.1 22.3 900 2412.7 1714.4 28.9 1000 2673.8 1899.4 29.0 1100 2746.3 2007.3 26.9 1200 2835.3 2279.5 19.6 1300 3449.1 2072.0 39.9 1400 3677.4 2169.7 41.0 1500 3324.3 2839.0 14.6 1600 3558.4 2430.2 31.7 1700 3518.8 2913.2 17.2 1800 5277.7 2940.4 44.3 1900 5206.2 3096.6 40.5 2000 5128.1 3784.9 26.2
40. 0 3000 6000 100 300 500 700 900 1100 1300 1500 1700 1900 ⏰ 속도? API WebGL 임의 필터에 대한 10회 실행시간 누적합 ( disabled cache, ms ) 개발환경에 따라 결과가 상이할 수 있습니다
41. + “그의 손에 쥐어지는..”
42. What? “무엇이 바뀌었을까요??”
43. 모자이크 효과
44. , WebGL 활약
45. , WebGL 활약
46. , WebGL 활약
47. , WebGL 활약
48. - “짝짝짝”
49. 1. ! 사진편집기에 WebGL 부어보기 2. " DOM딱 DOM딱 아이템 붙이기
50. . “어떤걸 선택하지?”
51. Canvas
52. Item
53. DOM Canvas 편의성 일관성
54. / 선택지 DOM Canvas 디버깅 난이도 낮음 높음 이벤트 처리 난이도 낮음 높음 CSS 사이드 이펙트 있음 없음 History - Canvas 동기화 있음 없음 공간 제약 없음 있음
55. 0 “여정의 시작”
56. ὥ “희망편”
57. Dev Easy 빠르고 쉬운 개발 편리한 사용 방법
58. 1 희노애락 const Decorator = (props: Props) => { // ...states // ... events // ... react hooks return <Draggable> <container> {children} <plugin-container> <Remove/> <Rotate/> <Resize/> </plugin-container> </container> </Draggable>; }; export default Decorator;
59. 1 희노애락 Components Decorator const Decorator = (props: Props) => { // ...states // ... events // ... react hooks return <Draggable> <container> {children} <plugin-container> <Remove/> <Rotate/> <Resize/> </plugin-container> </container> </Draggable>; }; export default Decorator; const ItemComponents = () => { const [state, setState] = useState(); return <Decorator> <element>Good</element> </Decorator> }
60. 1 희노애락 Components Decorator interface Props { moveable: boolean; resizable: boolean; removeable: boolean; rotatable: boolean; size?: Size; zoom?: number; degree?: number; position?: Position; fixRatio?: boolean; hidden?: boolean; onChange?: DecoratorOnChangeCallback; removeCallback?: DecoratorRemoveCallback; } const ItemComponents = () => { const [state, setState] = useState(); return <Decorator moveable={true} resizable={false} removeable={false} rotatable={true}> <element>Good</element> </Decorator> }
61. . “너무 과하지 않나?”
62. history preview menu 1 희노애락 update notify update render item
63. history preview menu 1 희노애락 update notify update render item
64. history preview menu 1 희노애락 update notify update render item
65. , “LGTM”
66. 2 “불편해..”
67. 3 ”코드 중복”
68. 1 희노애락 const Item = (props: Props) => { const dispatch = useDispatch(); const [state, setState] = useState(); const onMoveCallback = useCallback(() => { /* .. */ }, []); const onResizeCallback = useCallback(() => { /* .. */ }, []); const onRemoveCallback = useCallback(() => { /* .. */ }, []); const onRotateCallback = useCallback(() => { /* .. */ }, []); return <container> <element/> <handler-container> /* .. пઙ ೩ٜ۞ .. */ </handler-container> </container>; }; export default Item;
69. 1 희노애락 스티커 텍스트 서명 const Sticker = (props: Props) => { const dispatch = useDispatch(); const [state, setState] = useState(); const onMoveCallback = useCallback(() => { /* .. */ }, []); const onResizeCallback = useCallback(() => { /* .. */ }, []); const onRemoveCallback = useCallback(() => { /* .. */ }, []); const onRotateCallback = useCallback(() => { /* .. */ }, []); return <container> <sticker-element/> <handler-container> <Move onMove={onMoveCallack}> <Resize onResize={onResizeCallback}> <Remove onRemove={onRemoveCallback}> <Rotate onRotate={onRotateCallback}> </handler-container> </container>; }; export default Sticker; const Text = (props: Props) => { const dispatch = useDispatch(); const [state, setState] = useState(); const onMoveCallback = useCallback(() => { /* .. */ }, []); const onResizeCallback = useCallback(() => { /* .. */ }, []); const onRemoveCallback = useCallback(() => { /* .. */ }, []); const onRotateCallback = useCallback(() => { /* .. */ }, []); return <container> <text-element/> <handler-container> <Move onMove={onMoveCallack}> <Resize onResize={onResizeCallback}> <Remove onRemove={onRemoveCallback}> <Rotate onRotate={onRotateCallback}> </handler-container> </container>; }; export default Text; const Sign = (props: Props) => { const dispatch = useDispatch(); const [state, setState] = useState(); const onMoveCallback = useCallback(() => { /* .. */ }, []); const onResizeCallback = useCallback(() => { /* .. */ }, []); return <container> <sign-element/> <handler-container> <Move onMove={onMoveCallack}> <Resize onResize={onResizeCallback}> </handler-container> </container>; }; export default Sign;
70. 1 희노애락 스티커 텍스트 서명 const Sticker = (props: Props) => { const dispatch = useDispatch(); const [state, setState] = useState(); const onMoveCallback = useCallback(() => { /* .. */ }, []); const onResizeCallback = useCallback(() => { /* .. */ }, []); const onRemoveCallback = useCallback(() => { /* .. */ }, []); const onRotateCallback = useCallback(() => { /* .. */ }, []); return <container> <element/> <handler-container> <Move onMove={onMoveCallack}> <Resize onResize={onResizeCallback}> <Remove onRemove={onRemoveCallback}> <Rotate onRotate={onRotateCallback}> </handler-container> </container>; }; export default Sticker; const Text = (props: Props) => { const dispatch = useDispatch(); const [state, setState] = useState(); const onMoveCallback = useCallback(() => { /* .. */ }, []); const onResizeCallback = useCallback(() => { /* .. */ }, []); const onRemoveCallback = useCallback(() => { /* .. */ }, []); const onRotateCallback = useCallback(() => { /* .. */ }, []); return <container> <element/> <handler-container> <Move onMove={onMoveCallack}> <Resize onResize={onResizeCallback}> <Remove onRemove={onRemoveCallback}> <Rotate onRotate={onRotateCallback}> </handler-container> </container>; }; export default Text; const Sign = (props: Props) => { const dispatch = useDispatch(); const [state, setState] = useState(); const onMoveCallback = useCallback(() => { /* .. */ }, []); const onResizeCallback = useCallback(() => { /* .. */ }, []); return <container> <element/> <handler-container> <Move onMove={onMoveCallack}> <Resize onResize={onResizeCallback}> </handler-container> </container>; }; export default Sign;
71. 4 “으 냄새”
72. 5 ”관리 중복”
73. history preview menu 1 희노애락 update notify update render item
74. preview item menu 1 희노애락 notify update render update history
75. 6 “어라 왜이렇게 작동하지”
76. 7 “᤼ᔥᤍ᪫”
77. const Decorator = (props: Props) => { const [box, setBox] = useState<DecoratorBoxProps>(); const onMoveCallback = useCallback(() => { /* .. */ }, []); const onResizeCallback = useCallback(() => { /* .. */ }, []); const onRemoveCallback = useCallback(() => { /* .. */ }, []); const onRotateCallback = useCallback(() => { /* .. */ }, []); const onMoveStop = useCallback(() => { onChange(nextPosition) }, []); const onResizeStop = useCallback(() => { onChange(nextSize) }, []); const onRotateStop = useCallback(() => { onChange(nextAngle) }, []); return <container> { children } <handler-container> { props.moveable && <Move onMove={onMoveCallack}/> } { props.resizeable && <Resize onResize={onResizeCallback}/> } { props.removable && <Remove onRemove={onRemoveCallback}/> } { props.rotatable && <Rotate onRotate={onRotateCallback}/> } </handler-container> </container>; }; export default Decorator; 1 희노애락
78. const Decorator = (props: Props) => { const [box, setBox] = useState<DecoratorBoxProps>(); const onMoveCallback = useCallback(() => { /* .. */ }, []); const onResizeCallback = useCallback(() => { /* .. */ }, []); const onRemoveCallback = useCallback(() => { /* .. */ }, []); const onRotateCallback = useCallback(() => { /* .. */ }, []); const onMoveStop = useCallback(() => { onChange(nextPosition) }, []); const onResizeStop = useCallback(() => { onChange(nextSize) }, []); const onRotateStop = useCallback(() => { onChange(nextAngle) }, []); return <container> { children } <handler-container> { props.moveable && <Move onMove={onMoveCallack}/> } { props.resizeable && <Resize onResize={onResizeCallback}/> } { props.removable && <Remove onRemove={onRemoveCallback}/> } { props.rotatable && <Rotate onRotate={onRotateCallback}/> } </handler-container> </container>; }; export default Decorator; 1 희노애락
79. 1 희노애락 Component Props interface Props { moveable: boolean; resizable: boolean; removeable: boolean; rotatable: boolean; size?: Size; zoom?: number; degree?: number; position?: Position; fixRatio?: boolean; hidden?: boolean; onChange?: DecoratorOnChangeCallback; removeCallback?: DecoratorRemoveCallback; } const Decorator = (props: Props) => { const [box, setBox] = useState<DecoratorBoxProps>(); const onMoveCallback = useCallback(() => { /* .. */ }, []); const onResizeCallback = useCallback(() => { /* .. */ }, []); const onRemoveCallback = useCallback(() => { /* .. */ }, []); const onRotateCallback = useCallback(() => { /* .. */ }, []); const onMoveStop = useCallback(() => { onChange(nextPosition) }, []); const onResizeStop = useCallback(() => { onChange(nextSize) }, []); const onRotateStop = useCallback(() => { onChange(nextAngle) }, []); return <container> { children } <handler-container> { props.moveable && <Move onMove={onMoveCallack}/> } { props.resizeable && <Resize onResize={onResizeCallback}/> } { props.removable && <Remove onRemove={onRemoveCallback}/> } { props.rotatable && <Rotate onRotate={onRotateCallback}/> } </handler-container> </container>; }; export default Decorator;
80. history preview item menu 1 희노애락 notify update render update
81. history preview item menu 1 희노애락 notify update render update decorator update update
82. 1 희노애락
83. ၌ᛑ “절망편”
84. 8 “장미와 가시”
85. / 선택지 DOM Canvas 디버깅 난이도 낮음 높음 이벤트 처리 난이도 낮음 높음 CSS 사이드 이펙트 있음 없음 History - Canvas 동기화 있음 없음 공간 제약 없음 있음
86. 9 “Sync”
87. addon attacher resize/crop processing history adapter text / image addon image :; 희노애락 canvas processor
88. addon attacher resize/crop processing history adapter text / image addon image :; 희노애락 canvas processor
89. addon attacher resize/crop processing history adapter text / image addon image :; 희노애락 canvas processor
90. :; 희노애락 export const calcAbsPosition = ( position: Position, itemSize: Size, imgSize: Size ): Coordinate => { // ӝળ੼੉ হ׮ === left-top if (!position.centerPoint) return {x: position.x, y: position.y}; // ӝળ੼੉ য়ܲଃੋ҃਋ => left۽ ߸҃ const x = position.centerPoint[0] === 'l' ? position.x : imgSize.width - position.x - itemSize.width; // ӝળ੼੉ ইېੋ҃਋ => topਵ۽ ߸҃ const y = position.centerPoint[1] === 't' ? position.y : imgSize.height - position.y - itemSize.height; return {x, y}; };
91. addon attacher resize/crop processing history adapter text / image addon image :; 희노애락 canvas processor
92. :; 희노애락 const x = (position?.x || 0) + (flip?.flipV ? width : 0); const y = (position?.y || 0) + (flip?.flipH ? height : 0); const flipWidth = flip?.flipV ? -width : width; const flipHeight = flip?.flipH ? -height : height; const font = textStyle?.font || ''; const fontSize = textStyle?.size || DEFAULT_TEXT_SIZE; const fontHeight = fontSize * lineHeight; const align = textStyle?.align || 'left'; const textTop = ((fontHeight - fontSize) >> 1); const foreColor = color2String(textStyle?.textColor); const backColor = color2String(textStyle?.backgroundColor); const lines = content.split('\n');
93. ctx.save(); ctx.translate(x, y); ctx.globalAlpha = (transparent || PERCENT) / PERCENT; if (position?.rotate) this.rotateCanvas(ctx, position.rotate, flipWidth, flipHeight); if (flip?.flipV || flip?.flipH) ctx.scale(flip.flipV ? -1 : 1, flip.flipH ? -1 : 1); // Text Background ctx.fillStyle = backColor; ctx.fillRect(0, 0, width, height); // Text ctx.textBaseline = 'top'; ctx.font = toFontString(fontSize, font, textStyle?.bold, textStyle?.italic); ctx.fillStyle = foreColor; lines.forEach((line, idx) => { const lineY = textTop + (idx * fontHeight); const lineWidth = ctx.measureText(line).width; const alignPadding = align === 'left' ? 0 : (align === 'center' ? (width - lineWidth) >> 1 : (width - lineWidth)); ctx.fillText(line, alignPadding, lineY); // Text Underline if (textStyle?.underline) { const underlineHeight = Math.round(fontSize / FONT_BASE); const underlineY = lineY + fontSize - underlineHeight; ctx.beginPath(); ctx.lineWidth = underlineHeight; ctx.strokeStyle = foreColor; ctx.moveTo(alignPadding, underlineY); ctx.lineTo(alignPadding + lineWidth, underlineY); ctx.stroke(); } });
94. addon attacher resize/crop processing history adapter text / image addon image :; 희노애락 canvas processor
95. < “EZ”
96. = “나비효과”
97. :; 희노애락
98. :; 희노애락 const Preview = () => { return <preview-wrapper> <original-preview/> <canvas-base-layer/> <dom-base-layer/> </preview-wrapper> }
99. :; 희노애락 const Preview = () => { return <> <preview-wrapper> <crop-wrapper/> <original-preview/> <canvas-base-layer/> <dom-base-layer/> </crop-wrapper> <crop-wrapper-fit-box> <sign-layer/> </crop-wrapper-fit-box> </preview-wrapper> <crop-view/> </> }
100. :; 희노애락
101. :; 희노애락
102. :; 희노애락
103. :; 희노애락
104. > “?”
105. :; 희노애락
106. :; 희노애락
107. :; 희노애락
108. export const pos2CropPos = (item: Rectangle & {rotate?: number}, crop: Crop) => { // ੹୊ܻ ߂ nullish ୊ܻ const cropCtr = {x: crop.centerX, y: crop.centerY}; const cropAxis = {x: crop.width >> 1, y: crop.height >> 1}; const cropBoxLt = cropBoxLtPosition(crop); const angle = {item: item.rotate ?? 0, crop: crop.angle * ((crop.flipH + crop.flipV) % 2 ? -1 : 1)}; const resRotate = (FULL_ANGLE + angle.item - (crop.rotate * QUATER_ANGLE + angle.crop)) % FULL_ANGLE; // பߡझী ࠁৈ૑ח Cropbox੄ Left-top const rtCropBoxLt = reverseRotateDeg(cropBoxLt, crop.angle, cropCtr); // pos ӝળ item ઺बઝ಴ ୶୹ const itemCtr = {x: item.x + (item.width >> 1), y: item.y + (item.height >> 1)}; const {x, y} = flipOnAxis(itemCtr, cropAxis, {flipV: !!crop.flipV, flipH: !!crop.flipH}); const r = (crop.angle + crop.rotate * QUATER_ANGLE) / HALF_ANGLE * Math.PI; const a = x * Math.cos(r) + y * Math.sin(r); const b = x * Math.sin(r) - y * Math.cos(r); // pos ઝ಴҅ীࢲ CropPos ઝ಴҅۽ ੉ز const cropItemCtr = {x: rtCropBoxLt.x + a, y: rtCropBoxLt.y - b}; const ltCorr = {x: item.width >> 1, y: item.height >> 1}; const cropLt = {x: cropItemCtr.x - ltCorr.x, y: cropItemCtr.y - ltCorr.y}; const result = {x: cropLt.x, y: cropLt.y, rotate: resRotate}; return result; };
109. ? “이걸 이렇게 오래?”
110. :; 희노애락 export type Item = { id: number; src: string; width: number; height: number; custom: boolean; // for preview and canvas position: AnglePosition; flip: Flip; // for crop menu cropPosition: AnglePosition; cropFlip: Flip; };
111. @ “짠”
112. ᚨ “즐겨~”
113. A “많 관 부”
114. 참고 자료 & 이미지 출처 1 ) Web GL ӝୡ https://webglfundamentals.org/webgl/lessons/ko/webgl-fundamentals.html P 14, 15) https://commons.wikimedia.org/wiki/File:WebGL_Logo.svg?uselang=ko
115. E.O.D