Upgrade to Pro — share decks privately, control downloads, hide ads and more …

LINE Creative Lab : Create Catchy Ads with HTML5 Canvas

LINE DEVDAY 2021
PRO
November 11, 2021
81

LINE Creative Lab : Create Catchy Ads with HTML5 Canvas

LINE DEVDAY 2021
PRO

November 11, 2021
Tweet

More Decks by LINE DEVDAY 2021

Transcript

  1. None
  2. Agenda - Introduction of LINE Creative Lab - Project design

    - Image editor - Video editor - Our next steps
  3. Introduction of LINE Creative Lab

  4. What is LINE Creative Lab?

  5. How LINE Creative Lab was born Ads on LINE services

  6. How LINE Creative Lab was born Before June30 2020 Import

    creative Designer Ads Hire Advertiser
  7. How LINE Creative Lab was born From June30 2020 Export

    creatives Advertiser Create/Edit template
  8. LINE Creative Lab’s main features - Image editor - Video

    editor - Optimize for ads - For LINE users
  9. LINE Creative Lab’s main features Image layer Shape layer Logo

    layer Text layer Image editor
  10. LINE Creative Lab’s main features Scene Transition Sticker Video editor

  11. LINE Creative Lab’s main features Designed templates Image designed templates

    Video designed templates
  12. Project design

  13. Overview Designer Designed template Editor Advertisement Create User Edit Use

    Render
  14. Editor general architecture Tools Preview Template Edit Save Advertisement Render

  15. Image editor

  16. Content overview What do we have in image editor section?

    - Image editor architecture - Layers - Technical challenges
  17. Image editor architecture Designer Designed template Image editor User template

    (Image) Create User Edit Use Save Image advertisement Render
  18. Layers Image, Logo Upload logo Add logo to image ads

  19. Layers Text Choose font Add text to image ads

  20. Layers Shape Choose shape Add shape to image ads

  21. Technical challenges Content overview Shape Fonts loading

  22. Technical challenges Fonts loading > Large size problem Fonts with

    large size 15 text layers X 16 MB = 240 MB
  23. Technical challenges Fonts loading > Solutions Font server 1st Font

    subsets 2nd
  24. Technical challenges Fonts loading > Font server solution Font server

    Image of text Send data Generate Draw
  25. Technical challenges Fonts loading > Solutions Font server 1st Font

    subsets 2nd
  26. Technical challenges Fonts loading > Font subsets solution Split original

    font to subsets Register each subset by using font-face
  27. Technical challenges Fonts loading > Font subsets solution Split original

    font to subsets 15 text layers X 50 KB = 750 KB
  28. Technical challenges Fonts loading > Pros and Cons > Font

    server solution Cons Pros
  29. Technical challenges Fonts loading > Pros and Cons > Font

    subsets solution Cons Pros
  30. Technical challenges Fonts loading > Conclusion Font server 1st Font

    subsets 2nd
  31. Technical challenges Content overview Shape Fonts loading

  32. Technical challenges Shape

  33. Technical challenges Shape SVG Mathematical formulas Browser

  34. Technical challenges Shape > Problem with SVG SVG shape SVG

    shape with stroke Wrong shape
  35. Technical challenges Shape > Path2D solution Path APIs Path2D Shape

    Use Path2D – Browser compatibility Draw Canvas
  36. Technical challenges Shape > Path2D solution SVG Path2D Canvas Convert

    to Draw SVG shape with large stroke
  37. Video editor

  38. Content overview What do we have in video editor section?

    - Video editor architecture - Layers - Video rendering - Technical challenges
  39. Video editor architecture Designer Designed template Video editor User template

    (Video) Create User Edit Use Save Send to Browser Video advertisement Render Encoding server Video editor architecture Designer Designed template Video editor User template (Video) Create User Edit Use Save Send to Browser Video advertisement Render Encoding server
  40. Layers Image, Logo Upload image Add image to video scene

  41. Layers Text Text setting Add text to video scene

  42. Layers Sticker Choose sticker Add sticker to video scene

  43. Video rendering Draw and Encode methods - Canvas: 2d context

    - Canvas: WebGL context Drawing - MediaRecorder API - FFMPEG - Hardware encoder Encoding
  44. Video rendering Drawing > WebGL overview Application WebGL APIs GPU

    Call Communicate
  45. Video rendering Drawing > WebGL overview Application GPU Upload x

    N Download x N Application GPU Upload large data 1st slow case 2nd slow case
  46. Video rendering Drawing > Apply WebGL in CLAB - Text

    changes are expensive because we have to convert text to image - So text changes should be reduced Our situation Two properties of WebGL - Upload all resource to GPU memory before drawing - All resources must be images
  47. Video rendering Encoding > MediaRecorder API Canvas MediaRecorder Video captureStream

    Record MediaRecorder - Browser compatibility
  48. Video rendering Encoding > FFMPEG Raw frames FFMPEG Video Load

    Encode
  49. Video rendering Encoding > Hardware encoder Node.js server N-API binding

    Low-level SDK (C++) HW encoder
  50. Technical challenges Content overview

  51. Technical challenges Encoding speed > MediaRecorder API 30s MediaStream 30s

    Video Play Wait 30s Switch tab = Postpone Close tab = Cancel MediaRecorder
  52. Technical challenges Encoding speed > Server-side rendering Node.js Browser JS

    drawing codebase node-canvas headless-gl Canvas WebGL
  53. Technical challenges Encoding speed > FFMPEG encoding flow Video frames

    GPU memory SSD or RAM SW encoder HW encoder H264 video Or FFMPEG Encode Read Copy video frames from GPU memory to SSD/RAM is a bottleneck and make the encoding process slower Copy Save Node.js server Draw 6 GB 6 GB
  54. Technical challenges Encoding speed > HW encoder & low-level SDK

    Memory area for drawing Memory area for encoding GPU memory HW encoder Video frames H264 video Node.js server Draw Encode Copy uncompressed data by low-level SDK
  55. Technical challenges MediaRecorder 30s Node.js + SDK + HW 2s

    Node.js + FFMPEG 7s Encoding speed > How long does it take to create a 30 seconds video?
  56. Technical challenges Encoding speed > Conclusion MediaRecorder 30s Node.js +

    SDK + HW 2s Node.js + FFMPEG 7s We selected this solution for our users
  57. Technical challenges Content overview

  58. Technical challenges Difference of text: Client rendering vs Server rendering

    (MacOS) server Difference
  59. Technical challenges Difference of text: Client rendering vs Server rendering

    (MacOS) (Linux) Difference Headless
  60. Technical challenges Content overview

  61. Technical challenges Support new effects - Video effects = GLSL

    code - gl-transition = open source - New effects = increase dev time
  62. Our next steps

  63. Our next steps In the future, we will - Apply

    more GLSL effects - Support video layer - Solve difference of text issue - Scale encoding server
  64. Thank you