Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Video edition with CSS & JavaScript

Video edition with CSS & JavaScript

32b23faedcb33ad2079ad33658ac56bb?s=128

Eva Ferreira

June 19, 2019
Tweet

Transcript

  1. Video Editing with CSS and JavaScript Eva Ferreira – Smashing

    TV
  2. Hi! I’m Eva

  3. Hi! I’m Eva • Front-end Developer by day • Protector

    of useless creativity at night • Mozilla Tech Speaker • Organizer of CSSConf Argentina @evaferreira92
  4. Hi! I’m Maria Evangelina Ferreira Kuzminski • Front-end Developer by

    day • Protector of useless creativity at night • Mozilla Tech Speaker • Organizer of CSSConf Argentina @evaferreira92
  5. @evaferreira92

  6. Rotoscoping

  7. Take on me – A-ha 1985

  8. 21st Century Breakdown – Green Day 2009

  9. Sheezus – Lily Allen 2014

  10. Hard Times – Paramore 2017

  11. Caught in the Middle – Paramore 2018

  12. Rotoscoping 1. Record a video 2. Draw on top 15

    to 25 images per second @evaferreira92
  13. None
  14. None
  15. None
  16. None
  17. Animation on the web.

  18. Traditional Web • Stop Motion  Rachel Nabors • Motion

    Graphics  Chris Gannon @evaferreira92
  19. Can we go further?

  20. Can we make rotoscoping on the web? No.

  21. None
  22. Interactive media Take Rotoscoping idea into the web.

  23. Rotoscoping = Animation above video

  24. With CSS or with JavaScript

  25. The CSS Version 1. “Normal” video 2. “Animation” video @evaferreira92

  26. https://vimeo.com/58933055

  27. https://vimeo.com/49581518

  28. Mix them

  29. Blend them

  30. Blend Modes

  31. mix-blend-mode: luminosity;

  32. Make it interactive

  33. Add clip-path and filters

  34. Interactive wow-ness Ok-ish browser support

  35. https://caniuse.com/#feat=css-mixblendmode

  36. Let’s try with JavaScript now

  37. None
  38. None
  39. Chroma Key

  40. Chroma Key • Green or blue background • Really good

    light @evaferreira92
  41. No natural light

  42. Chroma key with user’s camera

  43. Chroma key in the browser 1. Take the user’s camera

    input 1. Put it in a video tag 2. Copy that into a canvas 3. Process canvas 1. Remove green or blue 2. Show background image or video @evaferreira92
  44. 1. Take the user’s camera input

  45. Take video input @evaferreira92 navigator.mediaDevices.getUserMedia({video: true}) .then() .catch();

  46. Put it in a video tag @evaferreira92 <video id="video" controls></video>

  47. Put it in a video tag @evaferreira92 .then(function (stream) {

    video.srcObject = stream; }) .catch(function (error) { console.log("Error:", error); }); navigator.mediaDevices.getUserMedia({video: true})
  48. @evaferreira92

  49. 2. Copy video into canvas

  50. Create canvas @evaferreira92 <canvas id="canvas" class="canvas"></canvas> c1 = document.getElementById("canvas"); ctx1

    = this.c1.getContext("2d");
  51. Create the draw function @evaferreira92 function draw () { ctx1.drawImage(video,

    0, 0, this.width, this.height); }
  52. Update the canvas function time_callback () { if (video.paused ||

    video.ended) { return; } draw(); // Update setTimeout(function () { this.time_callback(); }, 0); };
  53. @evaferreira92

  54. 3. Process the canvas Remove green or blue background

  55. Pixel manipulation

  56. What is a pixel?

  57. @evaferreira92

  58. @evaferreira92 Pixel

  59. RGB Format

  60. @evaferreira92 = 227 45 145

  61. Alpha Channel @evaferreira92

  62. Ask for pixel information (RGBa) • The red value •

    The green value • The blue value • The Alpha value @evaferreira92
  63. Ask for pixel information (RGBa) • The red value •

    The green value • The blue value • The Alpha value  It will always return 255  … Unless we change it @evaferreira92
  64. Making Chroma happen. 1. Find the green pixels 2. Remove

    them 1. Alpha value of 0 @evaferreira92
  65. Find the green pixels

  66. Ask for pixel data @evaferreira92 function findGreen () { //

    Ask for color data let frame = this.ctx1.getImageData(0, 0, this.width, this.height); ... }
  67. @evaferreira92

  68. @evaferreira92 1,228,800 640 x 480 x 4 =

  69. @evaferreira92

  70. @evaferreira92

  71. @evaferreira92

  72. @evaferreira92 Alpha value

  73. @evaferreira92 R G B A R G B A R

    G B A R G B A R G B A
  74. @evaferreira92 } rgba(55, 74, 86, 1)

  75. Find the green pixels @evaferreira92 function findGreen () { ...

    let allData = frame.data.length / 4; for (let i = 0; i < allData; i++) { ... } }
  76. @evaferreira92 // Save all the data of 1.2m pixels let

    allData = frame.data.length / 4; // Go through it and identify it for (let i = 0; i < allData; i++) { let r = frame.data[i * 4 + 0]; let g = frame.data[i * 4 + 1]; let b = frame.data[i * 4 + 2]; // Ask if the greatest value is green if (g > r && g > b) { // Remove opacity frame.data[i * 4 + 3] = 0; } }
  77. Return the updated data @evaferreira92 function findGreen () { let

    frame = this.ctx1.getImageData(0, 0, this.width, this.height); let l = frame.data.length / 4; for (let i = 0; i < l; i++) { ... frame.data[i * 4 + 3] = 0; } this.ctx1.putImageData(frame, 0, 0); return; }
  78. @evaferreira92

  79. Improve it 1. Better light 2. Remove only if the

    green is > 100 @evaferreira92 if (g > r && g > b) { if (g > 100 ) { frame.data[i * 4 + 3] = 0; } }
  80. None
  81. @evaferreira92

  82. Live demo

  83. Invisibility Cloak

  84. @evaferreira92

  85. MOAR Pixel Manipulation

  86. @evaferreira92

  87. None
  88. @evaferreira92

  89. Copy content into canvases @evaferreira92 <canvas id="canvas1" class="canvas canvas1"></canvas> <canvas

    id="canvas2" class="canvas canvas2"></canvas> <canvas id="canvas3" class="canvas canvas3"></canvas>
  90. Copy content into canvases @evaferreira92 function findColor (color) { let

    frame1 = this.ctx1.getImageData(0, 0, this.width, this.height); let frame2 = this.ctx1.getImageData(0, 0, this.width, this.height); let frame3 = this.ctx1.getImageData(0, 0, this.width, this.height); ... this.ctx1.putImageData(frame1, 0, 0); this.ctx2.putImageData(frame2, 0, 0); this.ctx3.putImageData(frame3, 0, 0); return; }
  91. @evaferreira92

  92. Turn what’s left into pink @evaferreira92 for (let i =

    0; i < l; i++) { ... if (g > r && g > b) { frame1.data[i * 4 + 3] = 0; frame2.data[i * 4 + 3] = 0; frame3.data[i * 4 + 3] = 0; } else { frame2.data[i * 4 + 0] = 200; frame3.data[i * 4 + 0] = 200; frame2.data[i * 4 + 2] = 200; frame3.data[i * 4 + 2] = 200; } } 0 = Red 1 = Green 2 = Blue 3 = Alpha
  93. @evaferreira92

  94. @evaferreira92

  95. @evaferreira92

  96. @evaferreira92

  97. Endless possibilities. Runs in your browser with JavaScript. No other

    software required.
  98. Protect the useless. @evaferreira92

  99. Thank you :) • Demos & links: https://github.com/evaferreira/smashing-tv • Green

    screen online: https://chroma-code.netlify.com/ • @evaferreira92