Take on me - NationJS

Transcript

1. Take on me Eva Ferreira – Nation JS, 2019
2. Take on me • Catchy tunes • Mesmerizing animations @evaferreira92
3. Mesmerizing animations My life goal.
4. Hi! I’m Eva • Front-end Developer by day • Protector of useless creativity at night • Mozilla Tech Speaker • Organizer of CSSConf Argentina @evaferreira92
5. Hi! I’m Maria Evangelina Ferreira Kuzminski • Front-end Developer by day • Protector of useless creativity at night • Mozilla Tech Speaker • Organizer of CSSConf Argentina @evaferreira92
6. What is this, anyway? @evaferreira92
7. Rotoscoping
8. Take on me – A-ha 1985
9. 21st Century Breakdown – Green Day 2009
10. Sheezus – Lily Allen 2014
11. Hard Times – Paramore 2017
12. Caught in the Middle – Paramore 2018
13. Rotoscoping 1. Record a video 2. Draw on top 15 to 25 images per second @evaferreira92
18. Animation on the web.
19. Animation on the web • Stop Motion • Motion Graphics @evaferreira92
20. Stop Motion – Rachel Nabors
21. Motion Graphics – Chris Gannon
22. Can we go further?
23. Can we make rotoscoping on the web? No.
25. Interactive media Take Rotoscoping idea into the web.
28. Chroma Key
29. Chroma Key • Green or blue background • Really good light  Otherwise, colors turn gray-ish @evaferreira92
30. No natural light
31. Chroma key with user’s camera
32. Chroma key in the browser 1. Take the user’s camera input 1. Put it in a video tag 2. Copy that into a canvas 3. Process canvas 1. Remove green or blue 2. Show background image or video @evaferreira92
33. 1. Take the user’s camera input
34. Take video input @evaferreira92 navigator.mediaDevices.getUserMedia({video: true}) .then() .catch();
35. Put it in a video tag @evaferreira92 <video id="video" controls></video>
36. Put it in a video tag @evaferreira92 .then(function (stream) { video.srcObject = stream; }) .catch(function (error) { console.log("Error:", error); }); navigator.mediaDevices.getUserMedia({video: true})
37. @evaferreira92
38. 2. Copy video into canvas
39. Create canvas @evaferreira92 <canvas id="canvas" class="canvas"></canvas> c1 = document.getElementById("canvas"); ctx1 = this.c1.getContext("2d");
40. Create the draw function @evaferreira92 function draw () { ctx1.drawImage(video, 0, 0, this.width, this.height); }
41. Update the canvas function time_callback () { if (video.paused || video.ended) { return; } draw(); // Update setTimeout(function () { this.time_callback(); }, 0); };
42. @evaferreira92
43. 3. Process the canvas Remove green or blue background
44. Pixel manipulation
45. What is a pixel?
46. @evaferreira92
47. @evaferreira92 Pixel
48. RGB Format
49. @evaferreira92 = 227 45 145
50. Alpha Channel @evaferreira92
51. Ask for pixel information (RGBA) • The red value • The green value • The blue value • The Alpha value @evaferreira92
52. Ask for pixel information (RGBA) • The red value • The green value • The blue value • The Alpha value  It will always return 255  … Unless we change it @evaferreira92
53. Making Chroma happen. 1. Find the green pixels 2. Remove them 1. Alpha value of 0 @evaferreira92
54. Find the green pixels
55. Ask for pixel data @evaferreira92 function findGreen () { // Ask for color data let frame = this.ctx1.getImageData(0, 0, this.width, this.height); ... }
56. @evaferreira92 1,228,800
57. @evaferreira92
58. @evaferreira92
59. @evaferreira92
60. @evaferreira92 Alpha value
61. @evaferreira92 R G B A R G B A R G B A R G B A R G B A
62. @evaferreira92 } rgba(55, 74, 86, 1)
63. @evaferreira92 1,228,800 640 x 480 x 4 =
64. Find the green pixels @evaferreira92 function findGreen () { ... let allData = frame.data.length / 4; for (let i = 0; i < allData; i++) { let r = frame.data[i * 4 + 0]; let g = frame.data[i * 4 + 1]; let b = frame.data[i * 4 + 2]; if (g > r && g > b) { frame.data[i * 4 + 3] = 0; } } }
65. @evaferreira92 // Save all the data of 1.2m pixels let allData = frame.data.length / 4; // Go through it and identify it for (let i = 0; i < allData; i++) { let r = frame.data[i * 4 + 0]; let g = frame.data[i * 4 + 1]; let b = frame.data[i * 4 + 2]; // Ask if the greater value is green if (g > r && g > b) { // Remove opacity frame.data[i * 4 + 3] = 0; } }
66. Return the updated data @evaferreira92 function findGreen () { let frame = this.ctx1.getImageData(0, 0, this.width, this.height); let l = frame.data.length / 4; for (let i = 0; i < l; i++) { ... frame.data[i * 4 + 3] = 0; } this.ctx1.putImageData(frame, 0, 0); return; }
67. @evaferreira92
68. Improve it 1. Better light 2. Remove only if the green is > 100 @evaferreira92 if (g > r && g > b) { if (g > 100 ) { frame.data[i * 4 + 3] = 0; } }
70. @evaferreira92
71. Green / Blue dropdown @evaferreira92 <label for="colorFilter">Select color</label> <select id="colorFilter"> <option>Green</option> <option>Blue</option> </select>
72. Green / Blue dropdown @evaferreira92 if (b > r && b > g) { if (b > 100 ) { frame.data[i * 4 + 3] = 0; } }
73. Live demo
74. Invisibility Cloak
75. @evaferreira92
76. MOAR Pixel Manipulation
77. @evaferreira92
79. @evaferreira92
80. Copy content into canvases @evaferreira92 <canvas id="canvas1" class="canvas canvas1"></canvas> <canvas id="canvas2" class="canvas canvas2"></canvas> <canvas id="canvas3" class="canvas canvas3"></canvas>
81. Copy content into canvases @evaferreira92 function findColor (color) { let frame1 = this.ctx1.getImageData(0, 0, this.width, this.height); let frame2 = this.ctx1.getImageData(0, 0, this.width, this.height); let frame3 = this.ctx1.getImageData(0, 0, this.width, this.height); ... this.ctx1.putImageData(frame1, 0, 0); this.ctx2.putImageData(frame2, 0, 0); this.ctx3.putImageData(frame3, 0, 0); return; }
82. @evaferreira92
83. Turn what’s left into pink @evaferreira92 for (let i = 0; i < l; i++) { ... if (g > r && g > b) { frame1.data[i * 4 + 3] = 0; frame2.data[i * 4 + 3] = 0; frame3.data[i * 4 + 3] = 0; } else { frame2.data[i * 4 + 0] = 200; frame3.data[i * 4 + 0] = 200; frame2.data[i * 4 + 2] = 200; frame3.data[i * 4 + 2] = 200; } } 0 = Red 1 = Green 2 = Blue 3 = Alpha
84. @evaferreira92
85. @evaferreira92
86. @evaferreira92
87. @evaferreira92
88. @evaferreira92
89. Endless possibilities. Runs in your browser with JavaScript. No other software required.
90. Protect the useless. @evaferreira92
91. Thank you :) • Demos & links: https://github.com/evaferreira/nationjs • Slides: https://speakerdeck.com/evaferreira • Green screen online*: https://chroma-code.netlify.com/ • @evaferreira92