Take on me - NationJS

Transcript

1. Take on me
   Eva Ferreira – Nation JS, 2019
   

   View Slide

2. Take on me
   • Catchy tunes
   • Mesmerizing animations
   @evaferreira92
   

   View Slide

3. Mesmerizing
   animations
   My life goal.
   

   View Slide

4. Hi! I’m Eva
   • Front-end Developer by day
   • Protector of useless creativity at night
   • Mozilla Tech Speaker
   • Organizer of CSSConf Argentina
   @evaferreira92
   

   View Slide

5. Hi! I’m Maria Evangelina
   Ferreira Kuzminski
   • Front-end Developer by day
   • Protector of useless creativity at night
   • Mozilla Tech Speaker
   • Organizer of CSSConf Argentina
   @evaferreira92
   

   View Slide

6. What is this, anyway?
   @evaferreira92
   

   View Slide

7. Rotoscoping
   

   View Slide

8. Take on me – A-ha
   1985
   

   View Slide

9. 21st Century Breakdown – Green Day
   2009
   

   View Slide

10. Sheezus – Lily Allen
    2014
    

    View Slide

11. Hard Times – Paramore
    2017
    

    View Slide

12. Caught in the Middle – Paramore
    2018
    

    View Slide

13. Rotoscoping
    1. Record a video
    2. Draw on top
    15 to 25 images per second
    @evaferreira92
    

    View Slide

14. View Slide

15. View Slide

16. View Slide

17. View Slide

18. Animation
    on the web.
    

    View Slide

19. Animation on the web
    • Stop Motion
    • Motion Graphics
    @evaferreira92
    

    View Slide

20. Stop Motion – Rachel Nabors
    

    View Slide

21. Motion Graphics – Chris Gannon
    

    View Slide

22. Can we go further?
    

    View Slide

23. Can we make
    rotoscoping on the
    web?
    No.
    

    View Slide

24. View Slide

25. Interactive media
    Take Rotoscoping idea into the web.
    

    View Slide

26. View Slide

27. View Slide

28. Chroma Key
    

    View Slide

29. Chroma Key
    • Green or blue background
    • Really good light
     Otherwise, colors turn gray-ish
    @evaferreira92
    

    View Slide

30. No natural light
    

    View Slide

31. Chroma key
    with user’s camera
    

    View Slide

32. Chroma key in the browser
    1. Take the user’s camera input
    1. Put it in a video tag
    2. Copy that into a canvas
    3. Process canvas
    1. Remove green or blue
    2. Show background image or video
    @evaferreira92
    

    View Slide

33. 1.
    Take the user’s
    camera input
    

    View Slide

34. Take video input
    @evaferreira92
    navigator.mediaDevices.getUserMedia({video: true})
    .then()
    .catch();
    

    View Slide

35. Put it in a video tag
    @evaferreira92
    
    

    View Slide

36. Put it in a video tag
    @evaferreira92
    .then(function (stream) {
    video.srcObject = stream;
    })
    .catch(function (error) {
    console.log("Error:", error);
    });
    navigator.mediaDevices.getUserMedia({video: true})
    

    View Slide

37. @evaferreira92
    

    View Slide

38. 2.
    Copy video
    into canvas
    

    View Slide

39. Create canvas
    @evaferreira92
    
    c1 = document.getElementById("canvas");
    ctx1 = this.c1.getContext("2d");
    

    View Slide

40. Create the draw function
    @evaferreira92
    function draw () {
    ctx1.drawImage(video, 0, 0, this.width, this.height);
    }
    

    View Slide

41. Update the canvas
    function time_callback () {
    if (video.paused || video.ended) {
    return;
    }
    draw();
    // Update
    setTimeout(function () {
    this.time_callback();
    }, 0);
    };
    

    View Slide

42. @evaferreira92
    

    View Slide

43. 3.
    Process the canvas
    Remove green or blue background
    

    View Slide

44. Pixel manipulation
    

    View Slide

45. What is a pixel?
    

    View Slide

46. @evaferreira92
    

    View Slide

47. @evaferreira92
    Pixel
    

    View Slide

48. RGB Format
    

    View Slide

49. @evaferreira92
    =
    227 45 145
    

    View Slide

50. Alpha Channel
    @evaferreira92
    

    View Slide

51. Ask for pixel information (RGBA)
    • The red value
    • The green value
    • The blue value
    • The Alpha value
    @evaferreira92
    

    View Slide

52. Ask for pixel information (RGBA)
    • The red value
    • The green value
    • The blue value
    • The Alpha value
     It will always return 255
     … Unless we change it
    @evaferreira92
    

    View Slide

53. Making Chroma happen.
    1. Find the green pixels
    2. Remove them
    1. Alpha value of 0
    @evaferreira92
    

    View Slide

54. Find the
    green pixels
    

    View Slide

55. Ask for pixel data
    @evaferreira92
    function findGreen () {
    // Ask for color data
    let frame = this.ctx1.getImageData(0, 0, this.width, this.height);
    ...
    }
    

    View Slide

56. @evaferreira92
    1,228,800
    

    View Slide

57. @evaferreira92
    

    View Slide

58. @evaferreira92
    

    View Slide

59. @evaferreira92
    

    View Slide

60. @evaferreira92
    Alpha value
    

    View Slide

61. @evaferreira92
    R
    G
    B
    A
    R
    G
    B
    A
    R
    G
    B
    A
    R
    G
    B
    A
    R
    G
    B
    A
    

    View Slide

62. @evaferreira92
    } rgba(55, 74, 86, 1)
    

    View Slide

63. @evaferreira92
    1,228,800
    640 x 480 x 4
    =
    

    View Slide

64. Find the green pixels
    @evaferreira92
    function findGreen () {
    ...
    let allData = frame.data.length / 4;
    for (let i = 0; i < allData; i++) {
    let r = frame.data[i * 4 + 0];
    let g = frame.data[i * 4 + 1];
    let b = frame.data[i * 4 + 2];
    if (g > r && g > b) {
    frame.data[i * 4 + 3] = 0;
    }
    }
    }
    

    View Slide

65. @evaferreira92
    // Save all the data of 1.2m pixels
    let allData = frame.data.length / 4;
    // Go through it and identify it
    for (let i = 0; i < allData; i++) {
    let r = frame.data[i * 4 + 0];
    let g = frame.data[i * 4 + 1];
    let b = frame.data[i * 4 + 2];
    // Ask if the greater value is green
    if (g > r && g > b) {
    // Remove opacity
    frame.data[i * 4 + 3] = 0;
    }
    }
    

    View Slide

66. Return the updated data
    @evaferreira92
    function findGreen () {
    let frame = this.ctx1.getImageData(0, 0, this.width, this.height);
    let l = frame.data.length / 4;
    for (let i = 0; i < l; i++) {
    ...
    frame.data[i * 4 + 3] = 0;
    }
    this.ctx1.putImageData(frame, 0, 0);
    return;
    }
    

    View Slide

67. @evaferreira92
    

    View Slide

68. Improve it
    1. Better light
    2. Remove only if the green is > 100
    @evaferreira92
    if (g > r && g > b) {
    if (g > 100 ) {
    frame.data[i * 4 + 3] = 0;
    }
    }
    

    View Slide

69. View Slide

70. @evaferreira92
    

    View Slide

71. Green / Blue dropdown
    @evaferreira92
    Select color
    
    Green
    Blue
    
    

    View Slide

72. Green / Blue dropdown
    @evaferreira92
    if (b > r && b > g) {
    if (b > 100 ) {
    frame.data[i * 4 + 3] = 0;
    }
    }
    

    View Slide

73. Live demo
    

    View Slide

74. Invisibility Cloak
    

    View Slide

75. @evaferreira92
    

    View Slide

76. MOAR
    Pixel Manipulation
    

    View Slide

77. @evaferreira92
    

    View Slide

78. View Slide

79. @evaferreira92
    

    View Slide

80. Copy content into canvases
    @evaferreira92
    
    
    
    

    View Slide

81. Copy content into canvases
    @evaferreira92
    function findColor (color) {
    let frame1 = this.ctx1.getImageData(0, 0, this.width, this.height);
    let frame2 = this.ctx1.getImageData(0, 0, this.width, this.height);
    let frame3 = this.ctx1.getImageData(0, 0, this.width, this.height);
    ...
    this.ctx1.putImageData(frame1, 0, 0);
    this.ctx2.putImageData(frame2, 0, 0);
    this.ctx3.putImageData(frame3, 0, 0);
    return;
    }
    

    View Slide

82. @evaferreira92
    

    View Slide

83. Turn what’s left into pink
    @evaferreira92
    for (let i = 0; i < l; i++) {
    ...
    if (g > r && g > b) {
    frame1.data[i * 4 + 3] = 0;
    frame2.data[i * 4 + 3] = 0;
    frame3.data[i * 4 + 3] = 0;
    } else {
    frame2.data[i * 4 + 0] = 200;
    frame3.data[i * 4 + 0] = 200;
    frame2.data[i * 4 + 2] = 200;
    frame3.data[i * 4 + 2] = 200;
    }
    }
    0 = Red
    1 = Green
    2 = Blue
    3 = Alpha
    

    View Slide

84. @evaferreira92
    

    View Slide

85. @evaferreira92
    

    View Slide

86. @evaferreira92
    

    View Slide

87. @evaferreira92
    

    View Slide

88. @evaferreira92
    

    View Slide

89. Endless
    possibilities.
    Runs in your browser with JavaScript.
    No other software required.
    

    View Slide

90. Protect the useless.
    @evaferreira92
    

    View Slide

91. Thank you :)
    • Demos & links: https://github.com/evaferreira/nationjs
    • Slides: https://speakerdeck.com/evaferreira
    • Green screen online*: https://chroma-code.netlify.com/
    • @evaferreira92
    

    View Slide