Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Frames Per Second (jQuery EU 2014)

9fb120a88c85d93be416ed62b8ab5e16?s=47 Paul Bakaus
February 28, 2014

Frames Per Second (jQuery EU 2014)

How does the human eye actually work, and how many frames can it process? What is the difference between display refresh rates and frame rate? What’s the deal about motion blur? Why does it matter?What is jitter, and how can I fix it? What’s the difference between responsiveness (perceived performance) and actual raw performance?

Find out.

9fb120a88c85d93be416ed62b8ab5e16?s=128

Paul Bakaus

February 28, 2014
Tweet

Transcript

  1. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Frames Per Second
  2. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

  3. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Clueless programmer: “Whatevs, 24fps works for movies, works for me, dumbass!”
  4. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Hate knowledge?! Want action? Take a quick nap.
  5. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Shutter speed Motion blur Persistence of Vision Phi Phenomenon Refresh Rate HFR Flicker Fusion Threshold Subfield Drive 2-3 pulldown Jitter requestAnimationFrame vsync Input lag Perceived performance
  6. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Shutter speed Motion blur Persistence of Vision Phi Phenomenon Refresh Rate HFR Flicker Fusion Threshold Subfield Drive 2-3 pulldown Jitter requestAnimationFrame vsync Input lag Perceived performance
  7. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Frames.. per second? !7
  8. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    When we started to record and project film, there was no scientific evidence behind many discoveries - just trial and error First cameras recorded at variable FPS, and projected at variable FPS (hand-controlled), usually at 16-24 fps Through lots of trial and error, and because sound playback had to be uniform and steady, people settled on 24fps, and it was deemed “good enough”, and film was expensive
  9. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    But what is the optimum frame rate?* * Trick question
  10. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !10 The eye is not a camera.
  11. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !11 Motion perception is the process of inferring the speed and direction of elements in a scene based on visual, vestibular and proprioceptive inputs. Although this process appears straightforward to most observers, it has proven to be a difficult problem from a computational perspective, and extraordinarily difficult to explain in terms of neural processing.
  12. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !12 So why do frames work, then?
  13. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !13 Persistence of Vision Persistence of vision is the phenomenon of the eye by which an afterimage is thought to persist for approximately one twenty-fifth of a second on the retina. Phi Phenomenon The phi phenomenon is the optical illusion of perceiving continuous motion between separate objects viewed rapidly in succession.
  14. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !13 Persistence of Vision Persistence of vision is the phenomenon of the eye by which an afterimage is thought to persist for approximately one twenty-fifth of a second on the retina. Phi Phenomenon The phi phenomenon is the optical illusion of perceiving continuous motion between separate objects viewed rapidly in succession.
  15. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !14 No frame rate will ever be optimal. But we can get closer..
  16. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    • 10 fps – absolute minimum • < 16 fps – causes visible stutter, headaches • 24 fps – tolerable, cost-efficient Crappy frame rates 12 fps
  17. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    • 10 fps – absolute minimum • < 16 fps – causes visible stutter, headaches • 24 fps – tolerable, cost-efficient Crappy frame rates 12 fps
  18. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    • >= 30 fps – decent, but not lifelike • 48 fps (HFR) – good, but not enough • 60 fps – sweet spot! • > 67 fps – does not improve the picture further* Good frame rates
  19. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

  20. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

  21. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !18 Most people, after watching Hobbit movie.
  22. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !19 “Film look” is a term that exists, sadly. http://en.wikipedia.org/wiki/Film_look
  23. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !20
  24. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Hz vs. FPS !21
  25. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !22 The refresh rate is the number of times in a second that a display hardware updates its buffer. This is distinct from the measure of frame rate in that the refresh rate includes the repeated drawing of identical frames, while frame rate measures how often a video source can feed an entire frame of new data to a display. http://en.wikipedia.org/wiki/Refresh_rate
  26. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !23 Describes the number of times the image on screen is fully redrawn / refreshed! The refresh rate can be terribly important or not at all important, depending on the display type and frame rate.
  27. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !24 Display types?
  28. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !25 Movie projectors
  29. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !26 CRT’s
  30. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !27 LCD - No flicker, no problem!
  31. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !28 So high refresh rates in LCDs are a marketing gag?!
  32. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !29 Mostly, yes. *
  33. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !30 Plasma
  34. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !31 “Activate the sub field drive!”
  35. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !32 To recap:
  36. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !33 Ever wondered why movies play back so crappy on your laptop?
  37. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !34 FPS and Hz are no mix and match!
  38. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !35 Telecine The process of transferring (converting) motion picture film into video. Three common methods Lazy method: Speed up Crazy method: 2:3 pulldown (PAL: 2:2) Judder method: 5th frame 2x
  39. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !36 German SEE Edition of Lord of the Rings had major sound fail:
  40. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !36 German SEE Edition of Lord of the Rings had major sound fail: (Movie version)
  41. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !36 German SEE Edition of Lord of the Rings had major sound fail: (BluRay Version)
  42. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !37 Speed change is not the best idea.
  43. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !38
  44. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !39
  45. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !40
  46. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !41 Easy! Just play every 5th frame twice! Nobody will notice, right?
  47. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !42
  48. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Jitter* !43 * Judder, Jank
  49. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !44 Jitter is the undesired deviation from true periodicity of an assumed periodic signal in electronics and telecommunications, often in relation to a reference clock source. Wikipedia
  50. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !45 Jitter is what happens when an animation is being played at variable frame rate, rather than a fixed frame rate.
  51. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !46 “Sweet then, I’ll settle at a steady 30fps for my game, thanks Paul!”
  52. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !47 “Sweet then, I’ll settle at a steady 30fps for my game, thanks Paul!” NOT SO FAST, YOUNG PADAWAN!
  53. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !48 Ever wondered why your site looks so shitty at 24fps compared to a movie?
  54. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Meet Motion Blur. !49 Photo by E01 on Flickr
  55. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !50 Motion blur is the apparent streaking of rapidly moving objects in a still image or a sequence of images such as a movie or animation. It results when the image being recorded changes during the recording of a single frame, either due to rapid movement or long exposure. Wikipedia
  56. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !51 via http://freespace.virgin.net/hugo.elias/graphics/x_motion.htm
  57. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !52 Without With
  58. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !53 The Rotary Disc Shutter
  59. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !53 The Rotary Disc Shutter
  60. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !54 Ironically, Flicker reduces motion blur: http://www.testufo.com/ #test=blackframes&count=2&equalizer=1
  61. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !55 Eye-induced motion blur in LCDs Source: http://msdn.microsoft.com/en-us/windows/hardware/gg463407.aspx
  62. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !56 The web is missing motion blur: https://dl.dropboxusercontent.com/u/ 8554242/dmitri/projects/MotionBlurDemo/ MotionBlurDemo.html
  63. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !57 Why don’t we just add it?
  64. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !58 Temporal anti-aliasing is expensive:
  65. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !59 60 fps, it is.
  66. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !60 Mind = Blown
  67. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !61 Photo by E01 on Flickr Hands on
  68. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !62 requestAnimationFrame ftw! window.requestAnimationFrame(function(/* time */ time) { // time ~= +new Date // the unix time }); syncs with vsync.
  69. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !63 Follow best rendering practices: http://calendar.perfplanet.com/2013/ the-runtime-performance-checklist/
  70. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !64 Understand your bottlenecks: Chances are JavaScript does not cause your perf issues!
  71. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !65 Debugging is hard.
  72. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !66 Measuring FPS accurately is hard.
  73. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !67 Remember: You have ~ 16.7ms for all your stuff – but it is shared with the browser!
  74. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    !68 Digging further
  75. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Input lag “In video games, input lag is either the delay between the television or monitor receiving a signal and it being displayed on the screen, or the delay between pressing a button and seeing the game react.”
  76. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Input lag > 200ms = unresponsive!! > 100ms = perceived as waiting! < 100ms = feels fast
  77. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Perceived performance “Perceived performance, in computer engineering, refers to how quickly a software feature appears to perform its task. The concept applies mainly to user acceptance aspects.”
  78. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Progress Bars http://www.newscientist.com/ video/76695476001- progress-bar-illusion.html Increase the Number of Pulsations - “The progress bar with increasing pulsation was more likely to be perceived as having a shorter duration” Accelerate the Progress and Avoid Pauses at the End - Progress bars “with the fastest progress occurring near the end of the process” were perceived faster than progress bars “with pauses near the process conclusion”
  79. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Progress Bars http://www.newscientist.com/ video/76695476001- progress-bar-illusion.html Increase the Number of Pulsations - “The progress bar with increasing pulsation was more likely to be perceived as having a shorter duration” Accelerate the Progress and Avoid Pauses at the End - Progress bars “with the fastest progress occurring near the end of the process” were perceived faster than progress bars “with pauses near the process conclusion”
  80. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Other mixed advice Hide waiting times between transitions Render the layout before content Lazy load content in the background Choose the right easing
  81. Frames Per Second 02/28/2014 Slide http://paulbakaus.com | @pbakaus | +PaulBakaus

    Thank you for listening! See you next time! Stay in touch!