Introduction to VP8

Transcript

1. Introduction to VP8 郭⾄至軒 (KuoE0) kuoe0.tw@gmail.com
2. Latest update: Nov 19, 2013, Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0) http://creativecommons.org/licenses/by-sa/3.0/
3. Situation
4. web m
5. web m
6. Video Codec VP8
7. An Open Source Codec
8. Developed by On2 Technology
9. Developed by On2 Technology February, 2010
10. Acquired by Google February, 2010
11. Patent
12. web m
13. March, 2013 web m
14. Royalty-Free Terms March, 2013 web m
15. Successor VP9
16. Successor VP9 May 15, 2013
17. Feature
18. focus on Internet web-based application
19. Low Bandwidth Requirement Image Quality: watchable (PSNR: ~30dB) visually lossless (PSNR: ~45dB)
20. Heterogeneous Client Hardware
21. Heterogeneous Client Hardware
22. Heterogeneous Client Hardware Efficient Implementations
23. Web Video Format YUV 420 color sampling 8 bit per channel depth Up to 16383 × 16383 pixels
24. Processing Flow
25. Coding Predict Transform + Quantize Entropy Code Loop Filter
26. Decoding Entropy Decode Predict Dequantize+Inverse Transform Loop Filter
27. Reference Frame
28. Golden Frame Last Frame Alternate Frame Reference Frame
29. Golden Frame Last Frame Alternate Frame At most 3 reference frames in VP8.
30. Last Frame
31. Last Frame
32. Last Frame
33. Last Frame Current Frame
34. Golden Frame Choose an arbitrary frame in the past. Define a number of flags to notify decoder when and how to update this buffer.
35. Golden Frame Choose an arbitrary frame in the past. Define a number of flags to notify decoder when and how to update this buffer.
38. set as the golden frame
39. Golden Frame Golden Frame
40. Golden Frame Golden Frame
41. Golden Frame Golden Frame
42. Golden Frame Golden Frame
43. Golden Frame Golden Frame
44. Golden Frame Golden Frame
45. Golden Frame Golden Frame
46. Golden Frame Golden Frame
47. Golden Frame Golden Frame
48. Reconstruct moving object background
49. Alternate Frame Other Frame Alternate Frame
50. Alternate Frame Other Frame Alternate Frame decode show
51. Alternate Frame Other Frame Alternate Frame decode show decode show
52. Alternate Frame Other Frame Alternate Frame decode show decode show store beneficial information
53. Construct from multi-frame
54. Construct from multi-frame
55. Construct from multi-frame
56. Construct from multi-frame Alternate Frame
57. Typical Frame I B B P B B P B B I B B P
58. VP8 L G A G G G G G L G G G A G L
59. Prediction
60. Intra Prediction Inter Prediction use data within a single video frame use data from previously encoded frames
61. Intra Prediction Luma Luma Chroma
62. Intra Prediction Luma Luma Chroma 16 4 8
63. H_PRED (horizontal prediction) V_PRED (vertical prediction) DC_PRED (DC prediction) TM_PRED (TrueMotion prediction) Four Prediction Modes:
64. Horizontal Prediction Fills each column of the block with a copy of the left column. a b c d e f g h i j k l m n o p q r s t u v w x y A B C D E F G H I J K L M N O P Q R S T U V W X Y
65. Horizontal Prediction Fills each column of the block with a copy of the left column. a b c d e f g h i j k l m n o p q r s t u v w x y A B C D E F G H I J K L M N O P Q R S T U V W X Y e j o t y
66. Horizontal Prediction Fills each column of the block with a copy of the left column. a b c d e f g h i j k l m n o p q r s t u v w x y A B C D E F G H I J K L M N O P Q R S T U V W X Y e j o t y e e e e e j j j j j o o o o o t t t t t y y y y y
67. Vertical Prediction Fills each row of the block with a copy of the above row. a b c d e f g h i j k l m n o p q r s t u v w x y A B C D E F G H I J K L M N O P Q R S T U V W X Y
68. Vertical Prediction Fills each row of the block with a copy of the above row. a b c d e f g h i j k l m n o p q r s t u v w x y A B C D E F G H I J K L M N O P Q R S T U V W X Y U V W X Y
69. Vertical Prediction Fills each row of the block with a copy of the above row. a b c d e f g h i j k l m n o p q r s t u v w x y A B C D E F G H I J K L M N O P Q R S T U V W X Y U V W X Y U V W X Y U V W X Y U V W X Y U V W X Y U V W X Y
70. DC Prediction Fills the block with a single value using the average of the pixels in the above row and the left column. a b c d e f g h i j k l m n o p q r s t u v w x y A B C D E F G H I J K L M N O P Q R S T U V W X Y
71. DC Prediction Fills the block with a single value using the average of the pixels in the above row and the left column. a b c d e f g h i j k l m n o p q r s t u v w x y A B C D E F G H I J K L M N O P Q R S T U V W X Y U V W X Y e j o t y Z = (U + V + W + X + Y + e + j + o + t + y) ÷ 10
72. DC Prediction Fills the block with a single value using the average of the pixels in the above row and the left column. a b c d e f g h i j k l m n o p q r s t u v w x y A B C D E F G H I J K L M N O P Q R S T U V W X Y U V W X Y e j o t y Z = (U + V + W + X + Y + e + j + o + t + y) ÷ 10 Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z Z
73. * * * * L0 * * * * L1 * * * * L2 * * * * L3 * * * * L4 * * * * * * * * * * * * * * * * * * * * A0 A1 A2 A3 A4 TrueMotion Prediction Horizontal differences between pixels in above row and vertical differences between pixels in left column are propagated (starting from C). * * * * * * * * * * * * * * * * * * * * * * * * C
74. * * * * L0 * * * * L1 * * * * L2 * * * * L3 * * * * L4 * * * * * * * * * * * * * * * * * * * * A0 A1 A2 A3 A4 A0 A1 A2 A3 A4 L0 L1 L2 L3 L4 TrueMotion Prediction Horizontal differences between pixels in above row and vertical differences between pixels in left column are propagated (starting from C). * * * * * * * * * * * * * * * * * * * * * * * * C C Xij = Ai + Lj - C
75. * * * * L0 * * * * L1 * * * * L2 * * * * L3 * * * * L4 * * * * * * * * * * * * * * * * * * * * A0 A1 A2 A3 A4 A0 A1 A2 A3 A4 L0 L1 L2 L3 L4 TrueMotion Prediction Horizontal differences between pixels in above row and vertical differences between pixels in left column are propagated (starting from C). * * * * * * * * * * * * * * * * * * * * * * * * C C Xij = Ai + Lj - C Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij Xij
76. Inter Prediction As mentioned above...
77. Inter Prediction Golden Frame Last Frame Alternate Frame
78. Motion Vector Reusing vectors from neighboring macroblocks. Flexible partitioning of a macroblock into sub- blocks.
79. Sub-pixel Interpolation Quarter pixel accurate motion vectors for luma pixels. High performance six-tap interpolation filters. [3, -16, 77, 77, -16, 3]/128 for 1⁄2 pixel positions [2, -11, 108, 36, -8, 1]/128 for 1⁄4 pixel positions [1, -8, 36, 108, -11, 2]/128 for 3⁄4 pixel positions
80. Hybrid Transform & Quantization
81. Divide into Macroblocks One 16×16 block of luma pixels (Y) Two 8×8 blocks of chroma pixels (U, V) Typical Method
87. 16 8 8
88. Divide into blocks VP8 Method All blocks of luma and chroma are 4×4 blocks
91. 4 4 4
92. Discrete Cosine Transform Fast implementation Slightly worse in energy compaction than KLT Content-independency
93. Coding 2-D DCT Decoding 4×4 variant of LLM implementation
94. Coding 2-D DCT Decoding 4×4 variant of LLM implementation Practical fast 1-D DCT algorithms with 11 multiplications
95. I1 I2 I3 I4 O1 O2 O3 O4 Inverse DCT Graph in VP8 y0 y1 x0 x1 y0 = √2(x0×sin(π/8)-x1×cos(π/8)) y1 = √2(x0×cos(π/8)+x1×sin(π/8))
96. H.264/AVC use multiplication-less integer transform slightly better than Energy compaction is
97. It is efficient in processors with SIMD capability.
98. Walsh-Hadamard Transform Y = HXHT H = 1 1 1 1 1 1 -1 -1 1 -1 1 -1 1 -1 -1 1 [ ] HT is the transpose of H. Take advantage of the correlation to reduce redundancy.
99. Adaptive Quantization 128 quantization level. Different quantization level in single frame. 1st order luma DC 1st order luma AC 2st order luma DC 2st order luma AC 2st order chroma DC 2st order chroma AC
100. Entropy Coding
101. Supports distribution updates on a per-frame basis Boolean arithmetic coder Stable probability distributions within one frame Keyframes reset the probability values to the defaults
102. Adaptive Loop Filter
103. Removing blocking artifacts introduced by quantization and transformation.
104. Removing blocking artifacts introduced by quantization and transformation.
105. Removing blocking artifacts introduced by quantization and transformation. Slight Filtering
106. Removing blocking artifacts introduced by quantization and transformation. Slight Filtering Strong Filtering
107. Removing blocking artifacts introduced by quantization and transformation. Slight Filtering Strong Filtering No Filtering
108. Parallel Processing
109. Data Partition Compressed Data
110. Data Partition Compressed Data marcoblock code mode & motion vector transform coefficients
111. More Transform Coefficient Partition transform coefficients support up to 8 token partitions
112. More Transform Coefficient Partition transform coefficients support up to 8 token partitions
113. Compare to H.264
114. 100 120 140 160 180 200 220 240 260 280 300 Night 720p 2000kbps Sheriff 720p 2000kbps Tulip 720p 2000kbps Deocding speed in Frame/second VP8 H.264 High Profile Intel Core i7 3.2GHz
115. 20 25 30 35 40 45 Night 720p 2000kbps Sheriff 720p 2000kbps Tulip 720p 2000kbps Deocding speed in Frame/second VP8 H.264 High Profile Intel Atom N270 1.66GHz
116. Any Questions?
117. Thanks for your listening :)