Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to VP8

KuoE0
June 13, 2013

Introduction to VP8

KuoE0

June 13, 2013
Tweet

More Decks by KuoE0

Other Decks in Technology

Transcript

  1. Introduction to VP8
    郭⾄至軒 (KuoE0)
    [email protected]

    View full-size slide

  2. Latest update: Nov 19, 2013,
    Attribution-ShareAlike 3.0 Unported
    (CC BY-SA 3.0)
    http://creativecommons.org/licenses/by-sa/3.0/

    View full-size slide

  3. Video Codec
    VP8

    View full-size slide

  4. An
    Open
    Source
    Codec

    View full-size slide

  5. Developed by
    On2 Technology

    View full-size slide

  6. Developed by
    On2 Technology
    February, 2010

    View full-size slide

  7. Acquired by
    Google
    February, 2010

    View full-size slide

  8. March, 2013
    web m

    View full-size slide

  9. Royalty-Free Terms
    March, 2013
    web m

    View full-size slide

  10. Successor
    VP9

    View full-size slide

  11. Successor
    VP9
    May 15, 2013

    View full-size slide

  12. focus on
    Internet
    web-based
    application

    View full-size slide

  13. Low Bandwidth Requirement
    Image Quality:
    watchable (PSNR: ~30dB)
    visually lossless (PSNR: ~45dB)

    View full-size slide

  14. Heterogeneous Client Hardware

    View full-size slide

  15. Heterogeneous Client Hardware

    View full-size slide

  16. Heterogeneous Client Hardware
    Efficient
    Implementations

    View full-size slide

  17. Web Video Format
    YUV 420 color sampling
    8 bit per channel depth
    Up to 16383 × 16383 pixels

    View full-size slide

  18. Processing Flow

    View full-size slide

  19. Coding
    Predict
    Transform + Quantize
    Entropy Code
    Loop Filter

    View full-size slide

  20. Decoding
    Entropy Decode
    Predict
    Dequantize+Inverse Transform
    Loop Filter

    View full-size slide

  21. Reference Frame

    View full-size slide

  22. Golden
    Frame
    Last Frame
    Alternate
    Frame
    Reference
    Frame

    View full-size slide

  23. Golden
    Frame
    Last Frame
    Alternate
    Frame
    At most 3 reference frames in VP8.

    View full-size slide

  24. Last Frame Current Frame

    View full-size slide

  25. Golden Frame
    Choose an arbitrary frame in the past.
    Define a number of flags to notify decoder
    when and how to update this buffer.

    View full-size slide

  26. Golden Frame
    Choose an arbitrary frame in the past.
    Define a number of flags to notify decoder
    when and how to update this buffer.

    View full-size slide

  27. set as the golden frame

    View full-size slide

  28. Golden Frame
    Golden Frame

    View full-size slide

  29. Golden Frame
    Golden Frame

    View full-size slide

  30. Golden Frame
    Golden Frame

    View full-size slide

  31. Golden Frame
    Golden Frame

    View full-size slide

  32. Golden Frame
    Golden Frame

    View full-size slide

  33. Golden Frame
    Golden Frame

    View full-size slide

  34. Golden Frame
    Golden Frame

    View full-size slide

  35. Golden Frame
    Golden Frame

    View full-size slide

  36. Golden Frame
    Golden Frame

    View full-size slide

  37. Reconstruct
    moving object
    background

    View full-size slide

  38. Alternate Frame
    Other
    Frame
    Alternate
    Frame

    View full-size slide

  39. Alternate Frame
    Other
    Frame
    Alternate
    Frame
    decode
    show

    View full-size slide

  40. Alternate Frame
    Other
    Frame
    Alternate
    Frame
    decode
    show
    decode show

    View full-size slide

  41. Alternate Frame
    Other
    Frame
    Alternate
    Frame
    decode
    show
    decode show
    store beneficial information

    View full-size slide

  42. Construct from multi-frame

    View full-size slide

  43. Construct from multi-frame

    View full-size slide

  44. Construct from multi-frame

    View full-size slide

  45. Construct from multi-frame
    Alternate
    Frame

    View full-size slide

  46. Typical Frame
    I B B P B B P B B I B B P

    View full-size slide

  47. VP8
    L G
    A
    G G G G G L G G G
    A
    G L

    View full-size slide

  48. Intra Prediction
    Inter Prediction
    use data within a single video frame
    use data from previously encoded frames

    View full-size slide

  49. Intra Prediction
    Luma
    Luma
    Chroma

    View full-size slide

  50. Intra Prediction
    Luma
    Luma
    Chroma
    16 4 8

    View full-size slide

  51. H_PRED (horizontal prediction)
    V_PRED (vertical prediction)
    DC_PRED (DC prediction)
    TM_PRED (TrueMotion prediction)
    Four Prediction Modes:

    View full-size slide

  52. Horizontal Prediction
    Fills each column of the block with
    a copy of the left column.
    a b c d e
    f g h i j
    k l m n o
    p q r s t
    u v w x y
    A B C D E
    F G H I J
    K L M N O
    P Q R S T
    U V W X Y

    View full-size slide

  53. Horizontal Prediction
    Fills each column of the block with
    a copy of the left column.
    a b c d e
    f g h i j
    k l m n o
    p q r s t
    u v w x y
    A B C D E
    F G H I J
    K L M N O
    P Q R S T
    U V W X Y
    e
    j
    o
    t
    y

    View full-size slide

  54. Horizontal Prediction
    Fills each column of the block with
    a copy of the left column.
    a b c d e
    f g h i j
    k l m n o
    p q r s t
    u v w x y
    A B C D E
    F G H I J
    K L M N O
    P Q R S T
    U V W X Y
    e
    j
    o
    t
    y
    e e e e e
    j j j j j
    o o o o o
    t t t t t
    y y y y y

    View full-size slide

  55. Vertical Prediction
    Fills each row of the block with a
    copy of the above row.
    a b c d e
    f g h i j
    k l m n o
    p q r s t
    u v w x y
    A B C D E
    F G H I J
    K L M N O
    P Q R S T
    U V W X Y

    View full-size slide

  56. Vertical Prediction
    Fills each row of the block with a
    copy of the above row.
    a b c d e
    f g h i j
    k l m n o
    p q r s t
    u v w x y
    A B C D E
    F G H I J
    K L M N O
    P Q R S T
    U V W X Y
    U V W X Y

    View full-size slide

  57. Vertical Prediction
    Fills each row of the block with a
    copy of the above row.
    a b c d e
    f g h i j
    k l m n o
    p q r s t
    u v w x y
    A B C D E
    F G H I J
    K L M N O
    P Q R S T
    U V W X Y
    U V W X Y
    U V W X Y
    U V W X Y
    U V W X Y
    U V W X Y
    U V W X Y

    View full-size slide

  58. DC Prediction
    Fills the block with a single value
    using the average of the pixels in
    the above row and the left column.
    a b c d e
    f g h i j
    k l m n o
    p q r s t
    u v w x y
    A B C D E
    F G H I J
    K L M N O
    P Q R S T
    U V W X Y

    View full-size slide

  59. DC Prediction
    Fills the block with a single value
    using the average of the pixels in
    the above row and the left column.
    a b c d e
    f g h i j
    k l m n o
    p q r s t
    u v w x y
    A B C D E
    F G H I J
    K L M N O
    P Q R S T
    U V W X Y
    U V W X Y
    e
    j
    o
    t
    y
    Z = (U + V + W + X +
    Y + e + j + o + t + y) ÷
    10

    View full-size slide

  60. DC Prediction
    Fills the block with a single value
    using the average of the pixels in
    the above row and the left column.
    a b c d e
    f g h i j
    k l m n o
    p q r s t
    u v w x y
    A B C D E
    F G H I J
    K L M N O
    P Q R S T
    U V W X Y
    U V W X Y
    e
    j
    o
    t
    y
    Z = (U + V + W + X +
    Y + e + j + o + t + y) ÷
    10
    Z Z Z Z Z
    Z Z Z Z Z
    Z Z Z Z Z
    Z Z Z Z Z
    Z Z Z Z Z

    View full-size slide

  61. * * * * L0
    * * * * L1
    * * * * L2
    * * * * L3
    * * * * L4
    * * * * *
    * * * * *
    * * * * *
    * * * * *
    A0 A1 A2 A3 A4
    TrueMotion Prediction
    Horizontal differences
    between pixels in
    above row and vertical
    differences between
    pixels in left column are
    propagated (starting
    from C).
    * * * * *
    * * * * *
    * * * * *
    * * * * *
    * * * * C

    View full-size slide

  62. * * * * L0
    * * * * L1
    * * * * L2
    * * * * L3
    * * * * L4
    * * * * *
    * * * * *
    * * * * *
    * * * * *
    A0 A1 A2 A3 A4
    A0 A1 A2 A3 A4
    L0
    L1
    L2
    L3
    L4
    TrueMotion Prediction
    Horizontal differences
    between pixels in
    above row and vertical
    differences between
    pixels in left column are
    propagated (starting
    from C).
    * * * * *
    * * * * *
    * * * * *
    * * * * *
    * * * * C
    C
    Xij = Ai + Lj - C

    View full-size slide

  63. * * * * L0
    * * * * L1
    * * * * L2
    * * * * L3
    * * * * L4
    * * * * *
    * * * * *
    * * * * *
    * * * * *
    A0 A1 A2 A3 A4
    A0 A1 A2 A3 A4
    L0
    L1
    L2
    L3
    L4
    TrueMotion Prediction
    Horizontal differences
    between pixels in
    above row and vertical
    differences between
    pixels in left column are
    propagated (starting
    from C).
    * * * * *
    * * * * *
    * * * * *
    * * * * *
    * * * * C
    C
    Xij = Ai + Lj - C
    Xij Xij Xij Xij Xij
    Xij Xij Xij Xij Xij
    Xij Xij Xij Xij Xij
    Xij Xij Xij Xij Xij
    Xij Xij Xij Xij Xij

    View full-size slide

  64. Inter Prediction
    As mentioned above...

    View full-size slide

  65. Inter Prediction
    Golden
    Frame
    Last Frame
    Alternate
    Frame

    View full-size slide

  66. Motion Vector
    Reusing vectors from neighboring
    macroblocks.
    Flexible partitioning of a macroblock into sub-
    blocks.

    View full-size slide

  67. Sub-pixel Interpolation
    Quarter pixel accurate motion vectors for
    luma pixels.
    High performance six-tap interpolation
    filters.
    [3, -16, 77, 77, -16, 3]/128 for 1⁄2 pixel positions
    [2, -11, 108, 36, -8, 1]/128 for 1⁄4 pixel positions
    [1, -8, 36, 108, -11, 2]/128 for 3⁄4 pixel positions

    View full-size slide

  68. Hybrid Transform
    & Quantization

    View full-size slide

  69. Divide into Macroblocks
    One 16×16 block of luma pixels (Y)
    Two 8×8 blocks of chroma pixels (U, V)
    Typical Method

    View full-size slide

  70. Divide into blocks
    VP8 Method
    All blocks of luma and chroma are 4×4
    blocks

    View full-size slide

  71. Discrete Cosine Transform
    Fast implementation
    Slightly worse in energy compaction
    than KLT
    Content-independency

    View full-size slide

  72. Coding
    2-D DCT
    Decoding
    4×4 variant of LLM
    implementation

    View full-size slide

  73. Coding
    2-D DCT
    Decoding
    4×4 variant of LLM
    implementation
    Practical fast 1-D DCT algorithms with 11 multiplications

    View full-size slide

  74. I1
    I2
    I3
    I4
    O1
    O2
    O3
    O4
    Inverse DCT Graph in VP8
    y0
    y1
    x0
    x1
    y0 = √2(x0×sin(π/8)-x1×cos(π/8))
    y1 = √2(x0×cos(π/8)+x1×sin(π/8))

    View full-size slide

  75. H.264/AVC
    use multiplication-less integer transform
    slightly better than
    Energy compaction is

    View full-size slide

  76. It is efficient in processors with
    SIMD capability.

    View full-size slide

  77. Walsh-Hadamard Transform
    Y = HXHT
    H =
    1 1 1 1
    1 1 -1 -1
    1 -1 1 -1
    1 -1 -1 1
    [ ]
    HT is the transpose of H.
    Take advantage of
    the correlation to
    reduce redundancy.

    View full-size slide

  78. Adaptive Quantization
    128 quantization level.
    Different quantization level in single frame.
    1st order luma DC
    1st order luma AC
    2st order luma DC
    2st order luma AC
    2st order chroma DC
    2st order chroma AC

    View full-size slide

  79. Entropy Coding

    View full-size slide

  80. Supports distribution updates on a per-frame
    basis
    Boolean arithmetic coder
    Stable probability distributions within one
    frame
    Keyframes reset the probability values to the
    defaults

    View full-size slide

  81. Adaptive Loop Filter

    View full-size slide

  82. Removing blocking artifacts introduced by
    quantization and transformation.

    View full-size slide

  83. Removing blocking artifacts introduced by
    quantization and transformation.

    View full-size slide

  84. Removing blocking artifacts introduced by
    quantization and transformation.
    Slight Filtering

    View full-size slide

  85. Removing blocking artifacts introduced by
    quantization and transformation.
    Slight Filtering
    Strong Filtering

    View full-size slide

  86. Removing blocking artifacts introduced by
    quantization and transformation.
    Slight Filtering
    Strong Filtering
    No Filtering

    View full-size slide

  87. Parallel Processing

    View full-size slide

  88. Data Partition
    Compressed Data

    View full-size slide

  89. Data Partition
    Compressed Data
    marcoblock code mode
    & motion vector
    transform coefficients

    View full-size slide

  90. More Transform Coefficient Partition
    transform coefficients
    support up to 8 token partitions

    View full-size slide

  91. More Transform Coefficient Partition
    transform coefficients
    support up to 8 token partitions

    View full-size slide

  92. Compare to H.264

    View full-size slide

  93. 100
    120
    140
    160
    180
    200
    220
    240
    260
    280
    300
    Night 720p 2000kbps Sheriff 720p 2000kbps Tulip 720p 2000kbps
    Deocding speed in Frame/second
    VP8 H.264 High Profile
    Intel Core i7 3.2GHz

    View full-size slide

  94. 20
    25
    30
    35
    40
    45
    Night 720p 2000kbps Sheriff 720p 2000kbps Tulip 720p 2000kbps
    Deocding speed in Frame/second
    VP8 H.264 High Profile
    Intel Atom N270 1.66GHz

    View full-size slide

  95. Any Questions?

    View full-size slide

  96. Thanks for your listening :)

    View full-size slide