Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AV1 Update by Jai Krishnan from Google

AV1 Update by Jai Krishnan from Google

Jai Krishnan from Google and Alliance for Open Media giving us the latest update on AV1 and how it is developed. Presented on Streaming Tech Sweden Nov 22 2018

Streaming Tech Sweden

November 22, 2017
Tweet

More Decks by Streaming Tech Sweden

Other Decks in Technology

Transcript

  1. project AV1 Update Streaming Tech Sweden 2017 Jai Krishnan Google

  2. project Why Google Invests in Codecs YouTube VP9 watch time

    gains vs. H.264 +25% +20% +15% +10% +5% +2%
  3. project AOM Goals • Video codec for the web with

    state of the art compression efficiency • Open source & royalty-free • Widely supported & adopted • Develop and deploy faster than industry standard
  4. project VoD Streaming Use Case • Studio content streamers require

    high quality encodes • User-generated content streamers require operation at massive scale
  5. project Live Streaming Use Cases • Broadcast-style encoding maintains high

    quality • RTC / Interactive requires ultra low latency
  6. project What will AV1 include? • Published Specification • High

    Level Syntax • Container Bindings & Metadata • Reference Encoder & Decoder • Reference Hardware RTL Design • Conformance Test Streams
  7. project What will AV1 include? • Published Specification • High

    Level Syntax • Container Bindings & Metadata • Reference Encoder & Decoder • Reference Hardware RTL Design • Conformance Test Streams ◦ 63 tools technically integrated and adopted ◦ 5 tools conditionally adopted, pending changes ◦ 26 tools under review
  8. project AV1 Tool Categories Entropy coding: “Daala” non-binary arithmetic coding,

    level map Intra prediction: Directional intra modes, palette, rectangular intra prediction Inter prediction: Warping, OBMC, 6 ref frames, wedge, inter-intra Partitioning: 4x4 coding units, 128x128 superblocks Transforms: separable hor/ver tx type, rectangular tx, quad-tree, 64x64 tx Quantization: DeltaQ, Quant matrices Filters: Three in-loop filters (deblock, CDEF, loop restoration)
  9. project Some things take more than 10 minutes... Compound ref

    modes 8-Compound ref modes; {Global Warp} 1-reference 7-references chosen per frame encoded frame Wedge INTER-INTRA modes: wedge-index + INTRA mode + single-ref INTER mode Wedge INTER-INTER: wedge-index + Compound ref mode Compound Segment INTER-INTER: cs-index + Compound ref mode 2-references 2-references 1-reference + Intra INTRA modes: {CfL, DC, PAETH, SMOOTH, SMOOTH_H, SMOOTH_V 8 directional modes + finer directions Gradual INTER-INTRA modes: INTERINTRA mode + Single-ref INTER mode 1-reference + Intra Ref1 Ref2 Ref3 MV1 MV2 MV1 MV2 MV1 MV2 Single ref lists Compound ref lists Ref1/R ef2 {MV1} {MV2} Ref1/R ef3 {MV1} {MV3} Ref2/R ef3 {MV2} {MV3} Create Ref lists Single ref modes 4 Single-ref modes; {Global; OBMC / Local Warp}
  10. project AV1 Tool Categories Entropy coding: “Daala” non-binary arithmetic coding,

    level map Intra prediction: Directional intra modes, palette, rectangular intra prediction Inter prediction: Warping, Wedge, OBMC, 6 ref frames, inter-intra Partitioning: 4x4 coding units, 128x128 superblocks Transforms: separable hor/ver tx type, rectangular tx, quad-tree, 64x64 tx Quantization: DeltaQ, Quant matrices Filters: Three in-loop filters (deblock, CDEF, loop restoration)
  11. project Example 1: Compound INTER prediction • Per-pixel blending P

    1 (i, j) P 2 (i, j) P f (i, j) m(i, j) 64-m(i, j) Blend Mask m(i, j) ∈ [0, 64] (integerized) P 1 P 2 P f {MV1, Ref1} {MV2, Ref2} INTER INTER Blend Mask depends on Side Info sent / Predictors themselves Compound-Segment Inter-Inter Wedge M P 1 P 2 P f {MV1, Ref1} {Mode} INTER INTRA Blend Mask depends on Side Info sent / Intra mode Inter-intra Gradual Inter-intra Wedge M (x + 32) >> 6
  12. project Example 1A: Compound INTER prediction with Wedge Square codebook

    Rectangular codebook Wedge codebook: Inter-Inter 4-bit shape 1-bit sign _________ 5 bits total Used for 8x8 up to 32x32 sizes
  13. project Example 1B: Compound INTER prediction with Warp Hor shear

    Ver shear Affine model
  14. project Example 2: Transform Block Variation Prediction Partitioning 16 transforms

    4 horizontal x 4 vertical Transform blocks DCT ADST FlipADST Identity
  15. project Example 3: CDEF (Constrained Directional Enhancement Filter) • CDEF:

    ◦ Combination of Daala Deringing filter and Cisco’s Constrained Low Pass Filter ◦ Adapts filtering to direction of edges and patterns in an image • Steps: ◦ Direction estimation: ▪ Conducted at 8x8 level by minimizing variances along predefined lines ◦ Based on direction use a nonlinear filter ▪ 5x5 support region
  16. project Example 3: CDEF (Constrained Directional Enhancement Filter) Primary filter

    Secondary filter
  17. project Preliminary Results, AV1 data savings vs. VP9 Metric VoD

    Savings Live Savings PSNR-Y -26.51% -29.25% PSNR-Cb -22.35% -28.43% PSNR-Cr -23.90% N/A PSNR-HVS -25.13% -27.09% CIEDE 2000 -25.12% -28.06% SSIM -24.48% -30.46% MS-SSIM -23.57% -28.43%
  18. project Timeline & More Information • Soft bitstream freeze, finalizing

    existing tools • Bitstream freeze in January 2018 • YouTube demos in Chrome Dev end of November 2017 • Codebase at aomedia.googlesource.com • Run tests at www.arewecompressedyet.com