AV1 Update by Jai Krishnan from Google

Jai Krishnan from Google and Alliance for Open Media giving us the latest update on AV1 and how it is developed. Presented on Streaming Tech Sweden Nov 22 2018

Transcript

1. project AV1 Update Streaming Tech Sweden 2017 Jai Krishnan Google
2. project Why Google Invests in Codecs YouTube VP9 watch time gains vs. H.264 +25% +20% +15% +10% +5% +2%
3. project AOM Goals • Video codec for the web with state of the art compression efficiency • Open source & royalty-free • Widely supported & adopted • Develop and deploy faster than industry standard
4. project VoD Streaming Use Case • Studio content streamers require high quality encodes • User-generated content streamers require operation at massive scale
5. project Live Streaming Use Cases • Broadcast-style encoding maintains high quality • RTC / Interactive requires ultra low latency
6. project What will AV1 include? • Published Specification • High Level Syntax • Container Bindings & Metadata • Reference Encoder & Decoder • Reference Hardware RTL Design • Conformance Test Streams
7. project What will AV1 include? • Published Specification • High Level Syntax • Container Bindings & Metadata • Reference Encoder & Decoder • Reference Hardware RTL Design • Conformance Test Streams ◦ 63 tools technically integrated and adopted ◦ 5 tools conditionally adopted, pending changes ◦ 26 tools under review
8. project AV1 Tool Categories Entropy coding: “Daala” non-binary arithmetic coding, level map Intra prediction: Directional intra modes, palette, rectangular intra prediction Inter prediction: Warping, OBMC, 6 ref frames, wedge, inter-intra Partitioning: 4x4 coding units, 128x128 superblocks Transforms: separable hor/ver tx type, rectangular tx, quad-tree, 64x64 tx Quantization: DeltaQ, Quant matrices Filters: Three in-loop filters (deblock, CDEF, loop restoration)
9. project Some things take more than 10 minutes... Compound ref modes 8-Compound ref modes; {Global Warp} 1-reference 7-references chosen per frame encoded frame Wedge INTER-INTRA modes: wedge-index + INTRA mode + single-ref INTER mode Wedge INTER-INTER: wedge-index + Compound ref mode Compound Segment INTER-INTER: cs-index + Compound ref mode 2-references 2-references 1-reference + Intra INTRA modes: {CfL, DC, PAETH, SMOOTH, SMOOTH_H, SMOOTH_V 8 directional modes + finer directions Gradual INTER-INTRA modes: INTERINTRA mode + Single-ref INTER mode 1-reference + Intra Ref1 Ref2 Ref3 MV1 MV2 MV1 MV2 MV1 MV2 Single ref lists Compound ref lists Ref1/R ef2 {MV1} {MV2} Ref1/R ef3 {MV1} {MV3} Ref2/R ef3 {MV2} {MV3} Create Ref lists Single ref modes 4 Single-ref modes; {Global; OBMC / Local Warp}
10. project AV1 Tool Categories Entropy coding: “Daala” non-binary arithmetic coding, level map Intra prediction: Directional intra modes, palette, rectangular intra prediction Inter prediction: Warping, Wedge, OBMC, 6 ref frames, inter-intra Partitioning: 4x4 coding units, 128x128 superblocks Transforms: separable hor/ver tx type, rectangular tx, quad-tree, 64x64 tx Quantization: DeltaQ, Quant matrices Filters: Three in-loop filters (deblock, CDEF, loop restoration)
11. project Example 1: Compound INTER prediction • Per-pixel blending P 1 (i, j) P 2 (i, j) P f (i, j) m(i, j) 64-m(i, j) Blend Mask m(i, j) ∈ [0, 64] (integerized) P 1 P 2 P f {MV1, Ref1} {MV2, Ref2} INTER INTER Blend Mask depends on Side Info sent / Predictors themselves Compound-Segment Inter-Inter Wedge M P 1 P 2 P f {MV1, Ref1} {Mode} INTER INTRA Blend Mask depends on Side Info sent / Intra mode Inter-intra Gradual Inter-intra Wedge M (x + 32) >> 6
12. project Example 1A: Compound INTER prediction with Wedge Square codebook Rectangular codebook Wedge codebook: Inter-Inter 4-bit shape 1-bit sign _________ 5 bits total Used for 8x8 up to 32x32 sizes
13. project Example 1B: Compound INTER prediction with Warp Hor shear Ver shear Affine model
14. project Example 2: Transform Block Variation Prediction Partitioning 16 transforms 4 horizontal x 4 vertical Transform blocks DCT ADST FlipADST Identity
15. project Example 3: CDEF (Constrained Directional Enhancement Filter) • CDEF: ◦ Combination of Daala Deringing filter and Cisco’s Constrained Low Pass Filter ◦ Adapts filtering to direction of edges and patterns in an image • Steps: ◦ Direction estimation: ▪ Conducted at 8x8 level by minimizing variances along predefined lines ◦ Based on direction use a nonlinear filter ▪ 5x5 support region
16. project Example 3: CDEF (Constrained Directional Enhancement Filter) Primary filter Secondary filter
17. project Preliminary Results, AV1 data savings vs. VP9 Metric VoD Savings Live Savings PSNR-Y -26.51% -29.25% PSNR-Cb -22.35% -28.43% PSNR-Cr -23.90% N/A PSNR-HVS -25.13% -27.09% CIEDE 2000 -25.12% -28.06% SSIM -24.48% -30.46% MS-SSIM -23.57% -28.43%
18. project Timeline & More Information • Soft bitstream freeze, finalizing existing tools • Bitstream freeze in January 2018 • YouTube demos in Chrome Dev end of November 2017 • Codebase at aomedia.googlesource.com • Run tests at www.arewecompressedyet.com