project AV1 Update Streaming Tech Sweden 2017 Jai Krishnan Google 

project Why Google Invests in Codecs YouTube VP9 watch time gains vs. H.264 +25% +20% +15% +10% +5% +2% 

project AOM Goals ● Video codec for the web with state of the art compression efficiency ● Open source & royalty-free ● Widely supported & adopted ● Develop and deploy faster than industry standard 

project VoD Streaming Use Case ● Studio content streamers require high quality encodes ● User-generated content streamers require operation at massive scale 

project Live Streaming Use Cases ● Broadcast-style encoding maintains high quality ● RTC / Interactive requires ultra low latency 

project What will AV1 include? ● Published Specification ● High Level Syntax ● Container Bindings & Metadata ● Reference Encoder & Decoder ● Reference Hardware RTL Design ● Conformance Test Streams 

project What will AV1 include? ● Published Specification ● High Level Syntax ● Container Bindings & Metadata ● Reference Encoder & Decoder ● Reference Hardware RTL Design ● Conformance Test Streams ○ 63 tools technically integrated and adopted ○ 5 tools conditionally adopted, pending changes ○ 26 tools under review 

project AV1 Tool Categories Entropy coding: “Daala” non-binary arithmetic coding, level map Intra prediction: Directional intra modes, palette, rectangular intra prediction Inter prediction: Warping, OBMC, 6 ref frames, wedge, inter-intra Partitioning: 4x4 coding units, 128x128 superblocks Transforms: separable hor/ver tx type, rectangular tx, quad-tree, 64x64 tx Quantization: DeltaQ, Quant matrices Filters: Three in-loop filters (deblock, CDEF, loop restoration) 

project Some things take more than 10 minutes... Compound ref modes 8-Compound ref modes; {Global Warp} 1-reference 7-references chosen per frame encoded frame Wedge INTER-INTRA modes: wedge-index + INTRA mode + single-ref INTER mode Wedge INTER-INTER: wedge-index + Compound ref mode Compound Segment INTER-INTER: cs-index + Compound ref mode 2-references 2-references 1-reference + Intra INTRA modes: {CfL, DC, PAETH, SMOOTH, SMOOTH_H, SMOOTH_V 8 directional modes + finer directions Gradual INTER-INTRA modes: INTERINTRA mode + Single-ref INTER mode 1-reference + Intra Ref1 Ref2 Ref3 MV1 MV2 MV1 MV2 MV1 MV2 Single ref lists Compound ref lists Ref1/R ef2 {MV1} {MV2} Ref1/R ef3 {MV1} {MV3} Ref2/R ef3 {MV2} {MV3} Create Ref lists Single ref modes 4 Single-ref modes; {Global; OBMC / Local Warp} 

project AV1 Tool Categories Entropy coding: “Daala” non-binary arithmetic coding, level map Intra prediction: Directional intra modes, palette, rectangular intra prediction Inter prediction: Warping, Wedge, OBMC, 6 ref frames, inter-intra Partitioning: 4x4 coding units, 128x128 superblocks Transforms: separable hor/ver tx type, rectangular tx, quad-tree, 64x64 tx Quantization: DeltaQ, Quant matrices Filters: Three in-loop filters (deblock, CDEF, loop restoration) 

project Example 1: Compound INTER prediction ● Per-pixel blending P 1 (i, j) P 2 (i, j) P f (i, j) m(i, j) 64-m(i, j) Blend Mask m(i, j) ∈ [0, 64] (integerized) P 1 P 2 P f {MV1, Ref1} {MV2, Ref2} INTER INTER Blend Mask depends on Side Info sent / Predictors themselves Compound-Segment Inter-Inter Wedge M P 1 P 2 P f {MV1, Ref1} {Mode} INTER INTRA Blend Mask depends on Side Info sent / Intra mode Inter-intra Gradual Inter-intra Wedge M (x + 32) >> 6 

project Example 1A: Compound INTER prediction with Wedge Square codebook Rectangular codebook Wedge codebook: Inter-Inter 4-bit shape 1-bit sign _________ 5 bits total Used for 8x8 up to 32x32 sizes 

project Example 1B: Compound INTER prediction with Warp Hor shear Ver shear Affine model 

project Example 2: Transform Block Variation Prediction Partitioning 16 transforms 4 horizontal x 4 vertical Transform blocks DCT ADST FlipADST Identity 

project Example 3: CDEF (Constrained Directional Enhancement Filter) ● CDEF: ○ Combination of Daala Deringing filter and Cisco’s Constrained Low Pass Filter ○ Adapts filtering to direction of edges and patterns in an image ● Steps: ○ Direction estimation: ■ Conducted at 8x8 level by minimizing variances along predefined lines ○ Based on direction use a nonlinear filter ■ 5x5 support region 

project Example 3: CDEF (Constrained Directional Enhancement Filter) Primary filter Secondary filter 

project Preliminary Results, AV1 data savings vs. VP9 Metric VoD Savings Live Savings PSNR-Y -26.51% -29.25% PSNR-Cb -22.35% -28.43% PSNR-Cr -23.90% N/A PSNR-HVS -25.13% -27.09% CIEDE 2000 -25.12% -28.06% SSIM -24.48% -30.46% MS-SSIM -23.57% -28.43% 

project Timeline & More Information ● Soft bitstream freeze, finalizing existing tools ● Bitstream freeze in January 2018 ● YouTube demos in Chrome Dev end of November 2017 ● Codebase at aomedia.googlesource.com ● Run tests at www.arewecompressedyet.com