I don’t care if you have 360 Intra directional predictors

Slides from my #VDD19 talk, learn about #AV1 #HEVC #AVC directional intra predictors and how to search them using image gradients

Transcript

1. I don’t care if you have 360 Intra directional predictors Luc Trudeau This presentation by Luc Trudeau is licensed under a Creative Commons Attribution 4.0 International License.
2. How did we get here?
3. H.263 ANNEX I - Advanced INTRA Coding mode Proposed by Telenor Reference : https://www.itu.int/rec/dologin_pub.asp?lang=e&id=T-REC-H.263-199802-S!!PDF-E&type=items H.263+ had horizontal and vertical Ref: https://www.itu.int/rec/dologin_pub.asp?lang=e&id=T-REC-H.263-199802-S!!PDF-E&type=items
4. Predicts DC, DC + AC (first row), DC + AC (first column) But not like we know it (in the transform domain) Ref: https://www.itu.int/wftp3/av-arch/video-site/h263plus/draft01.doc
5. Diagonals added in H.26L / H.264 (VCEG-N54) Ref : https://people.xiph.org/~xiphmont/demo/daala/demo2.shtml Ref : https://www.itu.int/wftp3/av-arch/video-site/0109_San/VCEG-N54.doc
6. Diagonals how do they work? a , = ( + + 1) / 2 b , = ( + + 1) / 2 , = ( + + 1) / 2 = ( + + 1) / 2 , = ( + + + + 2) / 4 , = ( + + + + 2) / 4 , = ( + + + + 2) / 4 = ( + + + + 2) / 4 = ( + + + + 2) / 4 = ( + + + + 2) / 4 Ref: Color Mixer http://trycolors.com X A B C D I a b c e d J f g h K i j k l L m n o p X A a b A B A X I X e f X A A B Mode 5: Vert, Right j k c l B C d C D n o g p A B B C h B C C D i X I I J m I J J L
7. Angles Added to H.265 / HEVC (JCTVC-A119) 34 Angles Ref: https://www.itu.int/wftp3/av-arch/jctvc-site/2010_04_A_Dresden/JCTVC-A119.zip
8. Biggest contribution to AV1 intra came from increasing the number of angles from 8 to 56 It works! Ref: https://aomedia.org/wp-content/uploads/2019/10/RyanLei_Intel.pptx
9. Number of Intra Prediction Angles
10. This is why encoding intra frames is getting slow
11. But wait! You don’t gotta search’em all
12. What’s the Image Gradient (aka Pixel Gradient)? The gradient of an image is a vector of its partials where: is the derivative with respect to x (gradient in the x direction) is the derivative with respect to y (gradient in the y direction) Ref: https://en.wikipedia.org/wiki/Image_gradient
13. How does one derive an image? A simple way is to use a Sobel Operator (other filters also work) Where A is your image
14. Let’s try this with some synthetic blocks
15. How do you get the angle? (x,y) Ref: https://en.wikipedia.org/wiki/Atan2
16. Let’s try with a real image
17. Protractor Time! Mode 0 Mode 1 Mode 3 Mode 4 Mode 5 Mode 6 Mode 7 Mode 8 Mode 1 Mode 8 Mode 3 Mode 7 Mode 0 Mode 5 Mode 4 Mode 6
18. Conclusion Already used in libAOM (see rdopt.c) Image Gradient-based approaches don’t require searching all directions Areas for improvement Improve precision of gradient operator Better alternative to histogram