Antman: Saving storage from a multimedia perspective

Seunghoon Baek
LINE Plus Vision Platform Software Engineer
https://linedevday.linecorp.com/2020/ja/sessions/7132
https://linedevday.linecorp.com/2020/en/sessions/7132

Transcript

5. HEIF JPEG 540 KB 360 KB 970 KB 204 KB 85 KB 490 KB
6. File size (bytes) 0 200 400 600 800 1000 1200 File Size JPEG HEIF
7. File size (bytes) 0 200 400 600 800 1000 1200 JPEG HEIF File Size x2 ▲ Compression rate
8. Antman: Saving Storage from a multimedia perspective Seunghoon, Baek / LINE Platform Engineering 4
9. How LINE Handles Media Files OBS (OBject Storage) LINE Family Services Storage Delivery Processing In-house Server Platform
10. LINE Family Services How LINE Handles Media Files In-house Server Platform Delivery Processing OBS (OBject Storage) Storage 60 PB ▲ 1 PB = 1,024 TB 1 TB = 1,024 GB
11. What Happens If Storage Gets Bigger? Operation Cost ▲ $1.4 million/ Month Space in Data Center ▼ 1) Figure the cost out using Amazon S3 storage pricing. 1)
12. OBS Storage LINE Services That Use Storage Intensively LINE Album feature LINE Timeline … LINE Messenger
13. OBS Storage LINE Services That Use Storage Intensively LINE Album feature LINE Timeline … LINE Messenger Numerous JPEG files stored
14. Goals Operation Cost ▼ Space in Data Center ▲ Text File JPEG JPEG JPEG JPEG … HEIF Text File … HEIF HEIF HEIF
15. Goals User Mobile Network Cost ▼ User Experience ▲ JPEG 100 KB HEIF 50 KB
16. 15 PB 22.5 PB 30 PB 37.5 PB 45 PB Feb '19 Apr '19 Jun '19 Aug '19 Oct '19 Dec '19 Mar '20 May '20 Jul '20 Sep '20 Results The usage of Media Storage for LINE Album feature Antman Started
17. 15 PB 22.5 PB 30 PB 37.5 PB 45 PB Feb '19 Apr '19 Jun '19 Aug '19 Oct '19 Dec '19 Mar '20 May '20 Jul '20 Sep '20 Antman exists If Antman not existed Results Antman Started The usage of Media Storage for LINE Album feature
18. 15 PB 22.5 PB 30 PB 37.5 PB 45 PB Feb '19 Apr '19 Jun '19 Aug '19 Oct '19 Dec '19 Mar '20 May '20 Jul '20 Sep '20 Antman exists If Antman not existed Results Antman Started 21 PB Saved The usage of Media Storage for LINE Album feature
19. Agenda › Struggling for Image Format Compatibility › Obsession with Image Quality › Antman Operation Cost › Results
20. Struggling for Image Format Compatibility
21. Original JPEG Reducing Storage Usage HEIF 1) https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format 1)
22. Deleted = Reduce Storage Usage Reducing Storage Usage Original JPEG HEIF 1) https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format 1)
23. Reducing Storage Usage Original JPEG HEIF Introduced in 2015 Can reduce file size by about 50% of JPEG The latest image format compared to JPEG(1992) Maintaining equivalent image quality 1) https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format 2) https://en.wikipedia.org/wiki/High_Efficiency_Image_File_Format#JPEG_and_HEIF 1) 2)
24. ! Device A (Support HEIF) OK Reducing Storage Usage HEIF
25. ! Device A (Support HEIF) OK ?? Reducing Storage Usage " Device B (Not support HEIF) HEIF
26. ! Device A (Support HEIF) OK ! Device B (Not support HEIF) Reducing Storage Usage Restored JPEG Convert OK HEIF
27. Reducing Storage Usage ! User A (can read HEIF) OK ! User B (can’t read HEIF) OK Needs to figure out the most optimal Quality Factor Convert Restored JPEG HEIF
28. Disk Low Quality Factor High Quality Factor Disk Disk Disk Disk Disk Disk Disk Disk Disk HEIF JPEG JPEG JPEG
29. Disk Low Quality Factor High Quality Factor Disk Disk Disk Disk Disk Disk HEIF JPEG JPEG Optimal Quality Factor Disk Disk Disk JPEG
30. JPEG Format 40*
    4UBSU
    0G
    *NBHF %25
    %FpOF
    2VBOUJ[BUJPO
    5BCMF ʜ $PNQSFTTFE
    %BUB &0*
    &OE
    0G
    *NBHF Quality Factor
31. DQT Backup Original JPEG %25 HEIF
32. DQT Backup Original JPEG %25 ‘Free’ Box HEIF
33. HEIF DQT Backup Original JPEG %25 ‘Free’ Box %25 Backup
34. HEIF DQT Backup ‘Free’ Box %25
35. HEIF DQT Backup ‘Free’ Box %25 Restored JPEG %25 Transcoding * LINE Engineering Blog (EN) https://engineering.linecorp.com/en/blog/developing-the-antman-project
36. Obsession with Image Quality
37. Original JPEG Preserve Image Quality Image Quality in HEIF HEIF
38. Original JPEG Preserve Image Quality Image Quality in HEIF HEIF ???
39. Disk Low Quality Factor High Quality Factor Disk Disk Disk Disk Disk Disk Disk Disk Disk JPEG HEIF HEIF HEIF
40. Disk Low Quality Factor High Quality Factor Disk Disk Disk Disk Disk Disk Disk Disk Disk JPEG HEIF HEIF HEIF
41. Original JPEG Fixed Compression Ratio Use Fixed Compression Ratio 50% Attempt #1 | HEIF
42. Result (Good) Attempt #1 | JPEG (540 KB) HEIF (204 KB)
43. Result (Bad) Attempt #1 | JPEG (141 KB) HEIF (71 KB)
44. HEIF (71 KB) Result (Bad) Attempt #1 | JPEG (141 KB)
45. Find Best Quality Factor Quality Factor Attempt #2 | Original JPEG HEIF PSNR 1) PSNR: Peak Signal-to-Noise Ratio 1)
46. Find Best Quality Factor 1) PSNR: Peak Signal-to-Noise Ratio PSNR 1) Quality Factor Attempt #2 | Original JPEG Recovered JPEG HEIF
47. Find Best Quality Factor 1) PSNR: Peak Signal-to-Noise Ratio PSNR 1) Answer Good or Bad Attempt #2 |
48. Corner Case of PSNR Attempt #2 | JPEG HEIF (PSNR: 40.21 dB)
49. Corner Case of PSNR Attempt #2 | HEIF JPEG (PSNR: 40.21 dB)
50. Grid PSNR Attempt #3 |
51. Grid PSNR Attempt #3 |
52. Grid PSNR Attempt #3 |
53. Grid PSNR Attempt #3 | Noise detection ability ▲
54. Grid Area Problem Attempt #3 |
55. 14/3
    
    E# Grid Area Problem 14/3
    
    E# Attempt #3 |
56. Grid Area Problem Attempt #3 | 14/3
    
    E#
57. Grid Area Problem Attempt #3 | Noise Not Detected 14/3
    
    E#
58. Scan window Image Sliding Window Attempt #4 | PSNR
59. Scan window Image Sliding Window Attempt #4 | PSNR
60. Image Scan Window P Good or Bad Sliding Window Summarize PSNR Results * LINE Engineering Blog (EN) https://engineering.linecorp.com/en/blog/developing-the-antman-project Attempt #4 |
61. Process for Most Optimal Quality Factor 1 Low High Initial Quality Factor No quality loss Quality loss
62. Process for Most Optimal Quality Factor 2 1 Low High Adjust Quality Factor No quality loss Quality loss
63. Process for Most Optimal Quality Factor 3 2 1 Low High Adjust Quality Factor No quality loss Quality loss
64. Process for Most Optimal Quality Factor 3 4 2 1 Low High Adjust Quality Factor No quality loss Quality loss
65. Process for Most Optimal Quality Factor 3 4 2 5 1 Low High No quality loss Quality loss Adjust Quality Factor
66. Process for Most Optimal Quality Factor 3 4 2 5 1 Low High No quality loss Quality loss Best Quality Factor
67. HEIF File Size Distribution # Samples HEIF bytes / JPEG bytes (percent) 10% 20% 30% 40% 50% 60% 70% 80%
68. HEIF File Size Distribution # Samples HEIF bytes / JPEG bytes (percent) 10% 20% 30% 40% 50% 60% 70% 80% Generally 30% ~ 40%
69. 50%
70. 50% 65%
71. Antman Operation Cost
72. GPU Parallel Processing JPEG Transcoding HEIF Transcoding Detecting Quality Loss Antman Operation Cost
73. Years 7+ Max image resolution 4K Images /day 60 million LINE Album feature Antman Operation Cost
74. LINE Album feature Years 7+ Max image resolution 4K Images /day 60 million Antman Operation Cost JPEG HEIF CPU CPU CPU CPU CPU CPU CPU
75. Antman Operation Cost GPU (Expensive equipments… ##)
76. Video Transcoder Server Group GPU Server GPU Server GPU Server Antman Server Group GPU Server GPU Server GPU Server GPU Server GPU Server GPU Server GPU Server Busy state Share GPU Equipments
77. Video Transcoder Server Group GPU Server GPU Server GPU Server Antman Server Group GPU Server GPU Server GPU Server GPU Server GPU Server GPU Server GPU Server Idle state Share GPU Equipments
78. Video Transcoder Server Group GPU Server GPU Server GPU Server Antman Server Group GPU Server GPU Server GPU Server GPU Server GPU Server GPU Server GPU Server Rent ! Share GPU Equipments
79. Video Transcoder Server Group GPU Server GPU Server GPU Server Antman Server Group GPU Server GPU Server GPU Server GPU Server GPU Server GPU Server GPU Server Need more GPU ! Share GPU Equipments
80. Video Transcoder Server Group GPU Server GPU Server GPU Server Antman Server Group GPU Server GPU Server GPU Server GPU Server GPU Server GPU Server GPU Server Return ! Share GPU Equipments
81. * LINE Engineering Blog (EN) https://engineering.linecorp.com/en/blog/developing-the-antman-project Zero Infrastructure Cost 0% 25% 50% 75% 100% 24 Hours 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Transcoder Antman
82. Results
83. 15 PB 22.5 PB 30 PB 37.5 PB 45 PB Feb '19 Apr '19 Jun '19 Aug '19 Oct '19 Dec '19 Mar '20 May '20 Jul '20 Sep '20 Antman exists If Antman not existed Results Completed processing JPEG files stored before Antman Antman started The usage of Media Storage for LINE Album feature
84. 15 PB 22.5 PB 30 PB 37.5 PB 45 PB Feb '19 Apr '19 Jun '19 Aug '19 Oct '19 Dec '19 Mar '20 May '20 Jul '20 Sep '20 Antman exists If Antman not existed Results Antman started 21 PB Saved The usage of Media Storage for LINE Album feature
85. 15 PB 22.5 PB 30 PB 37.5 PB 45 PB Feb '19 Apr '19 Jun '19 Aug '19 Oct '19 Dec '19 Mar '20 May '20 Jul '20 Sep '20 Antman exists If Antman not existed Results Antman started 1) Figure the cost out using Amazon S3 storage pricing. $ 470K / Month Storage Operation Cost 1) The usage of Media Storage for LINE Album feature
86. 15 PB 22.5 PB 30 PB 37.5 PB 45 PB Feb '19 Apr '19 Jun '19 Aug '19 Oct '19 Dec '19 Mar '20 May '20 Jul '20 Sep '20 Antman exists If Antman not existed Results $2 million ▲ saved 1) Figure the cost out using Amazon S3 storage pricing. 1) The usage of Media Storage for LINE Album feature
87. Future Work Video (H264) ??? Image (JPEG) Image (HEIF)
88. Thank you