Video Transcoding at Scale for ABC iview

ABC iview is a video on demand service from the Australian Broadcasting Corporation. In 2015, we built a new service in-house to handle transcoding for all of iview's content.

Metro is built on AWS, and uses FFmpeg, node.js and go components to transcode content quickly and cost-efficiently.

This talk was presented on August 4th 2016 at NDC Sydney.

Transcript

1. Video Transcoding at Scale for ABC iview Daphne Chong @daphnechong
5. metro
6. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
7. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
9. What changes are we making?
12. High speed Medium speed Low speed
13. Standard Encoding H.264 MP4
14. Transcoding Example
15. Step 1: Prepare watermark
16. Step 1: Prepare watermark
17. Step 1: Prepare watermark
18. Step 2: Input video (GXF)
19. Step 3: Asymmetrical cropping
20. Step 4: Scale and set aspect ratio
21. Step 5: Apply watermark
22. Step 6: Ready to stream 1500k
23. other rendition sizes 1000k 650k 500k 220k audio only
24. Original file audio track
25. Previous transcoder audio track
26. After audio normalisation
27. That’s a lot of changes...
28. Transcoding Profiles
29. GXF, ABC1 Profile: - Renditions (6) - Crop - Normalise audio - Adjust ratio - Add watermark - GXF to MP4
30. GXF, ABC1
31. GXF, ABC1 MOV, Arts Profile: - Renditions (6) - Adjust ratio - ProRes MOV to MP4
32. GXF, ABC1 MOV, Arts
33. What is this “Magic Video Software” you speak of?
35. ./ffmpeg -i utopia.gxf utopia.mp4
36. ./ffmpeg -i INPUT -c:v libx264 -filter_complex "[0:v]crop=720:576:0:32,yadif,scale=1024x576,setdar=16/9[ma in], [1:v]scale=54:-1[ovrl], [main][ovrl]overlay=main_w-overlay_w-47:main_h-overlay_h-44 " -crf 22 -maxrate 1404k -bufsize 1500k -pix_fmt yuv420p -profile:v high -level 4.0 -refs 3 -r 25 -g 50 -c:a libfdk_aac -b:a 96k -filter_complex "[0:a:0][0:a:1] amerge=inputs=2, dynaudnorm=f=500:g=31:p=0.82:m=30.0:r=0.0:s=0.0 [aout]" -ar 44100 -ac 2 -map 0:0 -map "[aout]" -movflags +faststart -threads 0 -y OUTPUT In reality, it looks like this
37. ./ffmpeg -i INPUT -c:v libx264 -filter_complex "[0:v]crop=720:576:0:32,yadif,scale=1024x576,setdar=16/9[ma in], [1:v]scale=54:-1[ovrl], [main][ovrl]overlay=main_w-overlay_w-47:main_h-overlay_h-44 " -crf 22 -maxrate 1404k -bufsize 1500k -pix_fmt yuv420p -profile:v high -level 4.0 -refs 3 -r 25 -g 50 -c:a libfdk_aac -b:a 96k -filter_complex "[0:a:0][0:a:1] amerge=inputs=2, dynaudnorm=f=500:g=31:p=0.82:m=30.0:r=0.0:s=0.0 [aout]" -ar 44100 -ac 2 -map 0:0 -map "[aout]" -movflags +faststart -threads 0 -y OUTPUT In reality, it looks like this
38. ./ffmpeg -i INPUT -c:v libx264 -filter_complex "[0:v]crop=720:576:0:32,yadif,scale=1024x576,setdar=16/9[ma in], [1:v]scale=54:-1[ovrl], [main][ovrl]overlay=main_w-overlay_w-47:main_h-overlay_h-44 " -crf 22 -maxrate 1404k -bufsize 1500k -pix_fmt yuv420p -profile:v high -level 4.0 -refs 3 -r 25 -g 50 -c:a libfdk_aac -b:a 96k -filter_complex "[0:a:0][0:a:1] amerge=inputs=2, dynaudnorm=f=500:g=31:p=0.82:m=30.0:r=0.0:s=0.0 [aout]" -ar 44100 -ac 2 -map 0:0 -map "[aout]" -movflags +faststart -threads 0 -y OUTPUT In reality, it looks like this
39. ./ffmpeg -i INPUT -c:v libx264 -filter_complex "[0:v]crop=720:576:0:32,yadif,scale=1024x576,setdar=16/9[ma in], [1:v]scale=54:-1[ovrl], [main][ovrl]overlay=main_w-overlay_w-47:main_h-overlay_h-44 " -crf 22 -maxrate 1404k -bufsize 1500k -pix_fmt yuv420p -profile:v high -level 4.0 -refs 3 -r 25 -g 50 -c:a libfdk_aac -b:a 96k -filter_complex "[0:a:0][0:a:1] amerge=inputs=2, dynaudnorm=f=500:g=31:p=0.82:m=30.0:r=0.0:s=0.0 [aout]" -ar 44100 -ac 2 -map 0:0 -map "[aout]" -movflags +faststart -threads 0 -y OUTPUT In reality, it looks like this
40. ./ffmpeg -i INPUT -c:v libx264 -filter_complex "[0:v]crop=720:576:0:32,yadif,scale=1024x576,setdar=16/9[ma in], [1:v]scale=54:-1[ovrl], [main][ovrl]overlay=main_w-overlay_w-47:main_h-overlay_h-44 " -crf 22 -maxrate 1404k -bufsize 1500k -pix_fmt yuv420p -profile:v high -level 4.0 -refs 3 -r 25 -g 50 -c:a libfdk_aac -b:a 96k -filter_complex "[0:a:0][0:a:1] amerge=inputs=2, dynaudnorm=f=500:g=31:p=0.82:m=30.0:r=0.0:s=0.0 [aout]" -ar 44100 -ac 2 -map 0:0 -map "[aout]" -movflags +faststart -threads 0 -y OUTPUT In reality, it looks like this
41. ./ffmpeg -i INPUT -c:v libx264 -filter_complex "[0:v]crop=720:576:0:32,yadif,scale=1024x576,setdar=16/9[ma in], [1:v]scale=54:-1[ovrl], [main][ovrl]overlay=main_w-overlay_w-47:main_h-overlay_h-44 " -crf 22 -maxrate 1404k -bufsize 1500k -pix_fmt yuv420p -profile:v high -level 4.0 -refs 3 -r 25 -g 50 -c:a libfdk_aac -b:a 96k -filter_complex "[0:a:0][0:a:1] amerge=inputs=2, dynaudnorm=f=500:g=31:p=0.82:m=30.0:r=0.0:s=0.0 [aout]" -ar 44100 -ac 2 -map 0:0 -map "[aout]" -movflags +faststart -threads 0 -y OUTPUT In reality, it looks like this
42. ./ffmpeg -i INPUT -c:v libx264 -filter_complex "[0:v]crop=720:576:0:32,yadif,scale=1024x576,setdar=16/9[ma in], [1:v]scale=54:-1[ovrl], [main][ovrl]overlay=main_w-overlay_w-47:main_h-overlay_h-44 " -crf 22 -maxrate 1404k -bufsize 1500k -pix_fmt yuv420p -profile:v high -level 4.0 -refs 3 -r 25 -g 50 -c:a libfdk_aac -b:a 96k -filter_complex "[0:a:0][0:a:1] amerge=inputs=2, dynaudnorm=f=500:g=31:p=0.82:m=30.0:r=0.0:s=0.0 [aout]" -ar 44100 -ac 2 -map 0:0 -map "[aout]" -movflags +faststart -threads 0 -y OUTPUT In reality, it looks like this
43. ./ffmpeg -i INPUT -c:v libx264 -filter_complex "[0:v]crop=720:576:0:32,yadif,scale=1024x576,setdar=16/9[ma in], [1:v]scale=54:-1[ovrl], [main][ovrl]overlay=main_w-overlay_w-47:main_h-overlay_h-44 " -crf 22 -maxrate 1404k -bufsize 1500k -pix_fmt yuv420p -profile:v high -level 4.0 -refs 3 -r 25 -g 50 -c:a libfdk_aac -b:a 96k -filter_complex "[0:a:0][0:a:1] amerge=inputs=2, dynaudnorm=f=500:g=31:p=0.82:m=30.0:r=0.0:s=0.0 [aout]" -ar 44100 -ac 2 -map 0:0 -map "[aout]" -movflags +faststart -threads 0 -y OUTPUT In reality, it looks like this
44. ./ffmpeg -i INPUT -c:v libx264 -filter_complex "[0:v]crop=720:576:0:32,yadif,scale=1024x576,setdar=16/9[ma in], [1:v]scale=54:-1[ovrl], [main][ovrl]overlay=main_w-overlay_w-47:main_h-overlay_h-44 " -crf 22 -maxrate 1404k -bufsize 1500k -pix_fmt yuv420p -profile:v high -level 4.0 -refs 3 -r 25 -g 50 -c:a libfdk_aac -b:a 96k -filter_complex "[0:a:0][0:a:1] amerge=inputs=2, dynaudnorm=f=500:g=31:p=0.82:m=30.0:r=0.0:s=0.0 [aout]" -ar 44100 -ac 2 -map 0:0 -map "[aout]" -movflags +faststart -threads 0 -y OUTPUT In reality, it looks like this
45. ./ffmpeg -i INPUT -c:v libx264 -filter_complex "[0:v]crop=720:576:0:32,yadif,scale=1024x576,setdar=16/9[ma in], [1:v]scale=54:-1[ovrl], [main][ovrl]overlay=main_w-overlay_w-47:main_h-overlay_h-44 " -crf 22 -maxrate 1404k -bufsize 1500k -pix_fmt yuv420p -profile:v high -level 4.0 -refs 3 -r 25 -g 50 -c:a libfdk_aac -b:a 96k -filter_complex "[0:a:0][0:a:1] amerge=inputs=2, dynaudnorm=f=500:g=31:p=0.82:m=30.0:r=0.0:s=0.0 [aout]" -ar 44100 -ac 2 -map 0:0 -map "[aout]" -movflags +faststart -threads 0 -y OUTPUT In reality, it looks like this
46. ./ffmpeg -i INPUT -c:v libx264 -filter_complex "[0:v]crop=720:576:0:32,yadif,scale=1024x576,setdar=16/9[ma in], [1:v]scale=54:-1[ovrl], [main][ovrl]overlay=main_w-overlay_w-47:main_h-overlay_h-44 " -crf 22 -maxrate 1404k -bufsize 1500k -pix_fmt yuv420p -profile:v high -level 4.0 -refs 3 -r 25 -g 50 -c:a libfdk_aac -b:a 96k -filter_complex "[0:a:0][0:a:1] amerge=inputs=2, dynaudnorm=f=500:g=31:p=0.82:m=30.0:r=0.0:s=0.0 [aout]" -ar 44100 -ac 2 -map 0:0 -map "[aout]" -movflags +faststart -threads 0 -y OUTPUT In reality, it looks like this
48. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
49. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
50. Cost Scale Flexibility
51. Cost Scale Flexibility
54. thesupercarkids.com
55. mini-me.com
56. Cost Scale Flexibility
58. “The Cloud”
59. Cost Scale Flexibility
61. Cost Scale Flexibility
62. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
63. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
64. Source Queue Process Transfer Notify
80. 8 minutes 27 seconds
81. Original length Time to produce rendition 60 min 7 min 42 min Lowest (audio) Highest (1500k)
83. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
84. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
85. Languages: Node.js Go
88. Node.js Existing team experience Rapid development for APIs Untyped. Sadface.
90. Go Typed and compiled Cross-platform Easy deployment Learning curve
91. Infrastructure: EC2, SQS, S3, SNS, Dynamo / RDS, Cloudformation
92. AWS - EC2 Autoscaling
95. AWS - SQS Simple Queuing System
97. SQS First in first out (FIFO) Single consumer per message Message has “visibility timeout” Adjustable visibility timeout
98. AWS - S3 File Storage
100. AWS - DynamoDB
101. AWS - DynamoDB
102. AWS - DynamoDB AWS - MySQL on RDS
104. DynamoDB NoSQL Key-value pair storage Pay for number of records/sec read
105. DynamoDB Our use case was not optimal Manual aggregations Higher cost than expected Doesn’t scale automatically
106. MySQL Switched to MySQL on RDS Managed hosting Aggregations! Data structure ported as-is
107. AWS Cloudformation
109. "ExampleLoadBalancer" : { "Type" : "AWS::ElasticLoadBalancing::LoadBalancer", "Properties" : { "AvailabilityZones" : [ "us-east-1a", "us-east-1b" ], "Listeners" : [ { "LoadBalancerPort" : "80", "InstancePort" : "8081", "Protocol" : "HTTP" } ] } }
111. Node.js + Go AWS Infrastructure
112. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
113. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
115. When should you scale?
116. Scaling Considerations Hourly billing Instance size: Cost vs Speed Content priority Ad-hoc requests Varied content length
117. Scaling Considerations: AWS limitations AWS scaling metrics can be slow No automatic off switch Can’t choose instances to scale down (Instance protection now available)
119. MVP: Don’t make things wait too long
120. MVP: Supply enough capacity at all times
121. MVP: Driven by drop folders
122. GXF, ABC1 Profile: - Renditions (2) - Crop - Normalise audio - Adjust ratio - Add watermark - GXF to MP4
128. var autoscaling = new AWS.AutoScaling(...); var params = { AutoScalingGroupName: groupName, DesiredCapacity: capacity Some white text }; autoscaling.setDesiredCapacity(params, ...);
129. var autoscaling = new AWS.AutoScaling(...); var params = { AutoScalingGroupName: groupName, DesiredCapacity: capacity Some white text }; autoscaling.setDesiredCapacity(params, ...);
130. Amazon Machine Images (AMIs)
131. Which instance size?
132. Time vs Cost for 1 hour of content Time Cost ?
133. Winner: c4.xlarge
134. Up: step scaling Down: stop everything
140. An interesting puzzle :)
141. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
142. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
143. 24 hours in Transcoding
144. (10am - 10am)
145. $57.22
146. $42.86 US Dollars
147. EC2 Instances: $34.31 EC2 (other): $ 4.24 RDS: $ 2.66 S3: $ 1.65 SQS: $ 0.00 Total (USD): $42.86 Transcoding costs: 24 hrs
148. Next 24 hours Approx 70% of the volume
149. $34.63 Australian Dollars
150. One year’s costs?
151. $16.5k Australian Dollars
152. $12.5k US Dollars
153. This is a BIG deal!
154. thesupercarkids.com
155. bikesandbeyond.ca
156. 8 minutes 27 seconds
158. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
159. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
160. AWS & throughput optimisations
163. Transcoding in chunks/segments
165. Introspection & Validation
166. GXF, ABC1 Profile: - Crop - Normalise audio - Adjust ratio - Add watermark - GXF to MP4
167. ffprobe
171. 1. What is Transcoding? 2. Why build our own system? 3. Architecture 4. Technology choices 5. Autoscaling transcoders 6. Costs 7. Future roadmap
172. Summary
177. 8 minutes 27 seconds
179. @daphnechong Thanks!
180. Q&A