UGC, Transcoding and Container Orchestration @ BR

This presentation covers three projects using Amazon Web Services at Bayerischer Rundfunk: user generated content, video transcoding and container orchestration

Transcript

1. UGC, Transcoding & Containers at Bayerischer Rundfunk Philipp Dressel Softwareentwicklung und Plattformen, BR
5. BR24 App
6. BR24 App Name Location Description Contact
7. API Gateway Metadata BR24 App
8. API Gateway Metadata BR24 App Lambda Metadata
9. API Gateway DynamoDB Metadata BR24 App Metadata Lambda Metadata
10. API Gateway DynamoDB Metadata BR24 App Metadata Lambda Metadata
11. API Gateway DynamoDB Metadata BR24 App S3 URL Metadata Lambda Metadata S3 URL
12. API Gateway S3 DynamoDB Metadata BR24 App S3 URL Media Metadata Lambda Metadata S3 URL
13. API Gateway S3 DynamoDB Metadata BR24 App S3 URL Lambda Media Metadata Lambda Metadata Completed S3 URL
14. API Gateway S3 DynamoDB Metadata BR24 App S3 URL Lambda Media Metadata Completed Lambda Metadata Completed S3 URL
15. API Gateway S3 DynamoDB Metadata Website BR24 App S3 URL Lambda Media Metadata Completed Lambda Region Metadata Completed S3 URL
16. API Gateway S3 DynamoDB Metadata Website BR24 App S3 URL Lambda Media Metadata Completed Lambda Lambda Region Metadata Region Completed S3 URL
17. API Gateway S3 DynamoDB Metadata Website BR24 App S3 URL Lambda Media Metadata Completed Lambda Lambda Region Region Metadata Region Completed S3 URL
18. API Gateway S3 DynamoDB Metadata Website BR24 App S3 URL Lambda Media Metadata Completed Lambda Lambda Region Count Region Count Metadata Region Completed Count S3 URL
19. Lessons learned
22. BR24 CMS
23. BR24 CMS Name Media File
24. API Gateway Metadata BR24 CMS S3 URL Lambda Metadata S3 URL
25. API Gateway DynamoDB Metadata BR24 CMS S3 URL Lambda S3 URL Metadata Metadata
26. API Gateway S3 Input DynamoDB Metadata BR24 CMS S3 URL Media Metadata Lambda Metadata S3 URL
27. API Gateway S3 Input DynamoDB Metadata BR24 CMS S3 URL Lambda Media Metadata Lambda Metadata Completed S3 URL
28. API Gateway S3 Input DynamoDB Metadata BR24 CMS S3 URL Lambda Media Metadata Start Job Lambda Metadata Completed S3 URL Elastic Transcoder
29. API Gateway S3 Input DynamoDB Metadata BR24 CMS S3 URL Lambda Media Metadata Start Job Lambda Metadata Completed S3 URL Elastic Transcoder Input Media
30. API Gateway S3 Input DynamoDB Metadata BR24 CMS S3 URL Lambda Media Metadata Start Job Lambda Metadata Completed S3 URL Elastic Transcoder S3 Output Input Media Output Media
31. API Gateway S3 Input DynamoDB Metadata BR24 CMS S3 URL Lambda Media Metadata Start Job Lambda Metadata Completed S3 URL Elastic Transcoder S3 Output Input Media Output Media Media CDN
32. Elastic Transcoder
34. Project Repository
35. Project Repository Services Deployment Build Ingress
36. Frontend Backend Database Project Repository Submodules Services Deployment Build Ingress
37. Frontend Backend Database Project Repository Submodules Code
 Dockerfile Dockerfile Code
 Dockerfile
38. Frontend Backend Database Project Repository Submodules Code
 Dockerfile Dockerfile Code
 Dockerfile Image Image Image
39. Frontend Backend Database Project Repository Submodules Code
 Dockerfile Dockerfile Code
 Dockerfile Image Image Image Deployment Deployment Deployment
40. Frontend Backend Database Project Repository Submodules AWS ELB Code
 Dockerfile Dockerfile Code
 Dockerfile Image Image Image Deployment Deployment Deployment
41. master staging feature Project Repository Branches Code
 Dockerfile Dockerfile Code
 Dockerfile Image Image Image Deployment AWS ELB Deployment Deployment Code
 Dockerfile Dockerfile Code
 Dockerfile Image Image Image Deployment AWS ELB Deployment Deployment Code
 Dockerfile Dockerfile Code
 Dockerfile Image Image Image Deployment AWS ELB Deployment Deployment
42. frontend.staging.project.cluster.io frontend master staging feature Project Repository Branches Code
 Dockerfile Dockerfile Code
 Dockerfile Image Image Image Deployment AWS ELB Deployment Deployment Code
 Dockerfile Dockerfile Code
 Dockerfile Image Image Image Deployment AWS ELB Deployment Deployment Code
 Dockerfile Dockerfile Code
 Dockerfile Image Image Image Deployment AWS ELB Deployment Deployment staging project
45. VPC
46. VPC AZ a ASG M1 Master 1
47. VPC AZ a ASG M1 Master 1 ASG W1 Worker 1 Worker 2
48. VPC AZ a ASG M1 Master 1 ASG W1 Worker 1 Worker 2 Worker 3
49. VPC AZ a ASG M1 Master 1 ASG W1 ASG W2 Worker 1 Worker 2 Worker 3 Worker 4
50. VPC AZ b AZ c AZ a ASG M3 Master 3 ASG M2 Master 2 ASG M1 Master 1 ASG W1 ASG W2 Worker 1 Worker 2 Worker 3 Worker 5 Worker 6 Worker 8 Worker 9 Worker 10 Worker 4 Worker 7
51. VPC AZ b AZ c AZ a ASG M3 Master 3 ASG M2 Master 2 ASG M1 Master 1 ASG W1 ASG W2 Worker 1 Worker 2 Worker 3 Worker 5 Worker 6 Worker 8 Worker 9 Worker 10 Worker 4 Worker 7
52. VPC AZ b AZ c AZ a ASG M3 Master 3 ASG M2 Master 2 ASG M1 Master 1 ASG W1 ASG W2 Worker 1 Worker 2 Worker 3 Worker 5 Worker 6 Worker 8 Worker 9 Worker 4 Worker 7
53. VPC AZ b AZ c AZ a ASG M3 Master 3 ASG M2 Master 2 ASG M1 Master 1 ASG W1 ASG W2 Worker 1 Worker 2 Worker 3 Worker 5 Worker 6 Worker 8 Worker 9 Worker 4 Worker 7 Worker 10
54. VPC AZ b AZ c AZ a ASG M3 Master 3 ASG M2 Master 2 ASG M1 Master 1 ASG W1 ASG W2 Worker 1 Worker 2 Worker 3 Worker 5 Worker 6 Worker 8 Worker 9 Worker 4 Worker 7 Worker 10
55. VPC AZ b AZ a ASG M2 Master 2 ASG M1 Master 1 ASG W1 ASG W2 Worker 1 Worker 2 Worker 3 Worker 5 Worker 6 Worker 4 Worker 7 Worker 10
56. VPC AZ b AZ a ASG M2 Master 2 ASG M1 Master 1 ASG W1 ASG W2 Worker 1 Worker 2 Worker 3 Worker 5 Worker 6 Worker 4 Worker 7 Worker 10
57. VPC AZ b AZ a ASG M2 Master 2 ASG M1 Master 1 ASG W1 ASG W2 Worker 1 Worker 2 Worker 3 Worker 5 Worker 6 Worker 4 Worker 7 Worker 10 Worker 8 Worker 9
58. VPC AZ c AZ b AZ a ASG M2 Master 2 ASG M1 Master 1 ASG W1 ASG W2 Worker 1 Worker 2 Worker 3 Worker 5 Worker 6 Worker 4 Worker 7 Worker 10 Worker 8 Worker 9 ASG M3 Master 3
59. VPC AZ b AZ c AZ a ASG M3 Master 3 ASG M2 Master 2 ASG M1 Master 1 ASG W1 ASG W2 Worker 1 Worker 2 Worker 3 Worker 5 Worker 6 Worker 8 Worker 9 Worker 10 Worker 4 Worker 7
60. Persistence
61. Worker 1 Worker 2 Worker 3 Worker 4 Work Overlay Network Elastic Loadbalancer
62. Worker 1 Worker 2 Worker 3 Worker 4 Work Overlay Network Elastic Loadbalancer
63. Worker 1 Worker 2 Worker 3 Worker 4 Work Overlay Network Elastic Loadbalancer
64. Worker 1 Worker 2 Worker 3 Worker 4 Work Overlay Network Elastic Loadbalancer
65. Worker 1 Worker 2 Worker 3 Worker 4 Work Overlay Network Elastic Loadbalancer
66. Worker 1 Worker 2 Worker 3 Worker 4 Work Overlay Network Elastic Loadbalancer
67. Worker 1 Worker 2 Worker 3 Worker 4 Work Overlay Network Elastic Loadbalancer
68. Worker 1 Worker 2 Worker 3 Worker 4 Work Overlay Network Elastic Loadbalancer
69. Worker 1 Worker 2 Worker 3 Worker 4 Work Overlay Network Elastic Loadbalancer
72. Logging
74. Monitoring
76. Thank you! philipp.dressel@br.de @philipp_dressel pdressel