Traefik + RiksTV

Traefik Online Meetup: RiksTV's Cloud Infrastructure Using Traefik. A Deep Dive and Demo by Trond Hindenes, SRE team lead at RiksTV

Description:
RiksTV is a Norwegian broadcast distributor and streaming services provider. Trond Hindenes, is the SRE team lead at RiksTV.

Traefik has been an important part of RiksTVs cloud infrastructure since day one, both for traditional services and for our Kubernetes-based workloads. In this session Trond will show how Traefik fits into RiksTVs stack, and how it fits together with other products such as ALB, Kubernetes and Consul.

Transcript

1. Traefik + RiksTV Trond Hindenes , SRE lead @trondhindenes
2. What is RiksTV
3. What is RiksTV - Traditionally, very Windows/.Net-centric - Currently moving all our stuff to AWS - Mix of “traditional vm” and kubernetes-based infrastructure
4. Traefik usage at RiksTV - “Everything” passes thru Traefik - We use it both for non-containerized and containerized (K8s) services
5. Traefik for non-containerized services
6. Some design points - Traefik nodes are defined in CloudFormation, we can increase capacity in a matter of minutes - SSL termination happens at ALB (we don’t use Traefik’s awesome LetsEncrypt integration) - Use aggressive health checking from ALB to Traefik - Consul Tags dictate internal/external endpoint - completely up to devs
7. Learnings from incidents and close calls - Take care to design the shutdown/drain process properly (Traefik has all the bells and whistles - use them!) - “stale” consul setting adds robustness - As always, watch open file descriptors - you could exceed the default if you get significant traffic (we did)
8. All the logs! - Traefik produces a json-structured log format, which is super-easy to parse in logstash etc - We’re moving from vm-based to traefik-based access logging
9. Dev responsibility - Add a “Consul Service definition” file to the app (which we pick up automatically during deploy) - Add a robust health endpoint
10. Some Config Details
11. Demo: Node Failure
12. Demo: Logging Visualization
13. Traefik in Kubernetes at RiksTV - RiksTV is in a state of “Kubernetes ramp-up” - we use it for most new services (and are slowly refactoring old ones too) - Challenge: Clean separation between “external” and “internal” endpoints -
14. Kubernetes Ingress primer
15. Traefik Kubernetes Design - Each Kubernetes worker has 4 traefik instances configured in Daemonsets - Traefik pods are published using HostPort (known port) - ALB is “aggressively probing” for health
16. Things we learned - Use “terminationGracePeriodSeconds” to allow draining connections when shutting down (again, Traefik has all the bells and whistles to make this work nicely) - Use regular CI/CD pipeline to deploy Traefik pod updates (make sure to use “RollingUpdate” in DaemonSet
17. Demo: Traefik Jaeger
18. Dev responsibility - We’ve abstracted away all Kubernetes minutae into our custom “ContainerDeploymentV2” library - Devs simply set hostnames, and internal/external and we do the rest
19. Some Config Details
20. Demo: Upgrading Traefik itself
21. Bonus Track: Hybrid request routing - Strangler-pattern type scenarios where some requests should be routed to “legacy intra” and some to kubernets-based pods
22. Demo: Hybrid routing with Traefik
23. A few links https://medium.com/@trondhindenes/exploring-hybrid-request-routing-in-order -to-support-strangler-pattern-migrations-to-kubernetes-7280b10058d1 https://medium.com/@trondhindenes/using-traefik-as-a-kubernetes-ingress-co ntroller-for-both-internal-and-external-traffic-c06e4177314 https://medium.com/jaegertracing/exploring-distributed-tracing-using-traefik-j aeger-and-flask-4f1835c8ed8d
24. Thank you!!