Avoid the maze of container networking: how we built a simple distributed SDN for Docker

Containerized applications had been the biggest driver for a number of SDN projects which came to life in the last year. There are several challenges that highly-dynamic and fully-distributed container orchestration systems impose on SDN architectures.

This presentation is geared towards anyone working with open source SDN technologies, as well as those who would like to ensure the robustness of their containers' connections / of their container connectivity.

Several approaches/options for container networking will be presented, including Weave Net, an open source overlay network for Docker containers. We will demonstrate the in-kernel Open vSwitch datapath used in Weave Net; this manages VXLAN tunnel configuration across hosts, without requiring the user to be a networking expert. We will also highlight how Weave Net provides service discovery and load balancing through a robust, distributed DNS. Users don't need to understand the underlying complexity, nor do they need to make changes to their application.

In this talk, Weave Net's engineers will share what is required to deliver this level of user experience, through the implementation decisions they have made.

Containerized applications had been the biggest driver for a number of SDN projects which came to life in the last year. There are several challenges that highly-dynamic and fully-distributed container orchestration systems impose on SDN architectures.

This presentation is geared towards anyone working with open source SDN technologies, as well as those who would like to ensure the robustness of their containers' connections / of their container connectivity.

Several approaches/options for container networking will be presented, including Weave Net, an open source overlay network for Docker containers. We will demonstrate the in-kernel Open vSwitch datapath used in Weave Net; this manages VXLAN tunnel configuration across hosts, without requiring the user to be a networking expert. We will also highlight how Weave Net provides service discovery and load balancing through a robust, distributed DNS. Users don't need to understand the underlying complexity, nor do they need to make changes to their application.

In this talk, Weave Net's engineers will share what is required to deliver this level of user experience, through the implementation decisions they have made.

https://fosdem.org/2016/schedule/speaker/ilya_dmitrichenko/

Transcript

1. Avoid the maze of container networking how we built a simple distributed SDN for Docker Ilya Dmitrichenko (@errordeveloper) FOSDEM 2016
2. • The Container Phenomenon • Challenges in Container Networking • Introduction to Weave Net • Usage example • Alternatives • Use Cases
3. Linux Containers •BSD Jails and Solaris Zones existed for quite sometime •Raw Linux containers are hard for developers to use •Docker made it very easy for everyone, kickstarted the new world •Many orchestration systems emerged — Kubernetes, Swarm, … •Many new open-source PaaS solutions grew: •Doku, Rancher, Kontena •Deis, OpenShift 3 •And commercial: Tutum, Cloud 66, Giant Swarm, DCHQ…
4. Container Networks •How will underlying network connect my app containers? •simple bridging doesn’t scale •VXLAN, VPN, BGP •OVS, DPDK •IaaS vendor-specific features •Orchestration systems can manage underlying IaaS network •PaaS can manage an IaaS network or manage its own overlay •But all of these are hard for application developers to grasp
5. (AppDev || DevOps) != NetOps
6. Weave Net • Lightweight application-oriented micro-SDN • Extremely easy to setup • Simple to manage and scale • No user training required • No code changes • Works with any container orchestrator • Built-in service discovery through DNS * — not a hard dependency!
7. The design of Weave Net • All peers are equal and each manages local: • IP address allocation and DNS records • VXLAN/OVS flows and/or packet forwarding • Flexibility and resilient mesh structure • No local dependencies other than Docker* • No external K/V stores, unlike other solutions * — not a hard dependency!
8. Forwarding Plane • Infrastructure agnostic • Supports multicast in any environment
9. Forwarding Plane — ODP User Space Kernel Space Network Host1 Host2 User App. User App. OVS Module OVS Module Weave Router Weave Router
10. Forwarding Plane — ODP • Fast data path (fastdp) • CONFIG_OPENVSWITCH_VXLAN • Managed via a Go library • github.com/weaveworks/go-odp • No user-space dependencies, such as OVDB
11. Forwarding Plane — user-space User Space Kernel Space Network Host1 Host2 User App. User App. Weave Router Weave Router
12. Forwarding Plane — user-space • UDP encapsulation • NaCL encryption • No kernel dependencies • Dynamic MTU size
13. Control Plane • Peer-to-peer mesh • Can be partially connected • Partition-tolerant & eventually consistent • Enables multi-cloud & cross-DC • Very simple to deploy and scale • No external K/V store
14. Control Plane • Transparent and fully-dynamic IPAM • Fast in-memory distributed DNS • service discovery • simple round-robin load balancing • Users don’t need to know about IPs
15. Usage — installation curl -s -L git.io/weave \ -o /usr/local/bin/weave chmod +x /usr/local/bin/weave
16. Usage — launch router and run containers weave launch host2 eval $(weave env) docker run … weave launch host1 eval $(weave env) docker run … host1: host2:
17. Usage — launch router and run containers docker run -tid \ —-name=pingme \ ubuntu docker run \ ubuntu \ ping -q —c1 pingme PING a2.weave.local (10.40.0.2) 56(84) bytes of data. --- pingme.weave.local ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.341/0.341/0.341/0.000 ms host1: host2:
18. Alternatives — bridge config and routing • Assign a CIDR to each host • Setup static routes (options vary) • External management layer required • Scaling properties depend on local setup • No explicit isolation • Depends external firewall
19. Alternatives — port mapping • Expose container port to host ports • External management layer required • Unique IP address are not guaranteed • Service discovery involves port look-up • Isolation is compromised • Depends on external firewall
20. Alternatives — host networking • No isolation at all • Only works for simplest use-cases • Service discovery involves port look-up
21. Usage — all new “Docker Network” # docker network
22. Usage — all new “Docker Network” One doesn’t just type: # docker network
23. Alternatives — all new “Docker Network” • Basic VXLAN, not OVS-based • Plugin framework (still immature, lacks DNS) • Requires external key-value sore • Strong consistency is hard • It is just not quite as simple!
24. Alternatives — all new “Docker Network” • Which key-value store to use? • Consul, etcd or Zookeeper? • Kubernetes & Mesos use some of those… • Is commercial support available? • How is best to setup the key-value store? • How to manage and scale the key-value store?
25. More Alternatives? You still have other problems to solve!
26. Weave Net Use Cases
27. Weave Net Use Cases
28. Weave Net Use Cases
29. Weave Net Use Cases
30. Questions!? weave-users@weave.works @errordeveloper @weaveworks github.com/weaveworks/weave