Avoid the maze of container networking how we built a simple distributed SDN for Docker Ilya Dmitrichenko (@errordeveloper) FOSDEM 2016 

• The Container Phenomenon • Challenges in Container Networking • Introduction to Weave Net • Usage example • Alternatives • Use Cases 

Linux Containers •BSD Jails and Solaris Zones existed for quite sometime •Raw Linux containers are hard for developers to use •Docker made it very easy for everyone, kickstarted the new world •Many orchestration systems emerged — Kubernetes, Swarm, … •Many new open-source PaaS solutions grew: •Doku, Rancher, Kontena •Deis, OpenShift 3 •And commercial: Tutum, Cloud 66, Giant Swarm, DCHQ… 

Container Networks •How will underlying network connect my app containers? •simple bridging doesn’t scale •VXLAN, VPN, BGP •OVS, DPDK •IaaS vendor-specific features •Orchestration systems can manage underlying IaaS network •PaaS can manage an IaaS network or manage its own overlay •But all of these are hard for application developers to grasp 

(AppDev || DevOps) != NetOps 

Weave Net • Lightweight application-oriented micro-SDN • Extremely easy to setup • Simple to manage and scale • No user training required • No code changes • Works with any container orchestrator • Built-in service discovery through DNS * — not a hard dependency! 

The design of Weave Net • All peers are equal and each manages local: • IP address allocation and DNS records • VXLAN/OVS flows and/or packet forwarding • Flexibility and resilient mesh structure • No local dependencies other than Docker* • No external K/V stores, unlike other solutions * — not a hard dependency! 

Forwarding Plane • Infrastructure agnostic • Supports multicast in any environment 

Forwarding Plane — ODP User Space Kernel Space Network Host1 Host2 User App. User App. OVS Module OVS Module Weave Router Weave Router 

Forwarding Plane — ODP • Fast data path (fastdp) • CONFIG_OPENVSWITCH_VXLAN • Managed via a Go library • github.com/weaveworks/go-odp • No user-space dependencies, such as OVDB 

Forwarding Plane — user-space User Space Kernel Space Network Host1 Host2 User App. User App. Weave Router Weave Router 

Forwarding Plane — user-space • UDP encapsulation • NaCL encryption • No kernel dependencies • Dynamic MTU size 

Control Plane • Peer-to-peer mesh • Can be partially connected • Partition-tolerant & eventually consistent • Enables multi-cloud & cross-DC • Very simple to deploy and scale • No external K/V store 

Control Plane • Transparent and fully-dynamic IPAM • Fast in-memory distributed DNS • service discovery • simple round-robin load balancing • Users don’t need to know about IPs 

Usage — installation curl -s -L git.io/weave \ -o /usr/local/bin/weave chmod +x /usr/local/bin/weave 

Usage — launch router and run containers weave launch host2 eval $(weave env) docker run … weave launch host1 eval $(weave env) docker run … host1: host2: 

Usage — launch router and run containers docker run -tid \ —-name=pingme \ ubuntu docker run \ ubuntu \ ping -q —c1 pingme PING a2.weave.local (10.40.0.2) 56(84) bytes of data. --- pingme.weave.local ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.341/0.341/0.341/0.000 ms host1: host2: 

Alternatives — bridge config and routing • Assign a CIDR to each host • Setup static routes (options vary) • External management layer required • Scaling properties depend on local setup • No explicit isolation • Depends external firewall 

Alternatives — port mapping • Expose container port to host ports • External management layer required • Unique IP address are not guaranteed • Service discovery involves port look-up • Isolation is compromised • Depends on external firewall 

Alternatives — host networking • No isolation at all • Only works for simplest use-cases • Service discovery involves port look-up 

Usage — all new “Docker Network” # docker network 

Usage — all new “Docker Network” One doesn’t just type: # docker network 

Alternatives — all new “Docker Network” • Basic VXLAN, not OVS-based • Plugin framework (still immature, lacks DNS) • Requires external key-value sore • Strong consistency is hard • It is just not quite as simple! 

Alternatives — all new “Docker Network” • Which key-value store to use? • Consul, etcd or Zookeeper? • Kubernetes & Mesos use some of those… • Is commercial support available? • How is best to setup the key-value store? • How to manage and scale the key-value store? 

More Alternatives? You still have other problems to solve! 

Weave Net Use Cases 

Weave Net Use Cases 

Weave Net Use Cases 

Weave Net Use Cases 

Questions!? [email protected] @errordeveloper @weaveworks github.com/weaveworks/weave