Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Avoid the maze of container networking: how we built a simple distributed SDN for Docker

Avoid the maze of container networking: how we built a simple distributed SDN for Docker

Containerized applications had been the biggest driver for a number of SDN projects which came to life in the last year. There are several challenges that highly-dynamic and fully-distributed container orchestration systems impose on SDN architectures.

This presentation is geared towards anyone working with open source SDN technologies, as well as those who would like to ensure the robustness of their containers' connections / of their container connectivity.

Several approaches/options for container networking will be presented, including Weave Net, an open source overlay network for Docker containers. We will demonstrate the in-kernel Open vSwitch datapath used in Weave Net; this manages VXLAN tunnel configuration across hosts, without requiring the user to be a networking expert. We will also highlight how Weave Net provides service discovery and load balancing through a robust, distributed DNS. Users don't need to understand the underlying complexity, nor do they need to make changes to their application.

In this talk, Weave Net's engineers will share what is required to deliver this level of user experience, through the implementation decisions they have made.

Containerized applications had been the biggest driver for a number of SDN projects which came to life in the last year. There are several challenges that highly-dynamic and fully-distributed container orchestration systems impose on SDN architectures.

This presentation is geared towards anyone working with open source SDN technologies, as well as those who would like to ensure the robustness of their containers' connections / of their container connectivity.

Several approaches/options for container networking will be presented, including Weave Net, an open source overlay network for Docker containers. We will demonstrate the in-kernel Open vSwitch datapath used in Weave Net; this manages VXLAN tunnel configuration across hosts, without requiring the user to be a networking expert. We will also highlight how Weave Net provides service discovery and load balancing through a robust, distributed DNS. Users don't need to understand the underlying complexity, nor do they need to make changes to their application.

In this talk, Weave Net's engineers will share what is required to deliver this level of user experience, through the implementation decisions they have made.

https://fosdem.org/2016/schedule/speaker/ilya_dmitrichenko/

121dcfb4f05ed316247772545e2cd590?s=128

Ilya Dmitrichenko

January 31, 2016
Tweet

Transcript

  1. Avoid the maze of container networking how we built a

    simple distributed SDN for Docker Ilya Dmitrichenko (@errordeveloper) FOSDEM 2016
  2. • The Container Phenomenon • Challenges in Container Networking •

    Introduction to Weave Net • Usage example • Alternatives • Use Cases
  3. Linux Containers •BSD Jails and Solaris Zones existed for quite

    sometime •Raw Linux containers are hard for developers to use •Docker made it very easy for everyone, kickstarted the new world •Many orchestration systems emerged — Kubernetes, Swarm, … •Many new open-source PaaS solutions grew: •Doku, Rancher, Kontena •Deis, OpenShift 3 •And commercial: Tutum, Cloud 66, Giant Swarm, DCHQ…
  4. Container Networks •How will underlying network connect my app containers?

    •simple bridging doesn’t scale •VXLAN, VPN, BGP •OVS, DPDK •IaaS vendor-specific features •Orchestration systems can manage underlying IaaS network •PaaS can manage an IaaS network or manage its own overlay •But all of these are hard for application developers to grasp
  5. (AppDev || DevOps) != NetOps

  6. Weave Net • Lightweight application-oriented micro-SDN • Extremely easy to

    setup • Simple to manage and scale • No user training required • No code changes • Works with any container orchestrator • Built-in service discovery through DNS * — not a hard dependency!
  7. The design of Weave Net • All peers are equal

    and each manages local: • IP address allocation and DNS records • VXLAN/OVS flows and/or packet forwarding • Flexibility and resilient mesh structure • No local dependencies other than Docker* • No external K/V stores, unlike other solutions * — not a hard dependency!
  8. Forwarding Plane • Infrastructure agnostic • Supports multicast in any

    environment
  9. Forwarding Plane — ODP User Space Kernel Space Network Host1

    Host2 User App. User App. OVS Module OVS Module Weave Router Weave Router
  10. Forwarding Plane — ODP • Fast data path (fastdp) •

    CONFIG_OPENVSWITCH_VXLAN • Managed via a Go library • github.com/weaveworks/go-odp • No user-space dependencies, such as OVDB
  11. Forwarding Plane — user-space User Space Kernel Space Network Host1

    Host2 User App. User App. Weave Router Weave Router
  12. Forwarding Plane — user-space • UDP encapsulation • NaCL encryption

    • No kernel dependencies • Dynamic MTU size
  13. Control Plane • Peer-to-peer mesh • Can be partially connected

    • Partition-tolerant & eventually consistent • Enables multi-cloud & cross-DC • Very simple to deploy and scale • No external K/V store
  14. Control Plane • Transparent and fully-dynamic IPAM • Fast in-memory

    distributed DNS • service discovery • simple round-robin load balancing • Users don’t need to know about IPs
  15. Usage — installation curl -s -L git.io/weave \ -o /usr/local/bin/weave

    chmod +x /usr/local/bin/weave
  16. Usage — launch router and run containers weave launch host2

    eval $(weave env) docker run … weave launch host1 eval $(weave env) docker run … host1: host2:
  17. Usage — launch router and run containers docker run -tid

    \ —-name=pingme \ ubuntu docker run \ ubuntu \ ping -q —c1 pingme PING a2.weave.local (10.40.0.2) 56(84) bytes of data. --- pingme.weave.local ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.341/0.341/0.341/0.000 ms host1: host2:
  18. Alternatives — bridge config and routing • Assign a CIDR

    to each host • Setup static routes (options vary) • External management layer required • Scaling properties depend on local setup • No explicit isolation • Depends external firewall
  19. Alternatives — port mapping • Expose container port to host

    ports • External management layer required • Unique IP address are not guaranteed • Service discovery involves port look-up • Isolation is compromised • Depends on external firewall
  20. Alternatives — host networking • No isolation at all •

    Only works for simplest use-cases • Service discovery involves port look-up
  21. Usage — all new “Docker Network” # docker network

  22. Usage — all new “Docker Network” One doesn’t just type:

    # docker network
  23. Alternatives — all new “Docker Network” • Basic VXLAN, not

    OVS-based • Plugin framework (still immature, lacks DNS) • Requires external key-value sore • Strong consistency is hard • It is just not quite as simple!
  24. Alternatives — all new “Docker Network” • Which key-value store

    to use? • Consul, etcd or Zookeeper? • Kubernetes & Mesos use some of those… • Is commercial support available? • How is best to setup the key-value store? • How to manage and scale the key-value store?
  25. More Alternatives? You still have other problems to solve!

  26. Weave Net Use Cases

  27. Weave Net Use Cases

  28. Weave Net Use Cases

  29. Weave Net Use Cases

  30. Questions!? weave-users@weave.works @errordeveloper @weaveworks github.com/weaveworks/weave