OpenSolaris Network

Transcript

1. Frankfurter OpenSolaris User Group Aklilu Hadish Senior Systems Engineer Sun Microsystems GmbH Germany / Langen
2. Agenda: Solaris Networking Crossbow Network Virtualization IP- Instances ... Clearview GLDv3 (Nemo) FireEngine
3. Pre - S10 TCP/IP Stack Pre - S10 TCP/IP Stack Solaris 2.x TCP/IP stack Uses STREAMS horizontal perimeter Setting up a STREAMs was expensive Connections were more long lived (NFS, ftp, etc.), Amortization over the life of the connection The world has changed: Web applications generated packets of heavy loads Multimedia and other real-time application require Real-time Constant-Rate Low-Latency Service Multimedia mostly datagram based connection No flow control
4. Solaris must deal with higher short-lived connections Servers became heavily MP/CMT and NUMA centric running large number of CPUs/Cores. STREAMs by design did not have any CPU affinity packets for a particular connections moved around to different CPU. The cost of context switching became high Solaris needed to move away from STREAMs architecture Pre - S10 TCP/IP Stack Pre - S10 TCP/IP Stack
5. Project FireEngine Project FireEngine
6. FireEngine FireEngine
7. FireEngine (continued ...) FireEngine (continued ...)
8. FireEngine (continued ...) FireEngine (continued ...)
9. FireEngine (continued ...) FireEngine (continued ...)
10. FireEngine (continued ...) FireEngine (continued ...)
11. FireEngine (continued ...) FireEngine (continued ...)
12. Nemo - GLDv3 Nemo - GLDv3
13. Nemo Nemo GLDv3 •Interrupt Moderation •VLAN •Trunking •IPMP •IP-Tunneling
14. Pre-Nemo Pre-Nemo VLAN Implementation DLPI: Monolitic driver utilizing the Generic LAN Driver (GLD)
15. Enter Nemo Enter Nemo
16. Nemo – New-Architecture Nemo – New-Architecture Virtualization of the Layer two of the Network Stack Performance improved: Kernel clients avoid using DLPI Avoid DLPI within the DLS and MAC modules VLAN Implementation
17. Nemo Nemo • The Nemo architecture: •Provides DLPI interface to the network layer services •Furthermore it provides a performance boost •Function call-based interface. •It virtualizes the Data Link layer of the network stack •No longer a one-to-one correspondence network interfaces and devices.
18. Nemo – GLDv3 Interrupt Moderation Nemo – GLDv3 Interrupt Moderation
19. Nemo – Interrupt Moderation Nemo – Interrupt Moderation
20. Nemo - Trunking Nemo - Trunking
21. Clearview Clearview
22. Clearview Clearview
23. Clearview Clearview
24. Clearview - Vanity Naming: Clearview - Vanity Naming:
25. Clearview - Vanity Naming Clearview - Vanity Naming
26. Clearview – IP Tunneling Clearview – IP Tunneling GLDV3 Driver GLDV3 Driver
27. IP Instances IP Instances
28. IP Instances IP Instances • Provide a more robust architecture for zones •networking •Provides per-zone routing tables •Provides per-zone ARP table •i.e. provide an instance of IP per-zone
29. IP Instances IP Instances
30. • Shared stack •Uses global routing table •Global interface management, performance tuning, etc •Short-cut routing •Packet filtering applied to the entire machine • Exclusive Instances •Routing table per zone •Each zone can tweak TCP, etc, settings in its own way •Each zone decides what filtering it wants IP Instances - Zones Networking IP Instances - Zones Networking S hared vs Exclusive Instances
31. • An exclusive instance cannot be managed from the global zone • Local zone root has full control over IP •Use of ndd to tune IP is allowed and is private • Network interfaces delegated for exclusive use are not visible in the global zone S hared vs Exclusive Instances IP Instances - Zones Networking IP Instances - Zones Networking
32. IP Instances – Changes to security • Privilege split of SYS_NET_CONFIG •SYS_IP_CONFIG (new) for zones • Can snoop from inside a zone •But can still snoop using the interface in global too! • Security Threat •Zone can forge ethernet packets IP Instances - Zones Networking IP Instances - Zones Networking
33. Crossbow Crossbow Network Network Virtualization Virtualization
34. • Stack and NIC Virtualization • Resource partitioning, QoS/Diffserv (without performance penalties) • SLA on a per connection basis • Better Defense against DDOS attacks • Real time usage and history • N2 performance > Polling on forwarding path (performance) > S/W fanout to multiple cores (utilization) • Class of service support • ISV support (APIs for configuration, statistics, traps, etc) • Network Device Consolidation Crossbow Features Crossbow Features
35. Technical Obstacles Technical Obstacles • Obstacles to achieving network virtualization: • Network processing in interrupt context • Anonymous packet processing in kernel • Common queues • Performance can be degraded by the extra processing to enforce fairness, resource control or network virtualization • No isolation for flows
36. • Divide NIC memory, DMA channels, etc and use a flow classifier to build a virtual stack on each H/W partition • Each Virtual NIC is owned by the FireEngine Squeue's which independently switch the VNIC between interrupt & polling mode • Rate of packet arrival from a VNIC is independently controlled by the Squeue owning the VNIC The Crossbow Architecture The Crossbow Architecture
37. Crossbow “Hardware Lanes” Crossbow “Hardware Lanes”
38. Crossbow “Traffic Lanes” Crossbow “Traffic Lanes” Classifier HW Classifier HW Classifier SW Classifier SW
39. • Sample mpstat output Mpstat (older driver) intr ithr csw icsw migr smtx srw syscl usr sys wt idl 10818 8607 4558 1547 161 1797 289 19112 17 69 0 12 Mpstat (GLDv3 based driver) intr ithr csw icsw migr smtx srw syscl usr sys wt idl 2823 1489 875 151 93 261 1 19825 15 57 0 27 • Notice the decrease in interrupts, context switches, mutex contentions, etc. and increase in idle time • Crossbow allows each VNIC to be dynamically polled by its Squeue Effects of Dynamic Polling Effects of Dynamic Polling
40. Virtual Network Containers Virtual Network Containers
41. Virtual Network – The Cloud Virtual Network – The Cloud
42. Solaris Networking Features Availability Solaris Networking Features Availability
43. Virtual Network – S etup Virtual Network – S etup
44. •Setting up a Virtual network •within a single box – automatically •Bourne Shell scripts suitable for setting up: •Demo - •Tests - •Jumpstart - Environments •Tested with OpenSolaris SXCE Build 130 • site.xml did not work to enable/disable Services • OpenSolaris some issues Virtual Network – Practical Example Virtual Network – Practical Example
45. -rwxrwxrwx 1 root root 2113 2010-05-25 11:22 cl_crossbow.sh -rwxrwxrwx 1 root root 142 2010-05-22 21:24 exec_cl_crossbow.sh -rwxr-xr-x 1 root root 194 2010-05-25 13:04 exec_nv_mk_crossbow.sh -rwxr-xr-x 1 root root 194 2010-05-25 13:04 exec_os_mk_crossbow.sh -rwxr-xr-x 1 root root 14120 2010-05-25 16:42 nv_mk_crossbow.sh -rwxrwxrwx 1 root root 12974 2010-05-25 12:06 os_mk_crossbow.sh -rw-r--r-- 1 root root 26816 2010-05-25 11:14 site.xml -rwxrwxrwx 1 root root 4120 2010-05-25 12:13 var.init Virtual Network – Practical Example Virtual Network – Practical Example List of the shell scripts
46. More Information More Information http://opensolaris.org/os/project/crossbow http://opensolaris.org/os/project/clearview http://opensolaris.org/os/community/networking http://hub.opensolaris.org/bin/download/Project+ clearview/ WebHome/clearview-overview.pdf http://www.c0t0d0s0.org/archives/5355- Upcoming-Solaris- Features-Crossbow-Part-1-Virtualisation.html
47. The Network is the The Network is the Computer Computer