Scale Your Services And Sleep At Night With LVS Nick Silkey | [email protected] Dept of Electrical and Computer Engineering The University of Texas at Austin 

LVS: Someone Say Virtual? Yes; but not in the x86 VMware/Xen/xVM sense Wikipedia says: advanced load balancing solution for Linux systems good scalability, reliability and serviceability build highly scalable and highly available network services, such as web service, email service, media service and VoIP services, and integrate scalable network services into large-scale reliable ... applications 

LVS: I Seen Things, Man Project has been around since roughly May 1998 Distros distributed patched kernels around 2003 ece.utexas.edu has been playing with it for a while; Went into production in 2003 serving www.ece Revamped implementation in 2006 New architecture more robust and serves more than just web presence via HTTP 

LVS: LVS 101 Give clients one place to go for things Let software manage how to route requests for things, especially if youre too broke to afford an appliance (us!) Fool the client’s idea of the network Or The Network’s idea of ‘the network’ Concepts of ‘director’ and ‘node’ Abstract away from things via ‘virtual addresses’ 

LVS: Topologies Three architectures: LVS-NAT, LVS-DR, LVS-TUN Ordered in increased awesomeness / complexity NAT: Connection tracking with address translation DR: Direct response back to clients from nodes Thanks to loopback, alias and arp hackery TUN: Tunneling between points breaks geo-faults 

LVS: House That ECE Built 1 2002 Architecture 4 x Dell PowerEdge 350s Debian GNU/Linux (Woody or Potato; cant recall) One directs traffic, other three serve Intarwebs Employs NAT between markup/binaries and clients 

LVS: House That ECE Built 2 Built by a curious and gifted student employee Good at the time but there were limitations Director doing routing & connection tracking = SPOF Same director was network-bound due to NAT Highly customized environment, including kernel He fled to .th and took his skills with him 

LVS: House That ECE Built 3 2006 Architecture 2 x PowerEdge 750s directing traffic active/passive Management complexity eased with wrappers UltraMonkey v3.x from Horms (ultramonkey.org) Think Red Hat’s Piranha or the GPL’ed keepalived Directors heartbeat every 2 seconds via 694/udp 

LVS: House That ECE Built 4 ECE Web via HTTP 4 x PowerEdge 750s running RHEL4 Apache 2.0.x ; mod_python + Django Point back to lone PostgreSQL 7.4.x instance Read-only NFS export for some markup and binaries Go up and down for testing/patching, no service interruptions to users 

LVS: House That ECE Built 5 Segment userland content from departmental content ~/public_html served from same systems earlier Problem with mod_auth_eid ; unauthorized cookie decrypting Throw more nodes at the problem & segment/redirect Newish servers in a new pool to handle user taint 

LVS: House That ECE Built 6 Lets do this for ECE mail too! Old mail was a nightmare 1 x PowerEdge 2550 ; Debian GNU/Linux (Woody) Exim 3.x & UW-IMAP ; mbox via NFS No SSL|TLS, No SMTP AUTH (POP-Before-SMTP) It had ... issues 

LVS: House That ECE Built 7 Segment inbound and outbound mail Outbound mail (Postfix 2.x) + MUA endpoint (dovecot) 2 x PowerEdge 2850s Active / passive (thanks to mbox via NFS) Can scale-up once we go mb2mdir Either way, no SPOF; can failover without impacting 

LVS: House That ECE Built 8 Inbound mail cleanup Handoff from edge IronPorts or advertised MXes 2 x PowerEdge 750s running Postfix 2.x Scheduling active/passive thanks to freakin mbox Can scale up the x86 beef if needed without notifying LVS-DR scales to n number of interface capacities 

LVS: Future of LVS in ECE Get some real DR/BCP up in this LVS-DR has limitations via ‘same subnet’ concept Cannot span VLANs on UTNet, need to solve via appliance-in-the-middle or via LVS-TUN Scale LVS out to databases MySQL slave repls with LVS in-front PostgreSQL too 

LVS: Linux Directs Traffic The nodes behind LVS can be any operating system ... even Windows Homebrewed non-persistent VDI/VDM solution IIS / MSSQL / Whatever In our case of UltraMonkey, it supports lots out of box: HTTP(S), FTP, SMTP, POP, IMAP, LDAP, NNTP, UDP DNS, MySQL, PostgreSQL, fwmark, ping, RADIUS, SIP 

LVS: You Hate Linux? Different topic, but ... FreeBSD has an IPVS port supports LVS-DR and LVS-TUN too Sparc Solaris is unknown ... still! Youll likely test LVS against a black-box appliance Cisco, F5, Foundry, Juniper, Nortel 

LVS: Fin Questions? Nick Silkey Sr Oper Sys Spec for ECE-IT [email protected]