Slide 1

Slide 1 text

Using FreeBSD to Build a Resilient Container (Jails) Infrastructure Md. Zubair Sharief Twitter: @mzs114

Slide 2

Slide 2 text

About Me ● Blog: kgibran.wordpress.com ● Current: Designing, managing infrastructure for a client of Imaginea/Pramati ● When possible: Developing a web app for BSD Jails, read about security, history, employ strategies during Wesnoth ● Twitter: @mzs114 (used occasionally)

Slide 3

Slide 3 text

My Challenges ● Web services used across Indian edu institutions, powered by OpenVZ, on a mixture of bare metal and AWS instances. ● Software affected by vulnerabilities – MediaWiki, WordPress results in spam, downtime. ● Backups using rsnapshot, rsync are slow to recover. ● Recovery/Rollback to a previous working copy takes time – data loss, frustrates people, loss of business.

Slide 4

Slide 4 text

FreeBSD, Jails, ZFS - Using the Right Tool ● One among the oldest Unices ● Support for binary packages with pkgng ● Wide choice of packet filters, my choice - pf ● Jails – first among containers, ~15 years old, mature ● ZFS support – enterprise file system for everyone

Slide 5

Slide 5 text

ZFS Goodness ● ZFS Snapshots reduce downtime – Apache foundation: https://blogs.apache.org/infra/entry/apache_org_downtime_re port ● Multinode copies of data (zfs send), multi disk redundancy using mirrors, all with two simple command line utilities - zpool, zfs ● Rolling back a container (jail) or a zfs dataset(could be data directory for applications) using zfs rollback ● DB backup? Lock DB, flush data to disk and take snapshot! ● Do we need backup software?

Slide 6

Slide 6 text

Jailing the daemons ● Lean on storage (~4MB each without applications), using nullfs ● Patching a single jail patches all other jails when mounted using nullfs ● Granular control using jail parameters and kernel variables for jails (sysctl -a | grep jail) ● Harden jails and host using securelevels, makes kernel & firewall settings immutable, compromised service/software cannot wreak havoc ● Wrappers exist to make jail management simple – ezjail, CBSD ● CBSD – supports spanning multiple jails nodes, replication and failover simplified

Slide 7

Slide 7 text

Current Use at Pramati – HA Squid Previous Setup ● Single host running on GNU/Linux Issues ● Internet access for ~700 users ● Wait for on call IT to fix any issues ● Initial plan for HA required an LB and two Squid nodes – raised the cost of implementation

Slide 8

Slide 8 text

Current Use at Pramati – HA Squid ● HA forward proxy on different nodes ● Used CARP on aliased interfaces with services inside Jails ● The failover is almost instant with few seconds of interruption ● Updating and moving services is simple and insured - create a fallback clone of the jail, rollback if the update fails. ● Remote webservices (gmail, etc) not affected – same public IP address

Slide 9

Slide 9 text

In the Future ● OpenVPN server ● Storage server ● Asset, Inventory management service

Slide 10

Slide 10 text

Thank You.