Chef ‣ And Hugs ‣ And Beer ‣ And CrossFit #opslife #opscode #opschef #hugops #beerops #crossfit Tuesday, August 20, 13 For those that don't know, this is a GitHub Identicon. I don't know what it means, maybe it's a J and a T. Or, maybe it's a hash tag that fell down.
Tuesday, August 20, 13 I could just talk about zones, but then, you could just go read the documentation, and my blog post about them. Instead, let's start off with my story of how I got here.
we had customers that wanted to run that. We had Solaris 2.5.1 through 8 across various customer environments, sometimes in the same customer environment.
worked with a lot of these. Mostly, running Solaris 8. We had other hardware too. There was no virtualization anywhere here, except on the few E10k and 15k systems.
the nonsense of Solaris's archaic ways. I was actually hired at IBM for my knowledge of Linux, and I moved over to the Linux team in our group. I worked here for a couple years, before finally leaving IBM and going to work for the SANS Institute as a Linux administrator.
at a consulting company that automated startup infrastructures and was building a new product. This is the company that became Opscode. The product was Chef :-).
companies that were early adopters of EC2. At the time, EC2 instances were only Linux. They're virtual machines, running on top of Xen with a great API and low operational expense. This isn't a talk about EC2 though. We also worked with companies that were doing Xen/KVM based virtualization.
it allows you to get more out of hardware resources. Supposedly? Consolidation is where it's at, I guess. Of course, mainframe people will tell you they've been virtualizing since the 70's.
complexity ‣ Hardware abstraction ‣ Resource intensive ‣ Image management ‣ Plethora of technologies Tuesday, August 20, 13 The hypervisor in a VM environment provides a full hardware abstraction. This means you have to have enough memory, CPU, disk space per VM you wish to run, making it resource intensive. Then there's the problem of image management. Finally, there's a plethora of technologies, VMware, KVM, LPARs, Domains, depending on the platform(s) you're using. But this isn't a talk about virtualization, per se...
virtualization. There exists lighter weight alternatives, in "container" technologies. Such as LXC/Cgroups on Linux Cue question, "how are zones better than lxc" - let's talk at The Bar :)
of technologies Unix/Linux: Good ol' "chroot" BSD: Jails Linux: LXC/cgroup, OpenVZ Solaris/illumos: Zones (aka Containers) Tuesday, August 20, 13 By kernel-level virtualization, we get performance benefits. Launching containers is extremely fast, and they're lightweight. Generally because there's no hardware abstraction, they're more simple that VM technologies There's a plethora of technologies, usually OS-specific: jails, openvz/ lxc/cgroups, and finally, solaris zones (containers)
This is a brief history of Solaris, OpenSolaris, and illumos. For the best background, listen to this talk from Bryan Cantrill: http://smartos.org/2011/12/15/fork-yeah-the-rise-and- development-of-illumos-2/
really just a story about me... Tuesday, August 20, 13 I installed and used OpenSolaris for ohai, chef resources/provider testing. Then later on I went back looking for it, and ... wat?
Tuesday, August 20, 13 Except, when Oracle bought Sun, they silently killed the OpenSolaris project. Seriously go listen to Bryan's talk. It's a great lesson in open source project and community governance, and why it is vitally important to be a good steward to your community.
Solaris ‣ Joyent SmartOS Tuesday, August 20, 13 Along the way, Joyent released SmartOS. This is the hypervisor OS that Joyent uses to build their public cloud offering. It has particular hardware requirements, and until recently, wasn't easy to run in a VM. http://cuddletech.com/blog/?p=821
‣ Joyent SmartOS ‣ OmniTI OmniOS Tuesday, August 20, 13 In April last year, OmniTI announced OmniOS, their illumos distribution. It's intended to be installed on real hardware. Well, maybe not the beloved E450 :).
http://illumos.org ‣ Includes all the goods: zones, zfs, smf, dtrace, crossbow ‣ Where all the innovation for technology from Solaris is happening Tuesday, August 20, 13 Again, watch Bryan's talk if you want to know the background on all this.
others ‣ OmniOS is a server-focused minimal installation ‣ OmniOS uses IPS, supports SVR4 ‣ OmniTI provides an OmniOS Vagrant box Tuesday, August 20, 13 OmniOS appeals to me for the "stable base platform" aspect of the minimal installation. I like that it strives for compatibility with older Solaris platforms, such as supporting the SVR4 package system. I also love that they make a Vagrant box, which means getting started is a Vagrantfile + "vagrant up"
technologies zfs, dtrace, crossbow ‣ Can't break out of a zone* There are other container technologies, why zones? * at least, I haven't found a reference to it being possible Tuesday, August 20, 13 If you know of research, blog posts, papers, or anything that proves that one can break out of a non-global zone into the global zone, I'd love to hear it. I've heard that it is possible to break out of KVM, Xen, LXC, Jails, but I also don't have references handy. Please email me, [email protected] if you have any.
2.3GHz CPU 4G memory OS disk (128G) Data disk (500G) 2x GigE NICs Tuesday, August 20, 13 This is just some baseline information about the hardware I'm running all this on. It's useful to note the disks and the NICs, and that this is a pretty "minimally spec'ed" machine (in comparison to what you can get for the money now - this computer is 6 years old now)
zpool - configure storage pools ‣ dladm - administer data links (network interfaces) ‣ zonecfg - set up zone configuration ‣ zoneadm - adminster zones Tuesday, August 20, 13 These are commands that we'll be using, they're all specific to Solaris/ illumos. Well, except format, but hey :). The man pages are really good, and contain everything you'll need to know about the sub-commands and options that I use. Also, all the documentation from Solaris 10 release era, 2005, is still relevant and totally works, available from Oracle's site.
DISK SELECTIONS: 0. c3t0d0 <ATA-WDCWD1500AHFD-0-7QR5 cyl 18238 alt 2 hd 255 sec 63> /pci@0,0/pci1043,cb84@d/disk@0,0 1. c3t1d0 <ATA-SAMSUNG HD501LJ-0-12-465.76GB> /pci@0,0/pci1043,cb84@d/disk@1,0 Specify disk (enter its number): Tuesday, August 20, 13 The format command is used for partitioning disks. It is also about the only reasonable command available to list the actual device names of the disks in the system. It's an interactive command, unless you give it stdin. Is there a command I'm missing to find the cXtXdX devices instead of this?
with Solaris 10 ‣ Copy on write filesystem ‣ Lightweight snapshots ‣ Volume management built in ‣ Enterprise-grade storage - built for data reliability Tuesday, August 20, 13 This isn't a talk about ZFS, so here are some highlights about ZFS. Maybe I'll come back for one another time :).
zones c3t1d0 Tuesday, August 20, 13 It is a best practice to have a zpool set aside for zones. A zpool is a collection of disk devices on which you build ZFS filesystems.
AVAIL REFER MOUNTPOINT zones/base 877M 455G 33K /zones/base zones/fpm 129M 455G 34K /zones/fpm zones/nginx0 783M 455G 34K /zones/nginx0 Tuesday, August 20, 13 This is what I have running before I started making the slides, just to show an example. base, fpm, nginx0 are all zones I was playing with.
DEVICE nge0 Ethernet up 1000 full nge0 nge1 Ethernet up 1000 full nge1 Tuesday, August 20, 13 We need to know the physical device names of the network interfaces. I'm already using nge0 for the global zone.
August 20, 13 Create a new Virtual Network Interface for the zone, associated with a physical ethernet link, nge1. The name of the vnic must end in a number.
autoboot=false set ip-type=exclusive add net set physical=vnicdemo0 end Tuesday, August 20, 13 Create a blank config. zonepath is where the zone's filesystem is. Set the brand to use. Brands are operating environments for non-global zones. ipkg is the default zone brand on OmniOS. Whether the zone should be booted automatically at system (global zone) boot. Give the zone its own IP stack, not sharing with the global zone host. This allows it to be on a separate network, and IIRC was required for DHCP Add a new network resource. The physical attribute is the name of the network device created on the global zone (host). 'end' is the end of the 'add net' block
demo install Tuesday, August 20, 13 This will install the operating system packages into the new zone under the specified zonepath on our storage zpool, zones.
/zones/demo/root/etc Tuesday, August 20, 13 We want to have name resolution use DNS, so copy the nsswitch.dns file to the new zone, even though it's 2013 and we've used DNS since dinosaurs roamed the earth. Presuming that the /etc/resolv.conf on the global zone is the one we want to use, we copy that to the zone's filesystem, too. This is a huge advantage for zones over image-based VM, as we can actually drop off required configuration before we start the zone, with a image-based VM, we'd have to rebuild an entirely new image.
'demo' pts/2] Last login: Sun Aug 18 20:36:28 on pts/2 OmniOS 5.11 omnios-8d266aa 2013.05.04 root@demo:~# logout [Connection to zone 'demo' pts/2 closed] root@menthe:~# Tuesday, August 20, 13
-z chefbase -f chefbase.conf zoneadm -z chefbase install # copy nsswitch, resolv, ipadm to zone... zoneadm -z chefbase boot zlogin chefbase \ 'curl -L https://www.opscode.com/chef/install.sh | bash' zoneadm -z chefbase halt https://github.com/jtimberman/zone-scripts/blob/master/mkchefbase.sh Tuesday, August 20, 13 This is an excerpt from the script in the zone-scripts repository I created.
<<EOF chef_server_url "https://api.opscode.com/organizations/ORGNAME" validation_client_name "ORGNAME-validator" EOF zlogin newzone /opt/chef/bin/chef-client Tuesday, August 20, 13 We'd replace "newzone" and "orgname" with the appropriate values here, of course.
[2013-08-18T21:16:27+00:00] INFO: Forking chef instance to converge... [2013-08-18T21:16:27+00:00] INFO: *** Chef 11.6.0 *** [2013-08-18T21:16:29+00:00] INFO: Client key /etc/chef/client.pem is not present - registering [2013-08-18T21:16:30+00:00] INFO: Run List is [] [2013-08-18T21:16:30+00:00] INFO: Run List expands to [] [2013-08-18T21:16:30+00:00] INFO: Starting Chef Run for newzone [2013-08-18T21:16:30+00:00] INFO: Running start handlers [2013-08-18T21:16:30+00:00] INFO: Start handlers complete. [2013-08-18T21:16:31+00:00] INFO: Loading cookbooks [] [2013-08-18T21:16:31+00:00] WARN: Node newzone has an empty run list. [2013-08-18T21:16:31+00:00] INFO: Chef Run complete in 1.472431742 seconds [2013-08-18T21:16:31+00:00] INFO: Running report handlers [2013-08-18T21:16:31+00:00] INFO: Report handlers complete Tuesday, August 20, 13
so to speak It's starting to eat the application delivery world ‣ An application isn't just a package anymore It's an entire environment (look at Java init scripts) ‣ Automation isn't about just installing packages It's about integrating application environments together Tuesday, August 20, 13
jtimberman
mii3king
rssytems
nagisa53
scarletplover
chanyou0311
fnifni
yabakokobayashi
marchin1989
stknohg
asei
shimacos
mochizuki875
keithpitt
cromwellryan
sonatard
denniskardys
tammyeverts
addyosmani
eileencodes
kneath
honnibal
samanthasiow
dougneiner
