Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Automated Configuration of Firmware
Puppet Labs
September 27, 2012
Technology
1
160
Automated Configuration of Firmware
Puppet Labs
September 27, 2012
Tweet
Share
More Decks by Puppet Labs
See All by Puppet Labs
puppetlabs
13
750
puppetlabs
1
340
puppetlabs
4
930
puppetlabs
1
230
puppetlabs
2
200
puppetlabs
2
270
Other Decks in Technology
See All in Technology
mizzy
2
140
sadayoshitada0919
0
340
iwashi
1
200
dena_tech
0
150
yasuyukiyamasaki
1
170
dena_tech
1
130
ryomasumura
0
110
koukyo1994
5
1.3k
kanaugust
PRO
0
160
mahito
0
220
chandi
0
140
htomine
0
160
Featured
See All Featured
nonsquared
81
3.4k
hannesfritz
28
950
cherdarchuk
71
260k
62gerente
587
200k
myddelton
109
11k
pedronauck
652
110k
kneath
219
15k
sstephenson
145
12k
mojombo
359
62k
zakiwarfel
88
3.4k
swwweet
206
6.9k
jmmastey
10
620
Transcript
APOLLO GROUP Automated Configuration of Firmware Michael Arnold Principal Systems
Engineer 27 September 2012
2 APOLLO GROUP Who Tools Example Workflow Example Code Links
Questions Agenda © 2012 Apollo Group
3 APOLLO GROUP APOLLO GROUP Who © 2012 Apollo Group
4 APOLLO GROUP Who is Apollo? © 2012 Apollo Group
Apollo Group is a leading provider of higher education programs for working adults.
5 APOLLO GROUP Systems Administrator Automation geek 13 years in
IT Hadoop Cluster Admin I deal with: –Server hardware specification/configuration –Server firmware –Server operating system –Hadoop Cluster performance and availability Who is Michael Arnold? © 2012 Apollo Group
6 APOLLO GROUP It has nothing to do with Puppet.
–Unless someone writes a module to program firmware from the running OS. • Which could work for RAID controllers. How does firmware tie in with Puppet? © 2012 Apollo Group
7 APOLLO GROUP APOLLO GROUP Tools © 2012 Apollo Group
8 APOLLO GROUP HP SmartStart Scripting Toolkit Dell OpenManage Deployment
Toolkit IBM ServerGuide Scripting Toolkit Fujitsu ServerView Scripting Toolkit Hardware Toolkits © 2012 Apollo Group
9 APOLLO GROUP Configuration of –BIOS –BMC (DRAC/iLO/RSA/etc.) –Hardware RAID
–FC HBA What do the tools provide? © 2012 Apollo Group
10 APOLLO GROUP Manually configure reference hardware Capture the firmware
configuration Clone the configuration to multiple machines How does the toolkit work? © 2012 Apollo Group
11 APOLLO GROUP Toolkit Components © 2012 Apollo Group Kernel
& Initrd Firmware Tools Toolkit
12 APOLLO GROUP Toolkit Requirements © 2012 Apollo Group Toolkit
DHCP TFTP NFS
13 APOLLO GROUP APOLLO GROUP Example Workflow © 2012 Apollo
Group
14 APOLLO GROUP Hardware is racked and cabled Switch ports
are up Server is manually powered on Firmware defaults are to boot from network if no other boot device is found. –No other boot devices are found Assumptions © 2012 Apollo Group
15 APOLLO GROUP All server hardware is from the same
vendor (HP) Constraints © 2012 Apollo Group
16 APOLLO GROUP Firmware Update Hardware Discovery Firmware Configuration Buildsystem
Registration The Process © 2012 Apollo Group
17 APOLLO GROUP Screencast © 2012 Apollo Group
18 APOLLO GROUP Unknown system netboots Workflow © 2012 Apollo
Group
19 APOLLO GROUP Unknown system netboots Boot into HP SmartStart
Scripting Toolkit Workflow © 2012 Apollo Group
20 APOLLO GROUP Unknown system netboots Boot into HP SmartStart
Scripting Toolkit Gather hardware information (CPU/RAM/disk/NIC) via hpdiscovery Workflow © 2012 Apollo Group
21 APOLLO GROUP Unknown system netboots Boot into HP SmartStart
Scripting Toolkit Gather hardware information (CPU/RAM/disk/NIC) via hpdiscovery If this is HP hardware, then –Update firmware –Configure BIOS –Configure iLO –Configure RAID Workflow © 2012 Apollo Group
22 APOLLO GROUP Unknown system netboots Boot into HP SmartStart
Scripting Toolkit Gather hardware information (CPU/RAM/disk/NIC) via hpdiscovery If this is HP hardware, then –Update firmware –Configure BIOS –Configure iLO –Configure RAID cobbler-register to the hardware profile Workflow © 2012 Apollo Group
23 APOLLO GROUP Unknown system netboots Boot into HP SmartStart
Scripting Toolkit Gather hardware information (CPU/RAM/disk/NIC) via hpdiscovery If this is HP hardware, then –Update firmware –Configure BIOS –Configure iLO –Configure RAID cobbler-register to the hardware profile System reboots Workflow © 2012 Apollo Group
24 APOLLO GROUP Modify Cobbler system profile, giving it an
identity (hostname, IP, OS metadata) Workflow © 2012 Apollo Group
25 APOLLO GROUP Modify Cobbler system profile, giving it an
identity (hostname, IP, OS metadata) Known system netboots Workflow © 2012 Apollo Group
26 APOLLO GROUP Modify Cobbler system profile, giving it an
identity (hostname, IP, OS metadata) Known system netboots Boot into OS installer Workflow © 2012 Apollo Group
27 APOLLO GROUP Modify Cobbler system profile, giving it an
identity (hostname, IP, OS metadata) Known system netboots Boot into OS installer Install OS Workflow © 2012 Apollo Group
28 APOLLO GROUP Modify Cobbler system profile, giving it an
identity (hostname, IP, OS metadata) Known system netboots Boot into OS installer Install OS Install Puppet Workflow © 2012 Apollo Group
29 APOLLO GROUP Modify Cobbler system profile, giving it an
identity (hostname, IP, OS metadata) Known system netboots Boot into OS installer Install OS Install Puppet … Workflow © 2012 Apollo Group
30 APOLLO GROUP Modify Cobbler system profile, giving it an
identity (hostname, IP, OS metadata) Known system netboots Boot into OS installer Install OS Install Puppet … Profit! Workflow © 2012 Apollo Group
31 APOLLO GROUP APOLLO GROUP Example Code © 2012 Apollo
Group
32 APOLLO GROUP LABEL dtk-install_hadoop kernel /images/dtk-3.5.1/SA.1 append initrd=/images/dtk-3.5.1/SA.2 ramdisk_size=156482
Stage3_type=cdrom quiet DEBUG=0 share_location=192.168.1.1:/srv/nfs/ro/dtk-3.5.1 share_type=nfs selinux=0 share_opts=ro,nolock share_script=install_hadoop.sh cprofile=Hadoop ipappend 2 Dell TFTP Config © 2012 Apollo Group
33 APOLLO GROUP LABEL sstk-install_hadoop kernel /images/sstk-8.50/vmlinuz append initrd=/images/sstk-8.50/initrd.img root=/dev/ram0
rw ramdisk_size=184248 ide=nodma ide=noraid pnpbios=off network=1 sstk_mount=192.168.1.1:/srv/nfs/ro/sstk-8.50 sstk_mount_type=nfs sstk_mount_options=ro,nolock sstk_script=install_hadoop.sh cprofile=Hadoop ipappend 2 HP TFTP Config © 2012 Apollo Group
34 APOLLO GROUP cd /opt/dell/toolkit/template/scripts /opt/dell/toolkit/template/scripts/syscap.sh /tmp/syscap /opt/dell/toolkit/template/scripts/raidcap.sh /tmp/raidcap /opt/dell/toolkit/template/scripts/raccap.sh
/tmp/raccap # MegaCLI does not come with the DTK so "install" it. cp -a /opt/dell/toolkit/systems/MegaCLI/opt/MegaRAID \ /opt/MegaRAID /opt/MegaRAID/MegaCli/MegaCli -CfgSave -a0 -f /tmp/mega.0 Dell Config Capture © 2012 Apollo Group
35 APOLLO GROUP cd /TOOLKIT ./conrep -s -f /tmp/conrep.dat #
BIOS ./hpacuscripting -c /tmp/cpqacuxe.dat # RAID ./hplpcfg /s /tmp/hplpcfg.dat # Emulex HBA ./hpqlarep /S /tmp/hpqlarep.dat # Qlogic HBA if ./ifhw /tmp/hw_discovery.dat allboards.xml \ "PCI:Integrated Lights-Out" 2> /dev/null ; then ./hponcfg -w /tmp/hponcfg.dat # iLO 2/3 else ./lo100cfg -o /tmp/lo100cfg.dat # LO100i fi HP Config Capture © 2012 Apollo Group
36 APOLLO GROUP *** Detecting system type *** System Information:
Name : ProLiant DL360 G5 Serial : USE123D456 Arch : x86_64 LNAME : ProLiantDL360G5 SNAME : DL360 G5 Asset Tag : 01234567 Platform Detection © 2012 Apollo Group
37 APOLLO GROUP HP SSSTK has a function library
that sets many variables. –$PRODUCT_NAME = •"ProLiant SL170z G6" •"ProLiant DL360 G6" –$SERIAL_ID = hardware serial number –$PRODUCT_LNAME = •"ProLiantSL170zG6" •"ProLiantDL360G6" –$PRODUCT_SNAME = •"SL170z G6" •"DL360 G6" –$TOOLKIT_MNTPNT = /mnt/main –$TOOLKIT_SERVER_IP = NFS server IP –$HWDISC_FILE = /TOOLKIT/hpdiscovery.xml Some Available Variables © 2012 Apollo Group
38 APOLLO GROUP Set Variables Based on Hardware Type export
"`./hwquery ${HWDISC_FILE} allboards.xml SERVERNAME=SystemName`" case "${SERVERNAME}" in "ProLiant DL160 G6"* ) CONREPDAT=${TOOLKIT_MNTPNT}/data_files/dl160_conrep.dat LO100CFGDAT=${TOOLKIT_MNTPNT}/data_files/dl160_lo100cfg.dat ;; "ProLiant SL170z G6"* ) CONREPDAT="${TOOLKIT_MNTPNT}/data_files/sl170z_conrep.dat -x \ ${TOOLKIT_MNTPNT}/conrep_xml/conrep_SL170zg6_20100401.xml" LO100CFGDAT=${TOOLKIT_MNTPNT}/data_files/sl170z_lo100cfg.dat ;; #... Inside install_hadoop.sh © 2012 Apollo Group
39 APOLLO GROUP #... "ProLiant DL360 G5"* ) CONREPDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_conrep.dat if
./ifhw ${HWDISC_FILE} allboards.xml "PCI:Smart Array E200i Controller" \ 2> /dev/null ; then HPACUDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_saE200i_cpqacuxe.dat fi if ./ifhw ${HWDISC_FILE} allboards.xml "PCI:Smart Array P400i Controller" \ 2> /dev/null ; then HPACUDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_saP400i_cpqacuxe.dat fi HPONCFGDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_hponcfg.dat ;; esac Inside install_hadoop.sh © 2012 Apollo Group
40 APOLLO GROUP if [ -n "$CONREPDAT" ] ; then
echo "** Applying BIOS Configuration" ./conrep -l -f ${CONREPDAT} fi if [ -n "$HPACUDAT" ] ; then echo "** Clearing RAID Configuration" ./hpacucli/hpacuscripting -i ${HPACUDAT}_clear; sleep 5 echo "** Applying RAID Configuration" ./hpacucli/hpacuscripting -i ${HPACUDAT} fi if [ -n "$LO100CFGDAT" ] ; then echo "** Applying BMC Configuration"; ./lo100cfg -i ${LO100CFGDAT} elif [ -n "$HPONCFGDAT" ] ; then echo "** Applying iLO Configuration"; ./hponcfg -f ${HPONCFGDAT} fi Inside install_hadoop.sh © 2012 Apollo Group
41 APOLLO GROUP echo "** Registering with cobbler" # Koan
does not come with SSSTK, so "install" it. ${TOOLKIT_MNTPNT}/koan/install # cprofile is set via kernel boot parameter and points at part of a # Cobbler profile name. # SERIAL_ID and PRODUCT_LNAME are set via SSSTK. # PRODUCT_LNAME = ProLiantSL170zG6, ProLiantDL360G6, etc. # SERIAL_ID = hardware serial number echo "** Registering to profile ${cprofile}-${PRODUCT_LNAME}" cobbler-register --server=${TOOLKIT_SERVER_IP} --fqdn=${SERIAL_ID} \ --profile=${cprofile}-${PRODUCT_LNAME} --batch Register with Buildsystem © 2012 Apollo Group
42 APOLLO GROUP $ cat dl360g5_saE200i_cpqacuxe.dat_clear ; Version: 8.50.6.0 ;
http://h30499.www3.hp.com/t5/ProLiant-Servers-ML-DL- SL/CPQACUXE-reset-utility-within-WinPE/td-p/3939475 Action= Reconfigure Method= Custom Controller= SLOT 0 ClearConfigurationWithDataLoss= Yes Clear the HP RAID Controller © 2012 Apollo Group
43 APOLLO GROUP $ cat dl360g5_saE200i_cpqacuxe.dat ; Version: 8.50.6.0 Action=
Configure Method= Custom ; Controller Specifications ; Controller HP Smart Array E200i ; Firmware Version 1.82 Controller= SLOT 0 ReadCache= 100 WriteCache= 0 RebuildPriority= Medium ExpandPriority= Medium SurfaceScanDelay= 15 DriveWriteCache= Disabled ; Array Specifications Array= A ; Array Drive Type is SAS ; 1I:1:1 (72 GB), 1I:1:2 (72 GB) Drive= 1I:1:1, 1I:1:2 OnlineSpare= No ; Logical Drive Specifications LogicalDrive= 1 RAID= 1 Size= 69973 Sectors= 32 StripeSize= 128 ArrayAccelerator= Enabled Configure the HP RAID Controller © 2012 Apollo Group
44 APOLLO GROUP You may have to test for any
failures when applying configurations. When reusing HP hardware, adding a user to the BMC that already exists will cause a failure to apply all of the configuration. Newer firmware can have different config options that will cause an older capture to fail. Error Handling © 2012 Apollo Group
45 APOLLO GROUP Firmware Update Hardware Discovery Firmware Configuration Buildsystem
Registration Summary © 2012 Apollo Group
46 APOLLO GROUP 46 APOLLO GROUP HP SmartStart Scripting
Toolkit http://h18013.www1.hp.com/products/servers/management/toolkit/index.html Dell OpenManage Deployment Toolkit http://en.community.dell.com/techcenter/systems-management/w/wiki/1772.dell- openmanage-deployment-toolkit.aspx IBM ServerGuide Scripting Toolkit http://www-03.ibm.com/systems/be/management/sgstk/ Fujitsu ServerView Scripting Toolkit http://www.fujitsu.com/fts/products/computing/servers/primergy/management/de ploy/ Cobbler - Linux Installation Server http://cobbler.github.com/ LSI MegaRAID MegaCLI http://www.lsi.com/downloads/Public/MegaRAID%20Common%20Files/8.02.24_ MegaCLI.zip Links © 2012 Apollo Group
47 APOLLO GROUP 47 APOLLO GROUP https://intelligentsysadmin.wordpress.com/ https://github.com/razorsedge
http://forge.puppetlabs.com/razorsedge Contact © 2012 Apollo Group
48 APOLLO GROUP APOLLO GROUP Questions? © 2012 Apollo Group