Deploying and maintaining your software with RPM/APT

Transcript

1. Deploying and maintaining software with RPM / APT Loadays - 16 & 17 April 2011 Antwerp - Belgium http://joind.in/3315 vrijdag 27 april 12
2. Who am I? vrijdag 27 april 12
3. Who am I? Joshua Thijssen (32) vrijdag 27 april 12
4. Who am I? Joshua Thijssen (32) Senior Software Engineer @ Enrise vrijdag 27 april 12
5. Who am I? Joshua Thijssen (32) Senior Software Engineer @ Enrise Development in PHP, Python, Perl, C, Java.... vrijdag 27 april 12
6. Who am I? Joshua Thijssen (32) Senior Software Engineer @ Enrise Development in PHP, Python, Perl, C, Java.... Blogs: http://www.adayinthelifeof.nl http://www.enrise.com/blog vrijdag 27 april 12
7. Who am I? Joshua Thijssen (32) Senior Software Engineer @ Enrise Development in PHP, Python, Perl, C, Java.... Blogs: http://www.adayinthelifeof.nl http://www.enrise.com/blog Email: joshua@enrise.com vrijdag 27 april 12
8. Who am I? Joshua Thijssen (32) Senior Software Engineer @ Enrise Development in PHP, Python, Perl, C, Java.... Blogs: http://www.adayinthelifeof.nl http://www.enrise.com/blog Email: joshua@enrise.com Twitter: @jaytaph Identi.ca: jaytaph vrijdag 27 april 12
9. What is this talk about? ‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD What are and why use package managers? About RPM About DEB Discussion vrijdag 27 april 12
10. Word of warning This talk is focussed on developers, not system administrators in general. vrijdag 27 april 12
11. Why use package managers? vrijdag 27 april 12
12. Why use a package manager (1) vrijdag 27 april 12
13. Why use a package manager (1) Takes care of dependencies vrijdag 27 april 12
14. Why use a package manager (1) Takes care of dependencies Less time to install vrijdag 27 april 12
15. Why use a package manager (1) Takes care of dependencies Less time to install No compilation needed vrijdag 27 april 12
16. Why use a package manager (1) Takes care of dependencies Less time to install No compilation needed Scrutinized by many users (stability) vrijdag 27 april 12
17. Why use a package manager (1) Takes care of dependencies Less time to install No compilation needed Scrutinized by many users (stability) Quick patches vrijdag 27 april 12
18. Why use a package manager (2) ‣ WHY NOT USE A PACKAGE MANAGER? vrijdag 27 april 12
19. Why use a package manager (2) Not processor specific (i386,i686,x64) ‣ WHY NOT USE A PACKAGE MANAGER? vrijdag 27 april 12
20. Why use a package manager (2) Not processor specific (i386,i686,x64) Missing feature X ‣ WHY NOT USE A PACKAGE MANAGER? vrijdag 27 april 12
21. Why use a package manager (2) Not processor specific (i386,i686,x64) Missing feature X Always behind latest versions ‣ WHY NOT USE A PACKAGE MANAGER? vrijdag 27 april 12
22. Why use a package manager (2) Not processor specific (i386,i686,x64) Missing feature X Always behind latest versions Installing newer versions can be a pain ‣ WHY NOT USE A PACKAGE MANAGER? vrijdag 27 april 12
23. Why use a package manager (2) Not processor specific (i386,i686,x64) Missing feature X Always behind latest versions Installing newer versions can be a pain Multiversioning (php5.2 + php5.3 ???) ‣ WHY NOT USE A PACKAGE MANAGER? vrijdag 27 april 12
24. Packageformats vs package frontends ‣ QUESTIONS? RAISE YOUR HAND OR YELL LOUD APT = Advanced Packaging Tool ‣ uses .DEB packages RPM = RedHat Package Manager ‣ uses .RPM packages vrijdag 27 april 12
25. The DEB vs RPM war vrijdag 27 april 12
26. .deb vs .rpm Different formats DEB for Debian based systems RPM for RedHat based system vrijdag 27 april 12
27. I like deb because... Maintaining custom packages is easier Asks what to do with your config (instead of rpm{save|new}) Creations are isolated (name.x.y dir) vrijdag 27 april 12
28. I hate deb because... Creation is hard (all separate files) Too many tools and possibilities (but that could be a pro as well) vrijdag 27 april 12
29. I like rpm because... Creating is easy as π (spec file) vrijdag 27 april 12
30. I hate rpm because... Everything in one directory. Spec file outside that directory. Macros can be confusing. vrijdag 27 april 12
31. .deb vs .rpm Use what your system uses. Use what you like when creating your own distribution. vrijdag 27 april 12
32. Creating an RPM package vrijdag 27 april 12
33. Anatomy of a .rpm package proprietary’ish format. Use rpm2cpio to extract. vrijdag 27 april 12
34. Anatomy of a .rpm package jthijssen@debian-jth:~/rpm$ rpm -qivlp --scripts enrise.rpm Name : enrise-repo Relocations: (not relocatable) Version : 1.2 Vendor: Enrise Release : 1 Build Date: Fri 21 Jan 2011 11:09:29 AM CET Install Date: (not installed) Build Host: rpmbuilder.enrise.com Group : Enrise Source RPM: enrise-repo-1.2-1.src.rpm Size : 1522 License: GPL Signature : RSA/SHA1, Fri 21 Jan 2011 11:09:31 AM CET, Key ID 84781eb0796900bd Packager : Joshua Thijssen Summary : Enrise Repository Configuration Description : This is the main configuration file (for /etc/yum.repos.d) to start using the repository. postinstall program: /bin/sh drwxr-xr-x 2 root root 0 Jan 21 11:09 /etc/pki drwxr-xr-x 2 root root 0 Jan 21 11:09 /etc/pki/rpm-gpg -rw-r--r-- 1 root root 967 Jan 21 11:09 /etc/pki/rpm-gpg/RPM-GPG-KEY- enrise drwxr-xr-x 2 root root 0 Jan 21 11:09 /etc/yum.repos.d -rw-r--r-- 1 root root 555 Jan 21 11:09 /etc/yum.repos.d/enrise.repo vrijdag 27 april 12
35. Building a package ‣ DON’T FORGET TO BRING... Your sources Your patches Your other stuff SPEC file vrijdag 27 april 12
36. Building a package Sources: myapp-v1.0.tar.gz Patches: myapp-v1.0-hotfix.patch Other files: myapp.init Spec: myapp.spec vrijdag 27 april 12
37. SPEC file Summary: Enrise Database Patcher Vendor: Enrise Name: patchdb Version: 3.0 Release: 1 Source0: patchdb-%{version}.tar.gz License: GPL Group: Enrise BuildArch: noarch BuildRoot: %{_tmppath}/%{name}-%{version}-buildroot %description This is the database patch script that allows you to quickly patch up a database to the correct version. vrijdag 27 april 12
38. SPEC file %prep %setup -q %build %install rm -rf $RPM_BUILD_ROOT mkdir -p $RPM_BUILD_ROOT/usr/local/enrise/bin mkdir -p $RPM_BUILD_ROOT/usr/share/man/man1 cp bin/patchdb $RPM_BUILD_ROOT/usr/local/enrise/bin cp man/patchdb.1 $RPM_BUILD_ROOT/usr/share/man/man1 %clean rm -rf $RPM_BUILD_ROOT vrijdag 27 april 12
39. SPEC file %pre %post %files %doc doc/* %dir /usr/local %dir /usr/local/enrise %dir /usr/local/enrise/bin /usr/share/man/man1/patchdb.1.gz /usr/local/enrise/bin/patchdb vrijdag 27 april 12
40. Creating your (S)RPM’s # rpmbuild -ba <specfile> vrijdag 27 april 12
41. Setting up a custom repository vrijdag 27 april 12
42. Creating a repository [me@intranet01 SPECS]# cat ~/.rpmmacros %packager Joshua Thijssen %_topdir /usr/src/enriserepo/rpm %_tmppath /usr/src/enriserepo/rpm/tmp %_signature gpg %_gpg_name Enrise Repository <repository@enrise.com> %_gpgbin /usr/bin/gpg vrijdag 27 april 12
43. Creating a repository #!/bin/sh # Export source-dirs from SVN,Git,etc into SOURCES # Create tar.gz files cd SOURCES for dir in $1* ; do if [ -d $dir ] ; then echo "Tarballing $dir..." tar czf $dir.tar.gz $dir fi done cd .. >>>> vrijdag 27 april 12
44. Creating a repository <<<< # Build RPMS's rpmbuild --quiet -ba SPECS/$1*.spec --sign 2>&1 | grep -v 'gpg: WARNING' # Add *ALL* files to the repository and re-create xml files REPOROOT=/data/www/enrise.com/repo.enrise.com/public/enrise /bin/cp SRPMS/* $REPOROOT/SRPMS/ /bin/cp RPMS/x86_64/* $REPOROOT/x86_64/ /bin/cp RPMS/noarch/* $REPOROOT/noarch/ createrepo -v $REPOROOT/x86_64 createrepo -v $REPOROOT/noarch createrepo -v $REPOROOT/SRPMS vrijdag 27 april 12
45. Creating a repository drwxr-sr-x 3 src apache 4096 Jan 21 11:13 . drwxr-sr-x 5 src apache 4096 Oct 5 2010 .. -rw-r--r-- 1 src apache 4219 Jan 21 11:12 enrise-repo-1.2-1.src.rpm -rw-r--r-- 1 src apache 7095 Jan 21 11:12 patchdb-3.0-1.src.rpm drwxr-sr-x 2 src apache 4096 Jan 21 11:13 repodata -rw-r--r-- 1 src apache 4323 Jan 21 11:12 varnishrotate-1.0-1.src.rpm -rw-r--r-- 1 src apache 3256951 Jan 21 11:12 zfm-1.1-2.src.rpm ./repodata: total 24 drwxr-sr-x 2 src apache 4096 Jan 21 11:13 . drwxr-sr-x 3 src apache 4096 Jan 21 11:13 .. -rw-r--r-- 1 src apache 658 Jan 21 11:13 filelists.xml.gz -rw-r--r-- 1 src apache 567 Jan 21 11:13 other.xml.gz -rw-r--r-- 1 src apache 1376 Jan 21 11:13 primary.xml.gz -rw-r--r-- 1 src apache 951 Jan 21 11:13 repomd.xml vrijdag 27 april 12
46. Using your repository http://moodleman.moodle.com.au/wp-content/uploads/2008/09/repository.jpg vrijdag 27 april 12
47. Using a repository Add repository to /etc/yum.repos.d Add key to /etc/pki/rpm-gpg yum! vrijdag 27 april 12
48. Using a repository Automate as well Add your /etc/yum.repos.d files + key files in package. “initial wget” rpm -hiv myrepo.rpm vrijdag 27 april 12
49. Creating and using debian’s .DEB http://moodleman.moodle.com.au/wp-content/uploads/2008/09/repository.jpg vrijdag 27 april 12
50. Anatomy of a .deb package jthijssen@debian-jth:~/deb$ ar tv apache2.2-common_2.2.9-10+lenny9_i386.deb rw-r--r-- 0/0 4 Dec 11 20:00 2010 debian-binary rw-r--r-- 0/0 13251 Dec 11 20:00 2010 control.tar.gz rw-r--r-- 0/0 770323 Dec 11 20:00 2010 data.tar.gz jthijssen@debian-jth:~/deb$ tar xvzf control.tar.gz ./ ./conffiles ./postinst ./preinst ./control ./postrm ./md5sums jthijssen@debian-jth:~/deb$ cat debian-binary 2.0 vrijdag 27 april 12
51. Anatomy of a .deb package jthijssen@debian-jth:~/deb$ cat control Package: apache2.2-common Source: apache2 Version: 2.2.9-10+lenny9 Architecture: i386 Maintainer: Debian Apache Maintainers <debian-apache@lists.debian.org> Installed-Size: 3416 Depends: libapr1, libaprutil1, libc6 (>= 2.7-1), libssl0.9.8 (>= 0.9.8f-5), zlib1g (>= 1:1.1.4), apache2-utils, net-tools, libmagic1, mime-support, lsb- base, procps, perl, psmisc Recommends: ssl-cert Suggests: www-browser, apache2-doc, apache2-suexec | apache2-suexec-custom ‣ CAT CONTROL vrijdag 27 april 12
52. Anatomy of a .deb package Conflicts: apache2-common, libapache2-mod-mime-xattr (<= 0.3-2), libapache2- mod-mono (<= 1.1.17-3), libapache2-mod-php4 (<= 4:4.4.4-2), libapache2-mod- php5 (<= 5.1.6-3), libapache2-mod-proxy-html (<= 2.4.3-2), libapache2-mod- scgi (<= 1.11-1), libapache2-mod-speedycgi (<= 2.22-3), libapache2-modxslt (<= 2005072700-1), libapache2-redirtoservername (<= 0.1-1), libapache2- webauth (<= 3.5.3-1), libapache2-webkdc (<= 3.5.3-1) Replaces: apache2-common Section: web Priority: optional Homepage: http://httpd.apache.org/ Description: Apache HTTP Server common files The Apache Software Foundation's goal is to build a secure, efficient and extensible HTTP server as standards-compliant open source software. The result has long been the number one web server on the Internet. ‣ CAT CONTROL (continued) vrijdag 27 april 12
53. Anatomy of a .deb package jthijssen@debian-jth:~/deb$ head conffiles /etc/logrotate.d/apache2 /etc/apache2/conf.d/security /etc/apache2/conf.d/charset /etc/apache2/envvars /etc/apache2/sites-available/default-ssl /etc/apache2/sites-available/default jthijssen@debian-jth:~/deb$ head postinst #!/bin/sh -e if [ "$1" != "configure" ]; then exit 0 fi if [ -e /usr/sbin/apache2 ]; then if [ "`dpkg-statoverride --list /usr/sbin/apache2`" = "" ]; then vrijdag 27 april 12
54. Creating a .deb package jthijssen@debian-jth:~/myapp-1.0$ dh_make -s -n -e jthijssen@enrise.com ‣ CREATE ./DEBIAN DIRECTORY TEMPLATE Directory: <appname>-<major>.<minor> vrijdag 27 april 12
55. Creating a .deb package Source: myapp Section: unknown Priority: extra Maintainer: jthijssen <jthijssen@enrise.com> Build-Depends: debhelper (>= 7) Standards-Version: 3.7.3 Homepage: <insert the upstream URL, if relevant> Package: myapp Architecture: any Depends: ${shlibs:Depends}, ${misc:Depends} Description: <insert up to 60 chars description> <insert long description, indented with spaces> ‣ EDIT ./DEBIAN/CONTROL ETC.. vrijdag 27 april 12
56. Creating a .deb package .TH myapp 1 "16 April 2011" "MyApp 1.00" .SH NAME myapp \- My Own Hello World Application. .SH SYNOPSIS myapp .SH DESCRIPTION Simple application that will print hello world to the standard output. .SH AUTHOR This manpage was written by Joshua Thijssen <jthijssen@enrise.com>. .\" end of man page ‣ EDIT ./DEBIAN/MYAPP.1 ETC.. myapp.1 myapp.manpages myapp.1 vrijdag 27 april 12
57. Creating a .deb package case "$1" in install|upgrade) touch /myapp.install ;; ‣ EDIT ./DEBIAN/{PREINST,PRERM} ETC.. vrijdag 27 april 12
58. Creating a .deb package usr/local/enrise/bin ‣ EDIT ./DEBIAN/DIRS vrijdag 27 april 12
59. Creating a .deb package jthijssen@debian-jth:~/myapp-1.0$ dpkg_buildpackage ‣ ACTUAL BUILDING OF THE PACKAGE jthijssen@debian-jth:~$ ls -la myapp* -rw-r--r-- 1 jthijssen jthijssen 2992 2011-04-11 15:50 myapp_1.0_all.deb -rw-r--r-- 1 jthijssen jthijssen 716 2011-04-11 15:50 myapp_1.0.dsc -rw-r--r-- 1 jthijssen jthijssen 1313 2011-04-11 15:50 myapp_1.0_i386.changes -rw-r--r-- 1 jthijssen jthijssen 4683 2011-04-11 15:50 myapp_1.0.tar.gz vrijdag 27 april 12
60. Creating a .deb package jthijssen@debian-jth:~/myapp-1.0$ lintian ../myapp_1.0_all.deb E: myapp: dir-in-usr-local usr/local/enrise/ E: myapp: dir-in-usr-local usr/local/enrise/bin/ E: myapp: file-in-usr-local usr/local/enrise/bin/myapp W: myapp: file-in-unusual-dir usr/local/enrise/bin/myapp W: myapp: readme-debian-contains-debmake-template W: myapp: copyright-lists-upstream-authors-with-dh_make-boilerplate ‣ LINTIAN CHECK vrijdag 27 april 12
61. Using your repository http://moodleman.moodle.com.au/wp-content/uploads/2008/09/repository.jpg vrijdag 27 april 12
62. Creating a repository jthijssen@deb:~$ mkdir -p /var/www/repo/enrise/{binary,source} jthijssen@deb:~$ cp *.deb /var/www/repo/enrise/binary jthijssen@deb:~$ cp *.dsc *.tar.gz /var/www/repo/enrise/source jthijssen@deb:~$ cd /var/www/repo/enrise jthijssen@deb:enrise$ dpkg-scanpackages binary /dev/null | gzip -9c > binary/Packages.gz jthijssen@deb:enrise$ dpkg-scansources source /dev/null | gzip -9c > source/Sources.gz vrijdag 27 april 12
63. Using your repository deb http://repo.url/enrise binary/ deb-src http://repo.url/enrise source/ /etc/apt/sources.list jthijssen@deb:~$ apt-get update jthijssen@deb:~$ apt-get install myapp vrijdag 27 april 12
64. Question & discussion http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg vrijdag 27 april 12
65. ‣ THANK YOU FOR YOUR ATTENTION Please rate my talk on joind.in: http://joind.in/3315 vrijdag 27 april 12