mikas blog

i386 systems are dying off and therefore building 32bit Debian packages in 64bit environments is important if you still need packages for i386. For some recent projects I had to set up 64bit-only environments with the need to provide 32bit packages for i386 as well as according i386 Linux kernel packages.

1) Common Debian packages

Just prepend the “linux32” command to the build commands and use the “-a386” option of dpkg-buildpackage(1). So use “linux32 dpkg-buildpackage -ai386”, “linux32 git-buildpackage -ai386”, “linux32 debuild -ai386”,… or whatever build tool you prefer to work with. I personally like using “git-buildpackage … -S” together with cowbuilder. When using cowbuilder it’s easy to get i386 Debian packages:

# create initial cow base directory, needs to be executed just once:
cowbuilder --create --basepath /var/cache/pbuilder/wheezy32.cow \
   --architecture i386 --distribution wheezy --mirror http://ftp.de.debian.org/debian

# the "debbuildopts -b" says I want a binary-only build, see dpkg-buildpackage(1):
linux32 cowbuilder --build --basepath /var/cache/pbuilder/wheezy32.cow \
   ~/foobar_0.42.dsc --debbuildopts "-b"

2) Debian Kernel packages

If you’re building your own kernel images you might be aware of setting ARCH for the common make targets, like:

ARCH=i386 make oldconfig

To build a linux-image Debian package you can use upstream’s deb-pkg target, like:

ARCH=i386 make deb-pkg

This should provide you according linux-headers, linux-firmware-image, linux-libc-dev and linux-image i386 Debian packages.

But if you need the according source and doc packages as well you might prefer make-kpkg of kernel-package instead, using options like:

DEB_HOST_ARCH=i386 setarch i386 make-kpkg --revision "$KERNELVERSION" \
  --cross-compile - --arch=i386 --us --uc --initrd \
  --rootcmd fakeroot kernel-image kernel-headers kernel-doc kernel-source

3) Debian linux-2.6 kernel packages

If you’re interested in building Debian kernel images the same way as the Debian kernel team does (AKA linux-2.6) make sure to check out the kernel-handbook.

After installing essential software packages I had to set up some symlinks for building:

apt-get install build-essential fakeroot devscripts fakeroot
apt-get build-dep linux-2.6
ln -s /usr/bin/gcc-4.4 /usr/bin/i486-linux-gnu-gcc-4.4
ln -s /usr/bin/ld /usr/bin/i486-linux-gnu-ld
ln -s /usr/bin/ar /usr/bin/i486-linux-gnu-ar
ln -s /usr/bin/objcopy /usr/bin/i486-linux-gnu-objcopy
ln -s /usr/bin/nm /usr/bin/i486-linux-gnu-nm
ln -s /usr/bin/objdump /usr/bin/i486-linux-gnu-objdump
ln -s /usr/bin/strip /usr/bin/i486-linux-gnu-strip

Then executing:

apt-get source linux-2.6
cd linux-2.6*
debuild -ai386 -us -uc

provided the according Debian packages (linux-doc, linux-headers, linux-image, linux-libc-dev, linux-manual, linux-patch-debian, linux-source, linux-support and linux-tools).

4) Out-of-tree kernel module packages

If you want to build external modules which are available as $MODULENAME-source in Debian (e.g. tp-smapi-source) using module-assistant then use the DEB_HOST_ARCH environment variable, like:

DEB_HOST_ARCH=i386 m-a -v --text-mode -k /home/mika/linux-$KERNELVERSION -l $KERNELVERSION build tp-smapi

If the external module is not available for use with module-assistant then compile the module running:

make -C /lib/modules/$KERNELVERSION/build M=$(pwd)

whereas /lib/modules/$KERNELVERSION/build is pointing to the build directory of your 32bit kernel.

Background:

Lars Wirzenius was searching for a tool to create virtual disk images. Turned out I needed something similar to improve the automatic deployment process of a customer’s platform installation.

Being the author of the mentioned grml-debootstrap tool I started to work on deploying virtual disk images using grml-debootstrap. As a result I just uploaded grml-debootstrap version 0.46 which is the first release being capable of installing virtual disk images. Thanks to great help by Thorsten Glaser also proper Grub integration is working now.

Instructions:

Grab the most recent grml-debootstrap version (>=0.46) from the grml-testing repository and make sure to install its recommends (kpartx mksh parted qemu-utils). Then just use the vmfile option to activate deployment of a virtual disk image, combined with the target option as usual. To adjust the disk size use the vmsize option.

Usage example:

# IMAGE_FILE="/mnt/sda1/qemu.img"
# grml-debootstrap --vmfile --vmsize 3G --target $IMAGE_FILE

I tested it from inside a running Grml live system, which is running inside a KVM image running on a Proxmox VE cluster. I can even invoke KVM/Qemu inside Grml inside KVM to test the image (using remote access e.g. via “vncviewer $IP_OF_KVM_HOST:5901”) running:

# kvm -hda $IMAGE_FILE -m 256 -vnc :1

And the best about it: it just works. :)
As usual: testing and feedback highly welcome.

I’m proud to be able to announce a new stable release of Grml, the Debian based Live system for system administrators.

This release is a very special one for me. On the one hand of course because of the special release name “Just Mari”, being dedicated to my lovely wife.

But it’s also special because of the way the release management worked out. I’ve been the release manager for Grml since the very beginning, which turned out to be more than 6 years since the first stable release already. I developed grml-live as build framework based on FAI for generating a Grml and Debian based Linux Live system to streamline the build process. Anyway it was mainly me who managed the release chroots, doing the update management during release freeze, editing main web page etc.

As I want to make the project as independent from myself as much as possible and to keep the Bus factor in balance we started to improve our project infrastructure so it’s not just me who can do this kind of release management. As a result Grml core developer Christian Hofstaedtler became the release sergeant of this release. The release candidate version was even released without myself being available (the release happened behind my back during our marriage). Major work on the final stable release was also done by Grml core developers Ulrich Dangel, Christian Hofstaedtler, Frank Terbeck and Alexander Wirt and I’m very happy about that. Kudos guys for all your work and all the hidden efforts going on behind my back. :)

The details regarding the new Grml release are available in the official release announcement, get the ISOs from grml.org/download.

I hope you enjoy the release as much as I do. Happy hacking!

4489 days later we did it, on 21st of May Marianne and I got married.

Turned out it became a pretty geeky marriage, as Grml 2011.05-rc1 with codename “Just Mari” was released during the ceremony (thanks guys!), I had to solve a nice groom quiz and our event had its own hashtag on twitter.

Thanks to everyone making the day unforgettable for us.

Just registered myself for DebConf11, see you in Banja Luka.

Last week a developer meeting of the FAI project took place in Cologne/Germany. Four core developers (project lead Thomas Lange and 3x Michael :)) met for two days to discuss and work on the FAI project.

The first day (2011-04-14) started at 11:45 and lasted until around 21:15, the second day (2011-04-15) started at 11:00 and lasted until 22:00. We made 134 svn commits in those two days. The developer meeting was great, we got tons of stuff done:

• New release candidate version for stable release 3.4.8
• Implemented support for CentOS 5
• Website improvements:
  • Put website into svn
  • Deployed FAQ and Team webpage
• Setup-storage improvements:
  • Backport from trunk/experimental of setup-storage into 3.4.8
  • Cleanup of setup-storage patches
  • Minor bugfixes
  • setup-storage in 3.4 got align-at and default to cylinder alignment if preserve is used
• Worked on autotesting using Kantan:
  • new kantan release with new features and better performane
  • prepared initial test-suite scenarios for usage inside Kantan
• Discussed Release management:
  • 3.4.x considered as long term stable release (includes tested bugfixes and tested features that have been backported from developer versions (4.x and experimental))
  • 4.x considered as developer version (inclusion of fast bugfixes, inclusion of big features which might break stable version,…)
• Documentation:
  • Officially documented release management notes for stable release
  • Officially documented developer policy (WIP)
• Unsorted other stuff:
  • Removed obsolete branches in svn
  • Grub now being installed from inside the target instead of nfsroot
  • Changelog in trunk got entries from 3.4.X with fixed git commit ids; entries were merged to find exact diff of trunk to 3.4.X
  • Defined what needs to be done after the developer meeting.

Nicht vergessen – am Samstag, dem 9. April finden an der FH Joanneum wieder die Grazer Linuxtage (GLT11) statt.

Das Programm bietet von 09:30 bis 19:00 Uhr über 30 Vorträge, LPI- und BSD-Prüfungen, eine Keysigning-Party und viele Infostände von renommierten Projekten wie z.B. Debian, LibreOffice und OpenStreetMap.

Ich selbst bin neben der Tätigkeit als Organisator auch am Grml-Projektstand sowie als Vortragender von Gute Open-Source-Projekte bestehen aus mehr als nur Code anzutreffen.

Das Programm sowie alle weiteren Informationen gibt es unter linuxtage.at.

The results of the Grml User Survey 2011 are available. I strongly recommend any open source project to run such a survey. Besides gathering really interesting feedback it’s motivating for developers to read what people think about your product.

Interesting facts for Planet Debian readers: With a leading 86% our users are Debian users. Further quoting:

Also just a short overview but it seems that the most important reasons for using Grml are:
* Based on Debian
* Command-line interface
* Ships all necessary command-line tools
* Zsh

I personally consider the survey a big success for the Grml project. Thanks to all participants for your feedback.

Am Samstag, dem 9. April 2011 finden an der FH Joanneum die Grazer Linuxtage 2011 (GLT11) statt. Die ganztägige Veranstaltung (9:00 bis 18:30 Uhr) richtet sich mit zahlreichen Vorträgen und Informationsständen nach den Interessen der Zielgruppen Anwender, Administratoren und Softwareentwickler.

Soeben sind Call for Lectures und Call for Projects online gegangen. Wer auf den GLT11 einen Vortrag halten oder ein Open-Source-Projekt mit einem Stand präsentieren möchte, kann das ab sofort über die Homepage einreichen.

Ich hoffe man sieht sich im schönen Graz!

Grml users out there: We are interested in getting your feedback so we can further improve Grml. We are also working on a webpage which lists happy Grml users and also some quotes and use cases. Now we need your help:

Please provide your feedback and take part in Grml’s user survey!

You might be aware of GRUB’s loopback option for booting an ISO, I wrote about it in Boot an ISO via Grub2 more than a year ago. A few months ago Goswin von Brederlow came up with this idea:

grml functions great as rescue system. So it would be nice to have a boot entry for it in grub instead of having to go look for the CD when needed. To make installing and updating simple it would be great if one could install this as a normal debian package which would register itself in grub (and maybe lilo too).

What a lovely idea: no need for a CD or USB stick as long as the bootloader and the harddisk are still working. Minimal manual intervention needed to keep it up2date and working – sounds like the perfect rescue system. 8-)

Now since end of 2010 a new stable version of sysadmin’s favorite live system (AKA Grml 2010.12) is available and the great news is that we came up with two independent solutions known as grml-rescueboot and grub-imageboot. Having shipped them to one of my customers already I’d like to write some words about it and why you should also consider using it on all your systems where GRUB2 is available.

grml-rescueboot uses GRUB2 and its loopback feature for booting. Grml as well as Ubuntu provide all what’s needed out-of-the-box. If you’re interested in the details check out Jordan Uggla’s excellent Loopback.cfg webpage in the supergrubdisk wiki. The best about it: you can provide custom bootoptions automatically when booting Grml. For example just set CUSTOM_BOOTOPTIONS=”ssh=grml2011″ in /etc/default/grml-rescueboot and the Grml rescue system will automatically start the OpenSSH server with the specified argument as password for user grml. Of course you can use all the other nifty bootoptions like scripts/netscript/… to further customize your Grml rescue system. [A note to other distributions like Ubuntu: I’d be interested to establish a mechanism to pass kernel options for loopback boot in a standardized way, drop me a note if you’re interested in this.]

grub-imageboot uses GRUB2 and syslinux’ memdisk to boot ISOs and floppy images. It doesn’t rely on the loopback feature but instead maps the ISO into the memory directly. This is great for Linux ISOs that can’t and won’t support the the loopback feature (and have everything inside their initrd so the ISO doesn’t have to find itself during booting), like BIOS or firmware updates. Also for example FreeDOS and Alpharev are known to work just fine. But sadly memdisk ISO emulation doesn’t work with all Linux systems, as documented in the syslinux wiki. The good news is that Grml supports the memdiskfind/phram/mtdblock approach out-of-the-box. As far as I know therefore Grml is the first Debian based live system supporting memdisk ISO boot, though my patches already went to the Debian-Live project so if you’re using live-boot >=2.0.14-1 or >=2.0.12-1+grml.04 your live system should be able to boot via memdisk ISO emulation as well.

Summary: If you want to boot a Linux system which supports loopback.cfg use grml-rescueboot as its a more powerful tool to boot Linux live systems like Grml and Ubuntu. If you want to boot non-Linux systems (BIOS-/Firmware-Updates/FreeDOS/….) use grub-imageboot.

Alright – how do you deploy this solution? Just grab and install grml-rescueboot and/or grub-imageboot.

To deploy grml-rescueboot (adjust $VERSION if you want to use another flavour):

# choose Grml version:
VERSION=grml64-medium_2010.12

# create directory
mkdir -p /boot/grml

# download and verify ISO
cd /boot/grml
wget download.grml.org/${VERSION}.iso{,.md5}
md5sum -c ${VERSION}.iso.md5

To deploy grub-imageboot:

# choose Grml version:
VERSION=grml64-medium_2010.12

# create directory
mkdir -p /boot/images

# download and verify ISO
cd /boot/images
wget download.grml.org/${VERSION}.iso{,.md5}
md5sum -c ${VERSION}.iso.md5

That’s it! Now when running update-grub you should get something like:

# update-grub
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.36-grml64
Found initrd image: /boot/initrd.img-2.6.36-grml64
Found memtest86+ image: /boot/memtest86+.bin
Found memtest86+ multiboot image: /boot/memtest86+_multiboot.bin
  No volume groups found
Found Grml ISO image: /boot/grml/grml64-medium_2010.12.iso
Found memdisk: /boot/memdisk
Found iso image: /boot/images/grml64-medium_2010.12.iso
done

Voilà, when rebooting your system you should see something like:

The “Grml Rescue System” entry is what grml-rescueboot provides and “Bootable ISO Image” is what’s provided by grub-imageboot. Just select the entry you’d like to use, press enter and and you should get the bootsplash of the according ISO.

BTW: I’ve tested this with Ubuntu 10.10 too, grml-rescueboot works out-of-the-box with the Ubuntu ISO as well, it just doesn’t support the memdisk ISO boot by grub-imageboot (yet).

Tip: if you want to use grml-rescueboot and grub-imageboot with the same ISOs without having them twice on the disk just point the configuration option IMAGES in /etc/default/grub-imageboot to the according directory (like /boot/grml).

Note: if you’re using the ext{2,3,4} filesystem on the partition that’s being used for /boot please be aware that you need GRUB >=1.98+20100804-12 (newer version is already available in Debian/unstable and hopefully migrates to squeeze in time) or GRUB >=1.99~20101122-1 because of the INDIRECT_BLOCKS support, see #543924 for the details.

PS: We think about providing grml-rescueboot and grub-imageboot within official Debian. If you’re interested in seeing integration support for other distributions as well please help and drop us a note.

Update: Ulrich ‘mru’ Dangel suggested pdnsd as nice alternative to dnsmasq and Cyril ‘KiBi’ Brulebois pointed out, that it’s not necessary to invoke dnsmasq after fresh installation as it’s running by default then – I adjusted the text accordingly, thanks for the pointers!

If /etc/resolv.conf doesn’t provide any nameserver entries glibc[1] will automatically initialize the nameserver to the loopback address. This is nice for dealing with chroots without having to manually edit resolv.conf to get working DNS.

Just install a basic DNS forwarder like dnsmasq (‘apt-get install dnsmasq’) or if it’s already on your box just start it (‘/etc/init.d/dnsmasq start’). That’s it. Now when chrooting into a system without an existing resolv.conf configuration it will give you a working DNS setup without any further work.

PS: Grml ships dnsmasq by default and ‘/etc/init.d/dnsmasq start’ or ‘Start dnsmasq’ will work out-of-the box. In Ubuntu the dnsmasq package is available only through the universe repository – but the dnsmasq-base package (providing e.g. the init script) is shipped by default, so replace the ‘/etc/init.d/dnsmasq’ command with a simple ‘dnsmasq’ there.

[1] At least the [e]glibc versions provided on Debian and Ubuntu are known to provide this glibc extension. Other libc implementations like dietlibc and uclibc don’t seem to provide it, so don’t strictly rely on this feature but use it as the icing on the cake.

Seit Ende September gibt es das Resultat von rund 3 Jahren Arbeit in Buchform im Handel. "Open Source Projektmanagement – Softwareentwicklung von der Idee zur Marktreife" ist mein literarischer Beitrag rund um das Management von Open-Source-Projekten. Dass das Buch so lange auf sich warten hat lassen, liegt – entgegen anders lautender Gerüchte – nicht an der schönen Seitenzahl 2⁸-1. Die Seitenzahl war natürlich reiner Zufall. Mir war es aber durchaus wichtig, dass das Buch einen leicht zu bewältigenden Umfang hat und nicht im Buchregal verstaubt.

Mein Buch ist ein Rundumschlag um alle möglichen Aspekte des Projektmanagements im Open-Source-Bereich – wie etwa Marketing, Kommunikation, Entwicklung und Infrastruktur. Darüber hinaus bietet dieses Buch Hilfestellung und Hinweise was es bei einem Open-Source-Projekt alles zu beachten gilt und wie man das in die Praxis umsetzt.

Warum dieses Buch? Seit mehr als 7 Jahren bin ich Projektleiter von Grml und darüber hinaus in vielen anderen Projekten wie u.a. Debian und FAI als Entwickler aktiv, seit mehr als 2 Jahren lebe ich ausschließlich von freier Software. Ich bin ein überzeugter Anhänger des Open-Source-Gedankens, stelle aber leider zu häufig fest, dass viel-versprechende Projekte beim Projektmanagement teilweise versagen. In meinem Buch berichte ich über diese, auch von mir begangene Fehler, warne vor bestimmten Praktiken und stelle Möglichkeiten zur Verbesserung des Entwicklungs- und Releaseprozesses vor.

Natürlich berichte ich nicht nur aus der Theorie oder meinen persönlichen Erfahrungen im Grml-Projekt, sondern lasse auch andere Personen zu Wort kommen: Alexandra Leisse (Nokia, Qt) erzählt vom Community Management bei Nokia, Lenz Grimmer (Oracle, MySQL) von der Arbeit in der virtuellen Firma MySQL. Wie das Releasemanagement in großen Projekten funktioniert erläutern Andreas Barth (Debian) und Michael Renner (PostgreSQL). Ferdinand Thommes (Sidux) schließlich erzählt von seinen Erfahrungen im Vereinswesen eines Open-Source-Projekts.

Für wen ist das Buch? Einerseits nimmt es den Einsteiger an der Hand und führt ihn mit Themen wie Projektstart, Lizenzwahl, Projekt-Logo und Namenswahl in die Open-Source-Welt ein. Andererseits zeigt der Blick in das Inhaltsverzeichnis, dass auch Beitragende von bereits bestehenden Open-Source-Projekten davon profitieren können: Kapitel zur Projektverwaltung, die Beleuchtung der unterschiedlichen Entwicklungsmodelle, Aspekte rund um das Team, Code und Releasemanagement, über Infrastruktur und Dokumentation bis hin zu Marketing und dem Thema Finanzierung sollten jedem Entwickler genug Anregungen und Stoff zur Selbstreflektion liefern.

Wer Interesse daran findet: Ich würde mich besonders freuen wenn ihr das Buch in der Buchhandlung eures Vertrauens bestellt und damit eure Buchhändler auf das Buch aufmerksam macht. Ich freue mich insbesondere über jede Rezension die für mein Buch zustande kommt. Ich halte es hier mit Tom vom Bestatterweblog: "Wenn viele gute Rezensionen bei einem Buch stehen, erhöht sich natürlich die Wahrscheinlichkeit, dass auch andere es noch kaufen, und darüber würde ich mich natürlich auch sehr freuen."

An dieser Stelle möchte ich mich noch einmal bei allen Beteiligten recht herzlich bedanken. Insbesondere bei meiner Verlobten, die auf ein 3 Jahre andauerndes Projekt Rücksicht genommen hat und mehr als nur den einen oder anderen Abend auf mich verzichten musste. Ich möchte mich auch noch bei Rhonda für die sehr schönen Worte über mich und Grml bedanken.

Also, viel Spass beim Lesen und ich hoffe man sieht sich auf der einen oder anderen Open-Source-Konferenz!

via Peter Kuhm:

Am Donnerstag, den 18. November 2010 finden um 18:30 Uhr die Vorträge

* Internet und Recht – Stolpersteine und Wegmarken
* Privatsphäre im Internet – Facebook, Xing, Twitter und Co statt.

Wo: TU Graz, HS E, Kopernikusgasse 24/1, 8010 Graz.

Vortragende: MMag. Dr. Albrecht HALLER, Rechtsanwalt, Wien
Mag. Markus KLEMEN, Geschäftsführer SBA Research GmbH, Wien

Nähere Informationen finden Sie unter:
http://www.ove.at/akademie/details.php?ID=1233

Die Vorträge werden in Kooperation mit der Österreichischen Computer Gesellschaft (OCG), der Fakultät für Elektrotechnik und Informationstechnik, der Fakultät für Informatik und dem Absolventenverein ELiTE veranstaltet und sind – wie immer – kostenlos. Im Anschluss gibt es bei einem Buffet (auf Einladung von OVE und OCG) die Möglichkeit zum Informationsaustausch in geselliger Atmosphäre.

Die Teilnahme ist kostenlos, um Anmeldung wird jedoch gebeten.

Remember my LART of the day? It looked like:

% cat foo.c
int main() { return 0; }
% gcc foo.c
foo.c:1: fatal error: can’t open /tmp/ccxgyEhb.s for writing: Permission denied
compilation terminated.

Congratulations to sunckell and Bernd for guessing the right answer.

Additionally to the blog comments (which I delayed through moderation to give everyone the same chance) I received further comments through ICQ/Jabber/IRC. It’s interesting that most people think it’s related to /tmp, but the issue isn’t related to /tmp at all.

The solution

Umask settings.

Just try my demo with ‘umask 222’. No mount/ACL/SELINUX/whatsoever tricks necessary. :) To clarify:

% umask
002
% gcc foo.c
% umask 222
% gcc foo.c
foo.c:1: fatal error: can’t open /tmp/ccteC66I.s for writing: Permission denied
compilation terminated.
% touch /tmp/ccteC66I.s
% ls -la /tmp/ccteC66I.s
-r--r--r-- 1 mika mika 0 Oct 31 12:56 /tmp/ccteC66I.s

I stumbled upon this LART by accident. As part of implementing a specialized configuration management system for a customer I had to deal with permission handling. The requirement was to generate configuration files based on the permissions of the template files but remove any existing write permissions. While playing with umask settings I noticed that you can’t set umask in the shell to get the executable flag being set. I was aware of umask(2) and always had its 0777 in mind. But files are actually generated via:

0666 & ~umask

by default instead. As I noticed that most people aren’t aware of this fact and that a umask of 222 turns out to be pretty unobtrusive this was the perfect base for a LART. Learning something new and coming up with a LART at the same time. Win-win! :)

Update: see LART of the day – the solution for the solution.

% cat foo.c
int main() { return 0; }
% gcc foo.c
foo.c:1: fatal error: can’t open /tmp/ccxgyEhb.s for writing: Permission denied
compilation terminated.

Uh?! Ok, does the file exist? Do I have write permissions? gcc ok?

% pwd
/tmp
% ls -lad /tmp
drwxrwxrwt 23 root root 1000 Oct 29 16:43 /tmp/
% ls -la /tmp/ccxgyEhb.s
ls: cannot access /tmp/ccxgyEhb.s: No such file or directory
% echo foo > /tmp/ccxgyEhb.s
% cat /tmp/ccxgyEhb.s
foo
% which gcc
/usr/bin/gcc
% gcc --version | head -1
gcc (Debian 4.4.3-7) 4.4.3

Alright, everything looks fine, right?

Assuming I would apply this LART to YOU, would you correctly guess what I did to you?

Auf der FrOSCon (Free and Open Source Software Conference) habe ich am letzten Wochenende einen Vortrag mit dem Titel "Gute Open-Source-Projekte bestehen aus mehr als nur Code" gehalten.

Das Videoteam war heuer schnell (danke!) und daher gibt es für Interessierte zu meinen Vortragsfolien (PDF, 10MB) auch bereits das Video zum Vortrag:

• h264-Format, 118MB
• theora-Format, 290MB

Finally I’m back from DebConf10 in New York. It was the first DebConf I attended and it was just great. Thanks a lot to all involved people for making DebConf such a great event.

I had a big todo list for DebCamp and DebConf and managed to get a bunch of work done. Besides several discussions, bug hunting, testing stuff and attending BoFs and talks the work that I could get done for Debian included:

• released FAI 3.4.0
• released initramfs-tools 0.98
• sponsored upload of:
  • pyneighborhood
  • md5deep
  • grokevt
  • kpicosim
  • shunit2
  • mpris-remote
• uploaded new version of stressapptest (required some build tuning)
• prepared new upload of tct
• fixed RC bugs:
  • #587557 – problems installing plymouth
  • #591257 – stressapptest: FTBFS on sparc: configure: error: sparc is not supported!
  • #589836 – mdadm: breaks initramfs on fresh (chroot) installation

My talk about "State of Debian (based) Linux live systems in 2010" went pretty well according to the feedback I got (thanks for that), even though OpenOffice failed horrible once again (one completely broken line on one slide, several broken gradients on several slides in presentation mode, the presenter screen just didn’t work at all,…). I just uploaded the slides of my talk (8.4MB, PDF), though they won’t be useful without the talk – but thanks to the awesome videoteam my talk is available as recording.

On the next weekend (August 21st/22nd) you can meet me at FrOSCon/Germany where I’ll be at the Grml booth and giving a talk titled "Gute Open-Source-Projekte bestehen aus mehr als nur Code" (english: Good open source projects are more than just code).

BTW: What’s a wiki? Say whaaaaaat?</insider>

On each last Friday of July the annual System Administrator Appreciation Day is taking place.

Matt Simmons organized a SysAdmin Day Meetup to celebrate this event in New York. If you’re in New York on 30th of July consider registering yourself (it’s free and takes just a few seconds).

If you are participant of the Debian Conference in New York and are already hacking at the DebCamp you might want to join our group of Debian people who plan to show up, currently consisting of Paul Wise, Lars Wirzenius, Thomas Lange and myself. If you plan to join please ping me so we can show up at the SysAdmin Day Meetup together.

Last weekend (2010-07-02 – 2010-07-04) nine people met at the FAI developer workshop at Linuxhotel in Essen/Germany. If you can’t remember: FAI is a non-interactive system to install, customize and manage Linux systems and software configurations on computers as well as virtual machines and chroot environments, from small networks to large-scale infrastructures and clusters.

The participants of the FAI meeting:

second row from left to right: Michael Goetze, Michael Prokop, Andreas Schuldei
first row from left to right: Sebastian Hetze, Manuel Hachtkemper, Thomas Lange, Mattias Jansson
missing on the picture: Thomas Neumann (left on sunday midday) and Stephan Hermann (only part-time)

Friday afternoon started with getting to know each other, continuing with discussions all around FAI. On saturday we started to hack on FAI.

Between the hack sessions and discussions the attending people presented their FAI usage and approaches. Some notes from the presentations:

FAI Manager webfrontend / Stephan Hermann

Stephan ‘\sh’ Hermann presented his FAI web frontend which should be released under the GPL license in those days. The frontend uses qooxdoo whereas the backend is based on django, rpc4django and python-tftpy.

A demo video is available at blip.tv. Currently Stephan is searching for a nice name for his FAI management tool – please send suggestions either to him or to the linux-fai-devel mailinglist.

Grml / Michael Prokop

Grml is a Debian based Linux live system specially made for system administrators. Grml uses grml-live for building the ISOs, whereas grml-live itself uses FAI’s dirinstall feature to build the live system. This provides the Grml team with a nice way to autobuild 18 ISOs per day, known as daily.grml.org. Mika also presented Grml’s netscript bootoption and the ethdevice bootoption of live-initramfs which is useful for booting Grml/FAI via PXE.

Host Europe / Michael Goetze

Host Europe uses FAI for installing Debian and Ubuntu (32+64 bit) in the support center. They have ~20 FAI classes and use a Debian lenny NFSROOT as base for all deployed systems. Their main problems with FAI aren’t related to FAI itself, but instead e.g. broadcom NICs with lack of support for it in Lenny’s kernel. They are not using softupdate (yet) and currently use Kickstart for deploying CentOS but are working on deploying CentOS with FAI as well.

LIS AG / Sebastian Hetze

Linux Information Systems AG (LIS AG) are using FAI 3.2.17 and provide a luma and PyQt based GUI to their customers. They use DHCP, LDAP and DDNS for inventory, configuration and deployment.

Mathematical Institute of the University of Bonn / Manuel Hachtkemper

The Mathematical Institute of the University of Bonn uses FAI 3.1.8 and 3.3.5 for managing ~150 systems. They are automatically running softupdates every day, reporting how many hosts actually did run the softupdate and how many didn’t run. The involved failogwatch tool supports two regex files, one for excluding specific hosts and the other one for grepping for known problems in the logs.

Spotify / Andreas Schuldei + Mattias Jansson

Spotify is a peer-to-peer music streaming service and the operating people at Spotify use FAI for deploying the systems. Currently they are using FAI 3.3.3 to deploy ~400 bare metal machines and ~150 virtualised machines. They have their class names in DNS using the txt/Text record entry. They are using a self written prepend_class script to manage dependencies between classes.

University Köln / Thomas Lange

Thomas uses FAI’s trunk version (of course :)), managing ~25 machines with less than 20 FAI classes. He’s not using softupdates as Lenny’s aptitude ignores the hold status of packages (this bug should be fixed for Squeeze).

$COMPANY

One of the big telecommunication providers in Germany uses FAI 3.3.3 for installing their bare-metal and virtual servers, providing Debian, Ubuntu and SLES. They are using Debian NFSROOT as a base for all systems as well and their main problems with FAI wasn’t FAI itself but how to manage installation of virtual machines.

On Saturday evening we had a nice barbecue which included beer and Kölsch *d&r*. ;) On Sunday we continued with discussions and development.

Our work-log of the weekend:

• identified important packages for the Debian/squeeze release
• discussed features that should be available (important packages and bugreports, missing features,…)
• fixed several bugs (wrong exit codes, error handling, variable handling,…)
• discussed FAI packaging for Ubuntu (packages for lucid are available at ppa at launchpad)
• implemented support for grub2 in fai-cd (will be merged soon)
• implemented initial support for retrieving sources with FAI (available in svn’s trunk)

Important decisions made:

• next major release will have version number 4.x
• we want to continue to provide a stable version 3.x (no new features, just bugfixes and maintenance) side-by-side with version 4.x
• deprecated setup-harddisk will be dropped, setup-storage is well established, works fine and is properly maintained by FAI developer Michael Tautschnig (who sadly couldn’t attend the FAI developer meeting)
• a FAQ section will be created on the FAI homepage

We noticed that many FAI users implement their own way how to handle dependency management between classes, we will re-consider how we could provide such a mechanism through FAI’s core. We also noted that it’s important that any self-written scripts used within FAI are fully idempotent and users should be aware of this.

Last but not least – many thanks to the sponsors of the FAI developer workshop 07/2010! The workshop wouldn’t have been possible without our generous sponsors, namely being:

• Debian
• Linuxhotel
• Linux Information Systems AG
• Netways
• Spotify
• Thomas-Krenn.AG