Finally I’m back from DebConf10 in New York. It was the first DebConf I attended and it was just great. Thanks a lot to all involved people for making DebConf such a great event.

I had a big todo list for DebCamp and DebConf and managed to get a bunch of work done. Besides several discussions, bug hunting, testing stuff and attending BoFs and talks the work that I could get done for Debian included:

• released FAI 3.4.0
• released initramfs-tools 0.98
• sponsored upload of:
  • pyneighborhood
  • md5deep
  • grokevt
  • kpicosim
  • shunit2
  • mpris-remote
• uploaded new version of stressapptest (required some build tuning)
• prepared new upload of tct
• fixed RC bugs:
  • #587557 – problems installing plymouth
  • #591257 – stressapptest: FTBFS on sparc: configure: error: sparc is not supported!
  • #589836 – mdadm: breaks initramfs on fresh (chroot) installation

My talk about "State of Debian (based) Linux live systems in 2010" went pretty well according to the feedback I got (thanks for that), even though OpenOffice failed horrible once again (one completely broken line on one slide, several broken gradients on several slides in presentation mode, the presenter screen just didn’t work at all,…). I just uploaded the slides of my talk (8.4MB, PDF), though they won’t be useful without the talk – but thanks to the awesome videoteam my talk is available as recording.

On the next weekend (August 21st/22nd) you can meet me at FrOSCon/Germany where I’ll be at the Grml booth and giving a talk titled "Gute Open-Source-Projekte bestehen aus mehr als nur Code" (english: Good open source projects are more than just code).

BTW: What’s a wiki? Say whaaaaaat?</insider>

Matt Simmons organized a SysAdmin Day Meetup to celebrate this event in New York. If you’re in New York on 30th of July consider registering yourself (it’s free and takes just a few seconds).

If you are participant of the Debian Conference in New York and are already hacking at the DebCamp you might want to join our group of Debian people who plan to show up, currently consisting of Paul Wise, Lars Wirzenius, Thomas Lange and myself. If you plan to join please ping me so we can show up at the SysAdmin Day Meetup together.

The participants of the FAI meeting:

second row from left to right: Michael Goetze, Michael Prokop, Andreas Schuldei
first row from left to right: Sebastian Hetze, Manuel Hachtkemper, Thomas Lange, Mattias Jansson
missing on the picture: Thomas Neumann (left on sunday midday) and Stephan Hermann (only part-time)

Friday afternoon started with getting to know each other, continuing with discussions all around FAI. On saturday we started to hack on FAI.

Between the hack sessions and discussions the attending people presented their FAI usage and approaches. Some notes from the presentations:

FAI Manager webfrontend / Stephan Hermann

Stephan ‘\sh’ Hermann presented his FAI web frontend which should be released under the GPL license in those days. The frontend uses qooxdoo whereas the backend is based on django, rpc4django and python-tftpy.

A demo video is available at blip.tv. Currently Stephan is searching for a nice name for his FAI management tool – please send suggestions either to him or to the linux-fai-devel mailinglist.

Grml / Michael Prokop

Grml is a Debian based Linux live system specially made for system administrators. Grml uses grml-live for building the ISOs, whereas grml-live itself uses FAI’s dirinstall feature to build the live system. This provides the Grml team with a nice way to autobuild 18 ISOs per day, known as daily.grml.org. Mika also presented Grml’s netscript bootoption and the ethdevice bootoption of live-initramfs which is useful for booting Grml/FAI via PXE.

Host Europe / Michael Goetze

Host Europe uses FAI for installing Debian and Ubuntu (32+64 bit) in the support center. They have ~20 FAI classes and use a Debian lenny NFSROOT as base for all deployed systems. Their main problems with FAI aren’t related to FAI itself, but instead e.g. broadcom NICs with lack of support for it in Lenny’s kernel. They are not using softupdate (yet) and currently use Kickstart for deploying CentOS but are working on deploying CentOS with FAI as well.

LIS AG / Sebastian Hetze

Linux Information Systems AG (LIS AG) are using FAI 3.2.17 and provide a luma and PyQt based GUI to their customers. They use DHCP, LDAP and DDNS for inventory, configuration and deployment.

Mathematical Institute of the University of Bonn / Manuel Hachtkemper

The Mathematical Institute of the University of Bonn uses FAI 3.1.8 and 3.3.5 for managing ~150 systems. They are automatically running softupdates every day, reporting how many hosts actually did run the softupdate and how many didn’t run. The involved failogwatch tool supports two regex files, one for excluding specific hosts and the other one for grepping for known problems in the logs.

Spotify / Andreas Schuldei + Mattias Jansson

Spotify is a peer-to-peer music streaming service and the operating people at Spotify use FAI for deploying the systems. Currently they are using FAI 3.3.3 to deploy ~400 bare metal machines and ~150 virtualised machines. They have their class names in DNS using the txt/Text record entry. They are using a self written prepend_class script to manage dependencies between classes.

University Köln / Thomas Lange

Thomas uses FAI’s trunk version (of course ), managing ~25 machines with less than 20 FAI classes. He’s not using softupdates as Lenny’s aptitude ignores the hold status of packages (this bug should be fixed for Squeeze).

$COMPANY

One of the big telecommunication providers in Germany uses FAI 3.3.3 for installing their bare-metal and virtual servers, providing Debian, Ubuntu and SLES. They are using Debian NFSROOT as a base for all systems as well and their main problems with FAI wasn’t FAI itself but how to manage installation of virtual machines.

On Saturday evening we had a nice barbecue which included beer and Kölsch *d&r*. On Sunday we continued with discussions and development.

Our work-log of the weekend:

• identified important packages for the Debian/squeeze release
• discussed features that should be available (important packages and bugreports, missing features,…)
• fixed several bugs (wrong exit codes, error handling, variable handling,…)
• discussed FAI packaging for Ubuntu (packages for lucid are available at ppa at launchpad)
• implemented support for grub2 in fai-cd (will be merged soon)
• implemented initial support for retrieving sources with FAI (available in svn’s trunk)

Important decisions made:

• next major release will have version number 4.x
• we want to continue to provide a stable version 3.x (no new features, just bugfixes and maintenance) side-by-side with version 4.x
• deprecated setup-harddisk will be dropped, setup-storage is well established, works fine and is properly maintained by FAI developer Michael Tautschnig (who sadly couldn’t attend the FAI developer meeting)
• a FAQ section will be created on the FAI homepage

We noticed that many FAI users implement their own way how to handle dependency management between classes, we will re-consider how we could provide such a mechanism through FAI’s core. We also noted that it’s important that any self-written scripts used within FAI are fully idempotent and users should be aware of this.

Last but not least – many thanks to the sponsors of the FAI developer workshop 07/2010! The workshop wouldn’t have been possible without our generous sponsors, namely being:

• Debian
• Linuxhotel
• Linux Information Systems AG
• Netways
• Spotify
• Thomas-Krenn.AG

As the name states the workshop is targeted towards FAI developers. We – the FAI developers – want to get FAI into shape for squeeze, discuss pending issues like Ubuntu packaging, release management and of course meet in real life for networking and socializing. Our rough roadmap for the FAI weekend looks like this:

Friday:

• Setup of the network
• Introduction round, hello to everybody
• Define things that need to be worked on with high priority for the squeeze release
• Start of work

Saturday:

• Work on things for squeeze release
• Discussion: Features for squeeze
• Read access from /dev/beer

Sunday:

• Work on things for squeeze release
• Final round: What did we manage to do this weekend

Further details are available in the FAI wiki at http://faiwiki.informatik.uni-koeln.de/index.php/DeveloperWorkshopJuly2010.

The meeting wouldn’t be possible without sponsors – so special thanks to:

• Debian
• Linuxhotel
• Netways
• Spotify

If you are interested in sponsoring the FAI Developer Workshop as well please contact FAI lead developer Thomas Lange.

Quoting our TOS / Privacy Info:

TitanPad was launched to provide an EtherPad setup which is unrelated to any commercial and political entities. Its goal is to offer a stable service through proper operating.
[...]

Now TitanPad is officially up and running and you’re free to use it for online collaboration. Feel free to drop your feedback, questions and suggestions to our team via mail to support (at) titanpad.com.

Fsck you, FIP.

After migrating one of my 32bit systems to a fresh 64bit system I suddenly couldn’t use this Remote Console feature any longer. The error message was (leaving it for search engines and help other affected users):

load: class vnc.VncViewer.class not found.
java.lang.ClassNotFoundException: vnc.VncViewer.class
	at sun.plugin2.applet.Applet2ClassLoader.findClass(Applet2ClassLoader.java:152)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
	at sun.plugin2.applet.Plugin2ClassLoader.loadCode(Plugin2ClassLoader.java:447)
	at sun.plugin2.applet.Plugin2Manager.createApplet(Plugin2Manager.java:2880)
	at sun.plugin2.applet.Plugin2Manager$AppletExecutionRunnable.run(Plugin2Manager.java:1397)
	at java.lang.Thread.run(Thread.java:619)
Caused by: java.net.ConnectException: Network is unreachable
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
	at java.net.Socket.connect(Socket.java:525)
	at sun.net.NetworkClient.doConnect(NetworkClient.java:161)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
	at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
	at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)
	at sun.net.www.http.HttpClient.New(HttpClient.java:306)
	at sun.net.www.http.HttpClient.New(HttpClient.java:323)
	at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:860)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:801)
	at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:726)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1049)
	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:373)
	at sun.plugin2.applet.Applet2ClassLoader.getBytes(Applet2ClassLoader.java:458)
	at sun.plugin2.applet.Applet2ClassLoader.access$000(Applet2ClassLoader.java:46)
	at sun.plugin2.applet.Applet2ClassLoader$1.run(Applet2ClassLoader.java:126)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.plugin2.applet.Applet2ClassLoader.findClass(Applet2ClassLoader.java:123)
	... 6 more
Exception: java.lang.ClassNotFoundException: vnc.VncViewer.class

The error message might not be obvious at a glance and that’s why I’m writing about it actually. It’s NOT the:

load: class vnc.VncViewer.class not found.

why it’s failing but instead the real reason for the failure is the:

java.net.ConnectException: Network is unreachable

As you can read in Debian’s Bug Tracking System in bug #560044:

Netbase has recently introduced the sysctl-setting
net.ipv6.bindv6only=1 in /etc/sysctl.d/bindv6only.conf and this setting will probably be the default in squeeze.

This setting breaks networking in java, and any traffic will always
result in a “java.net.SocketException: Network is unreachable”.

To quote /etc/sysctl.d/bindv6only.conf:

When disabled, IPv6 sockets will also be able to send and receive IPv4 traffic with addresses in the form ::ffff:192.0.2.1 and daemons listening on IPv6 sockets will also accept IPv4 connections.

When IPV6_V6ONLY is enabled, daemons interested in both IPv4 and IPv6 connections must open two listening sockets.

To work around this issue you can either execute the Java process through "java -Djava.net.preferIPv4Stack=true" or to change the IPv6 behaviour system wide execute "sysctl -w net.ipv6.bindv6only=0". To make this setting permanent across reboots adjust the setting inside /etc/sysctl.d/bindv6only.conf. After applying this workaround the Remote Console should work again.

  – arch/sparc/kernel/traps_64.c

Just booked my ticket for DebConf10. See you in New York.

# fdisk -l
Disk /dev/sda: 8 MB, 8388608 bytes
8 heads, 32 sectors/track, 64 cylinders
Units = cylinders of 256 * 512 = 131072 bytes

Disk /dev/sda doesn't contain a valid partition table

… then the scsi_debug kernel driver is present. To get rid of the device either unload the driver or (if e.g. statically compiled into the kernel) use sysfs, like:

# echo -1 > /sys/bus/pseudo/drivers/scsi_debug/add_host

Further details available at http://sg.danny.cz/sg/sdebug26.html.