Creating a SLES11 Install Disc With Infiniband Support

June 11, 2009 - 12:14 pm
sysadmin, suse, infiniband

Note that this may or may not work for any given version/update of SLES11; the main thing is whether or not the installer kernel can load the drivers installed by the OS itself. In a worst case scenario, one could always install SLES11, grab the appropriate kernel from kernel.org, grab the OFED packages from Mellanox/etc. and generate some compatible modules the old fashioned way. As I've not yet encountered a need to do that yet, I leave it up to the reader for now.

Creating the Disc:

Install SLES11 on a box. Options are generally whatever - all we care about are the Infiniband drivers that will be installed. Conversely, you need to select the Infiniband packages for installation.

Once installed, boot, and copy out the following directory: /lib/modules/*-default/kernel/drivers/infiniband

Pop your SLES11 install disc into a box and copy everything off into a staging directory.

Grab the following file from your newly-copied-to-disk install disc: /path/to/ripped/install_disc/boot/x86_64/loader/initrd

Copy the initrd file into a holding directory, then execute:

gzip -dc initrd | cpio -id
rm initrd

This extracts the initrd into it's fully evolved form. My Pokemanz. Let me show u them.

Now copy your previously-copied Infiniband modules to: /path/to/extracted/initrd/modules

Note, copy the modules, not the directory structure. Just dump all the .ko files directly into the modules symlink.

Now repackage everything:

cd /path/to/extracted/initrd
find ./ | cpio -H newc -o > /tmp/initrd
gzip /tmp/initrd
mv /tmp/initrd.gz /tmp/initrd

Grab /tmp/initrd and replace it into the directory structure of your SuSE rip; eg:

cp /tmp/initrd /path/to/ripped/install_disc/boot/x86_64/loader

Create a bootable .iso:

cd /path/to/ripped/install_disc
mkisofs -JR -o /tmp/suse-custom.iso -b boot/x86_64/loader/isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table .

Burn.

Actual Use:
  1. Agree to license
  2. Abort
  3. Expert
  4. Open shell
  5. Execute the following:
modprobe ib_mthca
modprobe ib_umad
modprobe ib_ipoib
ifconfig ib# ###.###.###.### up
exit

Obviously, ib# will depend on what port you're using; and naturally you'll also need an IP set aside.

  1. Exit shell
  2. Proceed with normal installation

And you're ready to install normally. (Or abnormally.)