HOWTO_use_OpenGFS_with_UML(User_Mode_Linux) (Jul 29 2004)


Contact the author.

		Running OpenGFS under User Mode Linux (UML)

Copyright 2003 The OpenGFS Project.

Introduction
------------

User mode Linux is a set of patches for the Linux kernel that allows running
a nested Linux kernel as a userland process.  As it is, UML is mainly
interesting for developers to debug kernel components.  It may also make sense
to use UML as a replacement for stomith hardware:  when a box falls out of the
configured cluster, use "kill -9" as a stomith method.

This document assumes you are already familiar with setting up OpenGFS on a
normal cluster.  Please refer to the HOWTO-generic for further details.
Compiling for UML is supported only for opengfs-0.3.0 and later.

UML Setup
---------

1. Download the target kernel source tarball, for example linux-2.4.20.tar.gz.

2. Download a matching UML patch from http://user-mode-linux.sourceforge.net/
   Example:  uml-patch-2.4.20-1.bz2

3. OpenGFS now (as of July 30, 2003, and any releases after that date)
   requires patches for the DM device mapper to be in your kernel.
   You have (at least) 2 choices for obtaining them:

	A.  If you are going to use EVMS, see (OpenGFS doc) HOWTO-evms for
            information on how to install EVMS and DM (EVMS uses DM), including
            information on the convenient tarballs available on the OpenGFS
            download site (*not* part of the OpenGFS CVS source tree).

	B.  If you will not use EVMS, and want to keep your patching as compact
            as possible, obtain patches directly from DM project website,
		http://sources.redhat.com/dm/
            Select the latest patches that apply to your kernel.

4. Unpack the Linux sources and the patches and apply the patches.
   HINT:  To avoid patch conflicts, you may want to start with a pristine
   kernel.

     $ tar xvzf linux-2.4.20.tar.gz
     $ bunzip2 uml-patch-2.4.20-1.bz2
     $ cd linux-2.4.20
     $ patch -p1 < ../
     $ patch -p1 < ../uml-patch-2.4.20-1

5. Apply the OpenGFS kernel patch(es).

     $ patch -p1 < /kernel_patches/2.4.20/linux-2.4.20-ogfs.patch

6. Configure and compile the kernel.  In the basic setup you do not need to
   change any configuration options.

     $ make menuconfig ARCH=um
     $ make dep ARCH=um
     $ make clean ARCH=um
     $ make linux ARCH=um

   IMPORTANT NOTE:  If you forget to add "ARCH=um" at the end of any of these
   commands, either the UML kernel or OpenGFS will not build properly.
   Specifically, you can not build OpenGFS with a kernel source tree that was
   configured without ARCH=um, even if you repeat the process later.  You have
   to remove the whole kernel tree and begin from scratch if that happens.
   Believe me.

7. Fetch a UML root file system of your choice.  You need to compile OGFS on the
   UML machine, so you will need a few libraries and development tools
   installed.  You can download a small Debian-3.0 form the OpenGFS sourceforge
   page that should help ease the initial setup (skip to step H):

     http://sourceforge.net/projects/opengfs

   If it is not good enough for you, you have to make your own.  For Debian,
   try this:

   A. Install the debootstrap packacge on your regular machine.

   B. Create a file for the root_fs image

        # 1GB sparse root image
        $ dd if=/dev/zero of=root_fs bs=1M seek=1023 count=1

   C. Create a file system.

        $ mke2fs root_fs
        (answer 'y'es in the confirmation dialog)

   D. Mount the image via the loop driver.

        $ mount root_fs /mnt -t ext2 -o loop

   E. Use debootstrap to create a minimal Debian installation:

        $ debootstrap woody /mnt

   F. Install more packages:

        $ chroot /mnt
        $ mount -t proc proc /proc
        # install packages as usual.
        # ...
        $ umount /proc
        $ exit

      Do not forget to unmount the procf.  If you forget it, you can not
      unmount the file system image anymore.

   G. Unmount the image and the procfs

        $ umount /mnt

      If you start UML before unmounting, you have to reboot your machine to
      remove the stale mount.

   H. Start the UML machine

        $ nice ./linux mem=200M

OGFS Setup
----------

Below this point, all commands are typed in the UML machine.

   I. Log in as root and mount the host file system in the UML machine:

        $ mount -t hostfs hostfs /mnt

      Note:  You need to compile hostfs support into the UML kernel to use this
      feature.

   J. Copy the opengfs tarball to the UML machine and unpack it

        $ cd /usr/local
        $ cp /mnt//opengfs-x.y.z.tar.gz .
        $ tar xvzf opengfs-x.y.z.tar.gz

   K. Configure opengfs

        $ cd opengfs-x.y.z
        $ ./configure --enable-uml \
          --with-linux_srcdir=/mnt//linux-2.4.20 \
          --with-linux_moduledir=/lib/modules/2.4.20-1um

   L. Build and install opengfs

        $ make
        $ make install

The instructions below assume you want to run an ogfs file system locally on a
single UML machine.  Things become much more complicated for a cluster setup.

   M. Load the kernel modules

        $ insmod lock_harness
        $ insmod divdi3
        $ insmod ogfs
        $ insmod nolock

   N. Create a file for the ogfs file system and a mountpoint and format the
      file

        $ mkdir /ogfs
        $ dd if=/dev/zero of=/ogfs_file bs=1M seek=255 count=1
        $ mkfs.ogfs -i -j 1 -J 32 -p nolock /ogfs_file

   O. Mount the ogfs file system

        $ mount -t ogfs /ogfs /ogfsmnt -o loop

Debugging
---------

Now we want to debug the kernel modules.  The following lines are typed on the
normal machine.

IMPORTANT NOTE:  OGFS versions prior to 0.3.0 are compiled with the
-fomit-frame-pointer gcc option by default which makes it difficult to use a
debugger with the modules.

   P. Attaching a debugger

      When the UML starts, it prints a message "tracing thread pid = ".
      Find out the  and start the UML debugger with

        $ kill -USR1 

      An xterm running gdb should show up.

      If you are running only one UML machine, you can use this script instead:
--------------------------------- snip ----------------------------------
#!/bin/sh

TTPID=`find /proc -name cmdline 2> /dev/null | xargs grep -il "\[(tracing thread)\]" 2> /dev/null | sed -e 's:/proc/\([[:digit:]]*\)/.*:\1:g' 2> /dev/null`

if test x"$TTPID" = x; then
  echo no tracing thread found
  exit 1
else
  echo "sending SIGUSR1 to thread $TTPID"
  kill -USR1 $TTPID
fi
--------------------------------- snip ----------------------------------

   Q. Loading the symbol table

      On the debugger prompt type

        (gdb) p module_list->name
        (gdb) p module_list->next->name
        (gdb) p module_list->next->next->name
        ...

      Until you find the "ogfs" module.  On my UML machine it is always
      module_list->next->name.  Get the address of the corresponding structure
      and add a certain constant offset:

        (gdb) p /x (unsigned int)module_list->next + module_list->size_of_struct

      This prints an address to the gdb window.  With this address, type

        (gdb) dir /src/fs/arch_linux
        (gdb) dir /src/include
        (gdb) add-symbol-file /src/fs/arch_linux/ogfs.o 
is the path to the ogfs sources on the normal machine. If you start UML and ogfs in the same way each time, the address will probably have the same value each time. You can then put the above steps in a .gdbinit file (in the directory from which you started the UML): --------------------------------- snip ---------------------------------- dir /src/fs/arch_linux dir /src/include add-symbol-file /src/fs/arch_linux/ogfs.o
--------------------------------- snip ---------------------------------- R. Set a breakpoint and start debugging For example: (gdb) b ogfs_mkdir (gdb) cont That's it.
SourceForge Logo