Difference between revisions of "LinuxMCE-0810 DVD-Installer-Implementation-Notes"

Revision as of 18:16, 27 June 2009

This entry is to document my progress (merkur2k) on the creation of a debian/ubuntu based installer for LinuxMCE 0810. I had some real detailed notes that got lost in a system crash (go windows) so I am doing most of this by memory.
Basically this has all come from this page: [1]

Current issues

• Have yet to get it to fully install. It is hard for me to tell if the errors are real or just from virtualbox (it likes to corrupt files during heavy i/o for me). Have completed an install of a kubuntu system using the custom iso.
• Still need more answers in the preseed file. Encrypted user home dir comes to mind.
• Of course I haven't even made it to the LinuxMCE installer yet, but I do have a script ready to try.

To start off with, you will need a machine to install a copy of kubuntu on for some resources needed by the installer. This machine can either be physical hardware or a virtual machine, and it would help if it has the right hardware to be used as a test mule for the finished installer (at least 2 network interfaces, etc). I also assume you have a build environment and have successfully completed a build.

Prepare build environment

Install some required build dependencies (Thanks Tschak909)

apt-get install libgd2-xpm-dev libx86-dev

You will need an existing machine with a copy of any flavor 8.10 *ubuntu on it. I use my core/hybrid.

• Create some intial directories

mkdir -p /opt/LMCE_DVD/LMCE_extras
mkdir /opt/LMCE_DVD/LMCE_extras/isolinux
mkdir /opt/LMCE_DVD/LMCE_extras/preseed
mkdir /opt/LMCE_DVD/LMCE_extras/lmce-installer
mkdir -p /opt/LMCE_DVD/LMCE_extras/pool/extras

• Everything in LMCE_Extras gets copied into the new cd's root filesystem, overwriting anything that was already there. This lets us customize any part of the resulting iso that we want. Create the following files:

/opt/LMCE_DVD/LMCE_Extras/lmce-installer/dvd-installer.sh (just a stub for now)

#!/bin/bash
dialog --title 'LinuxMCE 0810 Alpha 7' --msgbox 'This is the installer!' 5 30

/opt/LMCE_DVD/LMCE_extras/isolinux/text.cfg

default lmce
LABEL lmce
  menu label ^LinuxMCE installation
  kernel /install/vmlinuz
  append  preseed/file=/cdrom/preseed/lmce.seed initrd=/install/initrd.gz quiet --
label install
  menu label ^Install Kubuntu
  kernel /install/vmlinuz
  append  file=/cdrom/preseed/kubuntu.seed initrd=/install/initrd.gz quiet --
label check
  menu label ^Check CD for defects
  kernel /install/vmlinuz
  append  MENU=/bin/cdrom-checker-menu initrd=/install/initrd.gz quiet --
label memtest
  menu label Test ^memory
  kernel /install/mt86plus
label hd
  menu label ^Boot from first hard disk
  localboot 0x80

/opt/LMCE_DVD/LMCE_extras/preseed/lmce.seed

# netcfg will choose an interface that has link if possible. This makes it
# skip displaying a list if there is more than one interface.
d-i netcfg/choose_interface select auto
d-i netcfg/get_hostname string dcerouter
d-i netcfg/get_domain string local
# The kernel image (meta) package to be installed; "none" can be used if no
# kernel is to be installed.
d-i base-installer/kernel/image linux-generic
# To create a normal user account.
d-i passwd/user-fullname string LinuxMCE User
d-i passwd/username string linuxmce
# Normal user's password
d-i passwd/user-password password linuxmce
d-i passwd/user-password-again password linuxmce
tasksel tasksel/first multiselect standard, kubuntu-desktop
# Avoid that last message about the install being complete.
d-i finish-install/reboot_in_progress note
# Start the LMCE Installer script
d-i preseed/late_command string cp /cdrom/lmce-installer/dvd-installer.sh /target/root/; chroot /target chmod +x /root/dvd-installer.sh; chroot /target bash /root/dvd-installer.sh

• Download kubuntu-8.10-alternate-i386.iso from your favorite ubuntu mirror and place it in /opt/LMCE_DVD
• Copy all debs from the LinuxMCE 0810 builder into the installer (assuming same system, modify as required)

cp /media/kubuntu-8.10/var/www/*deb /opt/LMCE_DVD/LMCE_extras/pool/extras/

Install Kubuntu in test system

• Download kubuntu-8.10-alternate-i386.iso from your favorite ubuntu mirror or copy from above.
• Install it to the test system (burn the iso to a cd if required). Use the same options you would if installing the desktop version in preparation of the 0810 alpha installer.
• Once it is booted back up, log in and open a terminal.
• Clear out all the cached .deb files from the install

rm -f /var/apt/archives/*deb

• Now would be a good time to do a dist-upgrade so that everything on the iso is current as of the time of creation

apt-get update
apt-get dist-upgrade

• Install LinuxMCE dependencies (these may not be complete)

apt-get install -y -f apache2 apache2-mpm-prefork apache2-utils apache2.2-common atftpd attr autofs bind9 bind9utils cabextract citadel-common citadel-mta citadel-server curl db4.6-util ddclient debootstrap dhcp3-server discover discover-data etherwake ftp-upload gawk gsfonts-x11 java-common libapache2-mod-php5 libapr1 libaprutil1 libcitadel1 libconvert-binhex-perl libcrypt-smbhash-perl libcrypt-ssleay-perl libdigest-md4-perl libdiscover2 libevent1 libfcgi-perl libflickr-api-perl libgd2-xpm libglade2-0 libgssglue1 libical0 libid3-3.8.3c2a libio-socket-ssl-perl libio-stringy-perl libmime-tools-perl libnet-ssleay-perl libnfsidmap2 libossp-uuid-perl libossp-uuid15 librpcsecgss3 libsdl-gfx1.2-4 libsdl-image1.2 libsdl-sge libsdl-ttf2.0-0 libsdl1.2debian-all libsieve2-1 libsoap-lite-perl libstdc++5 libt1-5 libvte-common libvte9 libxml-parser-lite-tree-perl lsscsi mdadm msttcorefonts mtools nbtscan nfs-common nfs-kernel-server nis ntp odbcinst1debian1 openbsd-inetd openssh-server perlmagick php5-common php5-curl php5-gd php5-mysql portmap samba smbfs ssh sun-java6-bin sun-java6-jre syslinux unixodbc vim vim-runtime wakeonlan whois xutils-dev

• Copy all content from deb cache to the build environment. scp works well.

scp /var/cache/apt/archives/*deb user@host:/opt/LMCE_DVD/LMCE_extras/pool/extras/

Subsitite 'host' and 'user' with the hostname of your build environment machine and a user that has write access to /opt/LMCE_DVD/LMCE_extras/pool/extras.

Run the build script

This script came from the link above, but was (is?) full of bugs and so has been modified heavily already. Place it somewhere accessable (/opt/LMCE_DVD works), chmod it +x, and run it.
It will ask for a passphrase the first time through to create a gpg signing key, and will ask for it a couple additional times on each run. It's not important right now what this key is, so use something short.

#!/bin/bash
#
# Script by Leigh Purdie
#
# 1) Install a Ubuntu system, and remove packages according to your
#    requirements using synaptic.
#    Don't install any new packages that are NOT available from the CD,
#    at this stage, unless you include the packages in an 'extras' directory.
#
# 2) dpkg -l > PackageList
#    Copy this file to $BASEDIR/source on your build server.

# The Base Directory
BASEDIR="/opt/LMCE_DVD"

# This directory will contain files that need to be copied over
# to the new CD.
EXTRASDIR="$BASEDIR/LMCE_extras"
# Seed file
SEEDFILE="lmce.seed"

# Ubuntu ISO image
CDIMAGE="$BASEDIR/kubuntu-8.10-alternate-i386.iso"

# Where the ubuntu iso image will be mounted
CDSOURCEDIR="$BASEDIR/cdsource"

# Directory for building packages
SOURCEDIR="$BASEDIR/source"

# GPG
GPGKEYNAME="LinuxMCE Installation Key"
GPGKEYCOMMENT="Package Signing"
GPGKEYEMAIL="packages@linuxmce.org"
GPGKEYPHRASE="LinuxMCE"
MYGPGKEY="$GPGKEYNAME ($GPGKEYCOMMENT) <$GPGKEYEMAIL>"

# Package list (dpkg -l) from an installed system.
PACKAGELIST="$SOURCEDIR/PackageList"

# Output CD name
CDNAME=LinuxMCE-0810-alpha.iso-`date +%G%m%d%H%M%S`.iso

# 640x480 PNG with colours as specified in
# https://wiki.ubuntu.com/USplashCustomizationHowto
USPLASH="$SOURCEDIR/MyBuildSplash1.png"

# ------------ End of modifications.


################## Initial requirements
id | grep -c uid=0 >/dev/null
if [ $? -gt 0 ]; then
        echo "You need to be root in order to run this script.."
        echo " - sudo /bin/sh prior to executing."
        exit
fi

which gpg > /dev/null
if [ $? -eq 1 ]; then
        echo "Please install gpg to generate signing keys"
        exit
fi

# Create a few directories.
if [ ! -d $BASEDIR ]; then mkdir -p $BASEDIR; fi
if [ ! -d $BASEDIR/FinalCD ]; then mkdir -p $BASEDIR/FinalCD; fi
if [ ! -z $EXTRASDIR ]; then
        if [ ! -d $EXTRASDIR ]; then mkdir -p $EXTRASDIR; fi
        if [ ! -d $EXTRASDIR/preseed ]; then mkdir -p $EXTRASDIR/preseed; fi
        if [ ! -d $EXTRASDIR/pool/extras ]; then mkdir -p $EXTRASDIR/pool/extras; fi
fi
if [ ! -d $CDSOURCEDIR ]; then mkdir -p $CDSOURCEDIR; fi
if [ ! -d $SOURCEDIR ]; then mkdir -p $SOURCEDIR; fi
if [ ! -d $SOURCEDIR/keyring ]; then mkdir -p $SOURCEDIR/keyring; fi
if [ ! -d $SOURCEDIR/indices ]; then mkdir -p $SOURCEDIR/indices; fi
if [ ! -d $SOURCEDIR/kubuntu-meta ]; then mkdir -p $SOURCEDIR/kubuntu-meta; fi


if [ ! -f $CDIMAGE ]; then
        echo "Cannot find your ubuntu image. Change CDIMAGE path."
        exit
fi


gpg --list-keys | grep "$GPGKEYNAME" >/dev/null
if [ $? -ne 0 ]; then
        echo "No GPG Key found in your keyring."
        echo "Generating a new gpg key ($GPGKEYNAME $GPGKEYCOMMENT) with a passphrase of $GPGKEYPHRASE .."
        echo ""
        echo "Key-Type: DSA
Key-Length: 1024
Subkey-Type: ELG-E
Subkey-Length: 2048
Name-Real: $GPGKEYNAME
Name-Comment: $GPGKEYCOMMENT
Name-Email: $GPGKEYEMAIL
Expire-Date: 0
Passphrase: $GPGKEYPHRASE" > $BASEDIR/key.inc

        gpg --gen-key --batch --gen-key $BASEDIR/key.inc
        # Note: If you wish to remove the passphrase from the key:
        # (Don't do this if you want to use this key for ANYTHING other
        # than a temporary ubuntu CD installation signing key)
        # gpg --edit-key
        # passwd
        # (enter old phrase)
        # (blank)
        # (blank)
        # y
        # quit
        # y
fi

if [ ! -f $CDSOURCEDIR/md5sum.txt ]; then
        echo -n "Mounting Ubuntu iso.. "
        mount | grep $CDSOURCEDIR
        if [ $? -eq 0 ]; then
                umount $CDSOURCEDIR
        fi

        mount -o loop $CDIMAGE $CDSOURCEDIR/
        if [ ! -f $CDSOURCEDIR/md5sum.txt ]; then
                echo "Mount did not succeed. Exiting."
                exit
        fi
        echo "OK"
fi

if [ ! -f $SOURCEDIR/apt.conf ]; then
        echo -n "No APT.CONF file found... generating one."
        # Try and generate one?
        cat $CDSOURCEDIR/dists/intrepid/Release | egrep -v "^ " | egrep -v "^(Date|MD5Sum|SHA)" | sed 's/: / "/' | sed 's/^/APT::FTPArchive::Release::/' | sed 's/$/";/' > $SOURCEDIR/apt.conf
        echo "Ok."
fi

if [ ! -f $SOURCEDIR/apt-ftparchive-deb.conf ]; then
        echo "Dir {
  ArchiveDir \"$BASEDIR/FinalCD\";
};

TreeDefault {
  Directory \"pool/\";
};

BinDirectory \"pool/main\" {
  Packages \"dists/intrepid/main/binary-i386/Packages\";
  BinOverride \"$SOURCEDIR/indices/override.intrepid.main\";
  ExtraOverride \"$SOURCEDIR/indices/override.intrepid.extra2.main\";
};

Default {
  Packages {
    Extensions \".deb\";
    Compress \". gzip\";
  };
};

Contents {
  Compress \"gzip\";
};" > $SOURCEDIR/apt-ftparchive-deb.conf
fi

if [ ! -f $SOURCEDIR/apt-ftparchive-udeb.conf ]; then
        echo "Dir {
  ArchiveDir \"$BASEDIR/FinalCD\";
};

TreeDefault {
  Directory \"pool/\";
};

BinDirectory \"pool/main\" {
  Packages \"dists/intrepid/main/debian-installer/binary-i386/Packages\";
  BinOverride \"$SOURCEDIR/indices/override.intrepid.main.debian-installer\";
};

Default {
  Packages {
    Extensions \".udeb\";
    Compress \". gzip\";
  };
};

Contents {
  Compress \"gzip\";
};" > $SOURCEDIR/apt-ftparchive-udeb.conf
fi

if [ ! -f $SOURCEDIR/apt-ftparchive-extras.conf ]; then
        echo "Dir {
  ArchiveDir \"$BASEDIR/FinalCD\";
};

TreeDefault {
  Directory \"pool/\";
};

BinDirectory \"pool/extras\" {
  Packages \"dists/intrepid/extras/binary-i386/Packages\";
};

Default {
  Packages {
    Extensions \".deb\";
    Compress \". gzip\";
  };
};

Contents {
  Compress \"gzip\";
};" > $SOURCEDIR/apt-ftparchive-extras.conf
fi

if [ ! -f $SOURCEDIR/indices/override.intrepid.extra.main ]; then
        for i in override.intrepid.extra.main override.intrepid.main override.intrepid.main.debian-installer; do
                cd $SOURCEDIR/indices
                wget http://archive.ubuntu.com/ubuntu/indices/$i
        done
fi

# Create a 'fixed' version of the extras.main override package.
# Idea/Perl by Ferry Hendrikx, 2006
cat $SOURCEDIR/indices/override.intrepid.extra.main | egrep -v ' Task ' > $SOURCEDIR/indices/override.intrepid.extra2.main
cat $CDSOURCEDIR/dists/intrepid/main/binary-i386/Packages.gz | gzip -d | perl -e 'while (<>) { chomp; if(/^Package\:\s*(.+)$/) { $pkg=$1; } elsif(/^Task\:\s(.+)$/) { print "$pkg\tTask\t$1\n"; } }' >> $SOURCEDIR/indices/override.intrepid.extra2.main


################## Copy over the source data

echo -n "Resyncing old data...  "

cd $BASEDIR/FinalCD
rsync -atz --delete $CDSOURCEDIR/ $BASEDIR/FinalCD/
echo "OK"


################## Remove packages that we no longer require
# remove ppp packages due to a bug in the installer
find $BASEDIR/FinalCD -name "*ppp*.udeb" -exec rm {} \;
# PackageList is a dpkg -l from our 'build' server.
if [ ! -f $PACKAGELIST ]; then
        echo "No PackageList found. Assuming that you do not require any packages to be removed"
else
        cat $CDSOURCEDIR/dists/intrepid/main/debian-installer/binary-i386/Packages.gz | gzip -d > $SOURCEDIR/Packages
        cat $PACKAGELIST | grep "^ii" | awk '{print $2 "_" $3}' > $SOURCEDIR/temppackages

        echo "Removing files that are no longer required.."
        cd $BASEDIR/FinalCD
        # Only use main for the moment. Keep all 'restricted' debs
        rm -f $SOURCEDIR/RemovePackages
        # Note: Leave the udeb's alone.
        for i in `find pool/main -type f -name "*.deb" -print`; do
                FILE=`basename $i | sed 's/_[a-zA-Z0-9\.]*$//'`
                GFILE=`echo $FILE | sed 's/\+/\\\+/g' | sed 's/\./\\\./g'`
                # pool/main/a/alien/alien_8.53_all.deb becomes alien_8.53
                egrep "^"$GFILE $SOURCEDIR/temppackages >/dev/null
                if [ $? -ne 0 ]; then
                        # NOT Found
                        # Note: Keep a couple of anciliary files
                        
                        grep "Filename: $i" $SOURCEDIR/Packages >/dev/null
                        if [ $? -eq 0 ]; then
                                # Keep the debian-installer files - we need them.
                                echo "* Keeping special file $FILE"
                        else
                                echo "- Removing unneeded file $FILE"
                                rm -f $BASEDIR/FinalCD/$i

                        fi
                else
                        echo "+ Retaining $FILE"
                fi
        done
        rm -f $SOURCEDIR/Packages
fi


echo -n "Generating keyfile..   "

cd $SOURCEDIR/keyring
KEYRING=`find * -maxdepth 1 -name "ubuntu-keyring*" -type d -print`
if [ -z "$KEYRING" ]; then
        apt-get source ubuntu-keyring
        KEYRING=`find * -maxdepth 1 -name "ubuntu-keyring*" -type d -print`
        if [ -z "$KEYRING" ]; then
                echo "Cannot grab keyring source! Exiting."
                exit
        fi
fi

cd $SOURCEDIR/keyring/$KEYRING/keyrings
gpg --import < ubuntu-archive-keyring.gpg >/dev/null
rm -f ubuntu-archive-keyring.gpg
gpg --output=ubuntu-archive-keyring.gpg --export FBB75451 437D05B5 "$GPGKEYNAME" >/dev/null
cd ..
dpkg-buildpackage -rfakeroot -m"$MYGPGKEY" -k"$MYGPGKEY" >/dev/null
rm -f $BASEDIR/FinalCD/pool/main/u/ubuntu-keyring/*
cp ../ubuntu-keyring*deb $BASEDIR/FinalCD/pool/main/u/ubuntu-keyring/
if [ $? -gt 0 ]; then
        echo "Cannot copy the modified ubuntu-keyring over to the pool/main folder. Exiting."
        exit
fi

echo "OK"


################## Copy over the extra packages (if any)
if [ ! -z $EXTRASDIR ]; then
        echo -n "Copying Extra files...  "
        rsync -az $EXTRASDIR/ $BASEDIR/FinalCD/
        echo "OK"

        if [ ! -f "$EXTRASDIR/preseed/$SEEDFILE" ]; then
                echo "No seed file found. Creating one in $EXTRASDIR/preseed/$SEEDFILE."
                echo "- You will probably want to modify this file."
                echo "base-config  base-config/package-selection      string ~tkubuntu-minimal|~tkubuntu-desktop" > $EXTRASDIR/preseed/$SEEDFILE
        fi

        if [ -f $PACKAGELIST ]; then
                echo "Replacing kubuntu-desktop with a pruned package list.. "
                cd $SOURCEDIR/kubuntu-meta
                rm -rf kubuntu-*
                apt-get source kubuntu-meta
                META=`find * -maxdepth 1 -name "kubuntu-meta*" -type d -print`
                if [ -z "$META" ]; then
                      echo "Cannot grab source to kubuntu-meta. Exiting."
                      exit
                fi

                cd $META
                for i in `ls desktop*`; do
                        grep "^ii" $PACKAGELIST | awk '{print $2}' > $i.tmp
                        mv $i.tmp $i
                done

                dpkg-buildpackage -rfakeroot -m"$MYGPGKEY" -k"$MYGPGKEY" >/dev/null
                cd ..
                rm -f $BASEDIR/FinalCD/pool/main/k/kubuntu-meta/kubuntu-desktop*deb
                mv kubuntu-desktop*.deb  $BASEDIR/FinalCD/pool/main/k/kubuntu-meta/

                cp $EXTRASDIR/preseed/$SEEDFILE $BASEDIR/FinalCD/preseed/$SEEDFILE

        fi

        if [ ! -f "$EXTRASDIR/isolinux/isolinux.cfg" ]; then
                cat $CDSOURCEDIR/isolinux/isolinux.cfg | sed "s/^APPEND.*/APPEND   preseed\/file=\/cdrom\/preseed\/$SEEDFILE vga=normal initrd=\/install\/initrd.gz ramdisk_size=16384 root=\/dev\/rd\/0 DEBCONF_PRIORITY=critical debconf\/priority=critical rw --/" > $BASEDIR/FinalCD/isolinux/isolinux.cfg
        fi

        echo "OK"
fi

if [ ! -z "$USPLASH" ]; then
        echo "Modifying Usplash (NOTE: libgd2-dev required)"

        cd $SOURCEDIR
        if [ ! -d usplash ]; then
                mkdir usplash
        fi
        cd usplash
        SPLASH=`find * -maxdepth 1 -type d -name "usplash*" -type d -print`
        if [ -z "$SPLASH" ]; then
                apt-get source usplash
                SPLASH=`find * -maxdepth 1 -type d -name "usplash*" -type d -print`
        fi
        if [ -z "$SPLASH" ]; then
                echo "Cannot download USPLASH source. Exiting."
                exit
        fi

        cp $USPLASH $SOURCEDIR/usplash/$SPLASH/usplash-artwork.png
        cd $SOURCEDIR/usplash/$SPLASH
        dpkg-buildpackage -rfakeroot -m"$MYGPGKEY" -k"$MYGPGKEY" >/dev/null
        cd ..
        rm -f $BASEDIR/FinalCD/pool/main/u/usplash/usplash*deb
        mv usplash*.deb $BASEDIR/FinalCD/pool/main/u/usplash/
fi

echo "Creating apt package list.."
cd $BASEDIR/FinalCD

apt-ftparchive -c $SOURCEDIR/apt.conf generate $SOURCEDIR/apt-ftparchive-deb.conf
apt-ftparchive -c $SOURCEDIR/apt.conf generate $SOURCEDIR/apt-ftparchive-udeb.conf
if [ ! -z $EXTRASDIR ]; then
        if [ ! -f $BASEDIR/FinalCD/dists/intrepid/main/binary-i386/Release ]; then                cat $BASEDIR/FinalCD/dists/intrepid/main/binary-i386/Release | sed 's/Component: main/Component: extras/' > $BASEDIR/FinalCD/dists/intrepid/extras/binary-i386/Release
        fi
        ## Henrique Haas - check if exsits extras dists directory     
        if [ ! -d $BASEDIR/FinalCD/dists/intrepid/extras/binary-i386 ]; then
                mkdir -p $BASEDIR/FinalCD/dists/intrepid/extras/binary-i386
        fi
        apt-ftparchive -c $SOURCEDIR/apt.conf generate $SOURCEDIR/apt-ftparchive-extras.conf
fi




# Kill the existing release file
rm -f $BASEDIR/FinalCD/dists/intrepid/Release*
cd $BASEDIR/FinalCD
apt-ftparchive -c $SOURCEDIR/apt.conf release dists/intrepid/ > $BASEDIR/FinalCD/dists/intrepid/Release

echo "$GPGKEYPHRASE" | gpg --default-key "$MYGPGKEY" --passphrase-fd 0 --output $BASEDIR/FinalCD/dists/intrepid/Release.gpg -ba $BASEDIR/FinalCD/dists/intrepid/Release
echo "OK"

cd $BASEDIR/FinalCD
echo -n "Updating md5 checksums.. "
chmod 666 md5sum.txt
rm -f md5sum.txt
find . -type f -print0 | xargs -0 md5sum > md5sum.txt
echo "OK"

cd $BASEDIR/FinalCD
echo "Creating and ISO image..."
mkisofs -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -J -hide-rr-moved -o $BASEDIR/$CDNAME -R $BASEDIR/FinalCD/

echo "CD Available in $BASEDIR/$CDNAME"
echo "You can now remove all files in:"
echo " - $BASEDIR/FinalCD"

# Unmount the old CD
umount $CDSOURCEDIR 

You will hopefully now have a ~3GB iso image in /opt/LMCE_DVD. Copy/burn as required and install to a virtual or physical machine.