Building LinuxMCE 0804

With this tutorial I'll try to summarize the step involved in building the LinuxMCE CDs and DVDs together with some changes that I've done when switching from Kubuntu 0704 to Kubuntu 0710.

Requirements

Requirements:

• Powerful enough system to do the compilation and building of packages as is a cpu intensive task and it can take quite a long time.
• Lot of disk space, at least 40G hard drive.
• A working internet connection as the build and preparation process will need to access get some files from the public domain.

First Steps:

• Install Kubuntu 0804. I use the desktop installation CD and i would recommend to do the same as this steps will consider this kind of installation performed. Of course an Ubuntu Server or Alternate CD will work too but you may need to install some build dependencies / runtime libraries that I have by default installed on Kubuntu 0804.
• Install some subversion so you can checkout a copy of the build script from LinuxMCE's svn.

apt-get install subversion

Build Scripts

The next step after having 0804 installed is to get a copy of the build script:

sudo su
cd $HOME
svn co http://svn.linuxmce.com/pluto/trunk/src/Ubuntu_Helpers_NoHardcode/
mv Ubuntu_Helpers_NoHardcode/ /usr/local/lmce-build

I will try to explain them quickly so you'll know what each of them does.

build.sh

This is the main script and calls all the other script of the build process to perform simple task.

loop-build.sh

This is a script used on our build machines and it does the next steps in a infinite loop:

1. Calls build.sh script to performa a build.
2. Stores the build output in a ftp viewable directory.
3. Sends mail to developers on fail and on success.

Scripts used for preaparing the build system

prepare-scripts/install-build-pkgs.sh

This script will download and install all the build dependencies needed to create the linuxmce packages.

prepare-scripts/cd1-prepare.sh

Download the ubuntu packages that will be copied on LinuxMCE CD1

prepare-scripts/cd2-packages.sh

Download the ubuntu packages that will be copied on LinuxMCE CD2

prepare-scripts/create-diskless-debootstrap.sh

Creates a debootsrap image used later as a base for creating the diskless image.

prepare-script/import-external-files.sh

Downloads the linuxmce media file : skins, demo videos ..

prepare-script/preseed-build-pkgs.sh

It was used to preseed some src packages like lirc-module so that they will build in a non interactive mode by the build process. It is not used for 0804.

Scripts used for building CDs

build-scripts/checkout-svn.sh

Gets a updated copy of the source tree from svn and stores it in /var/lmce-build/svn/

build-script/import-win32bins.sh

Copies the latest windows binaries from the linuxmce ftp

buils-script/build-replacements.sh

Creates a set of non-linuxmce-specific deb packages which don't usually come with Ubuntu but are needed by our applications. It may also build some packages that are already in Ubuntu but are patched and needs to be repacked. The sources for this debs are located in trunk/ubuntu directory of the svn this means /var/lmce-build/svn/trunk/ubuntu on your local machine (after the svn checkout). Moust of this sources are in a deb-src format. The created debs will be stored in /var/lmce-build/replacements/ directory.

build-script/build-makerelease.sh

Build a c++ linuxmce application and it's dependencies from the linuxmce svn. This application, called MakeRelease it's used to build and pack all the linuxmce packages. The output files will be stored in /var/lmce-build/MakeRelease/ when the script finishes.

build-script/import-database.sh

This script takes the latest version of the LinuxMCE MySQL databases from the LinuxMCE server. The information from this databases is used by MakeRelease to build and compile the software. The databases will also be used to create some sqldumps that will be packaged on the LinuxMCE CDs and DVSs that are created.

build-scripts/build-maindebs.sh

Runs MakeRelease and based on the information from the database compiles and packs the linuxmce packages using the sources stored in /var/lmce-build/svn. The created debs will be stored in /var/lmce-build/out/tmp directory.

build-scripts/get-closed-source-debs.sh

Download the latest pluto debs which are build on a remote server. This debs are compiled from source which are object of some Pluto NDAs and cannot be made public (ex: pluto-zwave).

build-script/create-repo.sh

After all the needed packages are created / downloaded, this script copies them from their output directories and stores them in /var/www/ directory. It also creates a Packages.gz file so you can use that directory as a repository which you can add in sources.list.

build-script/cd1-build.sh

Creates LinuxMCE CD1. This cd will contain moust of the linuxmce specific debs but also some Ubuntu debs which are needed for by LinuxMCE application and are not installed by default in Kubuntu. The linuxmce debs are copied from /var/www' and the extra ubuntu debs are fetched from /var/lmce-build/cd1-packages/ directory.

build-scripts/cd2-build.sh

Creates LinuxMCE CD2. This cd contains only few linuxmce debs and a lot of ubuntu debs which are not installed by default on a Kubuntu system but are needed douring the install process. The Ubuntu debs are copied from /var/lmce-build/cd2-packages/ folder.

Scripts used for building DVDs

vmware-install/install_in_vmware.sh

This script uses the LinuxMCE CD1&2 to automatically install LinuxMCE on a clean Kubuntu system. All the installation takes place in vmware virtual machine and at the end of the installation the script will create a tar.gz image of the vmware disk directory structure. This tar.gz image will be used for creating the dvd image.

build-dvd/build-cd.sh

This script takes the tar.gz image created by the install_in_vmware.sh script and generates the bootable dvd image. For generating the bootable dvd image this script also needs

• The content of a striped squashfs image found on the Kubuntu CD : /var/lmce-build/lite-installer/ubuntu-squashfs-base.tgz
• The content of a Kubuntu CD without the squashfs file: /var/lmce-build/lite-installer/ubutu-livecd.tgz

First Time Build

At this point you should already have the Kubuntu system installed and some basic knowledge of what each build script is doing. I recommend running the each script by hand for the first time and not use the build.sh or prepare.sh scripts. It will be a lot easer to debug problems this way.

Runing the prepare script

This set of scripts should be run only once to prepare a Kubuntu system to act as a build server. The order is not so important for them but I recomend to stick with it. You'll need to run all the script as root as they will need write access in several directories.

• The next script will create the /etc/lmce-build configuration directory and copy all the default configuration files there. The default files are stored in the conf-files folder and spited other folders depending on release and architecture. I recommend you check the install.sh script to see how it does this before running it.

./install.sh

• Install the build dependencies. Takes aprox. 30min depending on you internet connection speed. The packages that get installed are listed in /etc/lmce-build/build-packages:

./prepare-scripts/install-build-pkgs.sh

• Download Ubuntu packages needed by CD1&2. The downloded packages are liste in files /etc/lmce-build/cd1-packages and /etc/lmce-build/cd2-packages

./prepare-scripts/cd1-prepare.sh
./prepare-scripts/cd2-prepare.sh

• Download Ubuntu packages found on a default Kubuntu CD. This ones are used when installing the system in vmware:

mkdir -p /var/lmce-build/kubuntu-cd
pushd /var/lmce-build/kubuntu-cd
aptitude download `cat /etc/lmce-build/cdK-packages`
popd

• Run the other prepare scripts :

./prepare-scripts/create-diskless-debootstrap.sh
./prepare-scripts/import-external-files.sh
./prepare-scripts/preseed-build-pkgs.sh

Preparing for DVD Creation

If you want only to be able to create linuxmce packages and create the standard CDs you can skip this steps. This steps are only needed for creating the DVD and involve some advanced skills as you must manually do most of the things here.

• Create the /var/lmce-build/lite-installer/ubuntu-livecd.tgz and /var/lmce-build/lite-installer/ubuntu-squashfs-base.tgz archives that are used as a base of the LinuxMCE DVD:
  • ubuntu-livecd.tgz: This is the simple one. It contains the all files from the kubnuntu dvd except for the squashfs image file. The steps to create this file are:
    1. Mount the kubuntu cd
    2. Copy the contents of the cd in a directory
    3. Remove the squashfs image from the directory (casper/filesystem.squashfs)
    4. Create a the tar.gz
  • filesystem.squashfs: You'll need to use the casper/filesystem.squashfs which you eliminate in creation of the previous tar.gz:
    1. Mount the squashfs image in a directory
    2. Copy the entire chroot content into another directory so you can modify it
    3. Chroot in the new directory and uinstall all the x11/kde related packages
    4. Exit the chroot and targz what is left of the unpacked squashfs image
• Create the vmware virtual machine and install vmware player:
  • The build scripts expect a vmware virtual machine that can be started with vmware player. It's important for the disk image of the vmware machine to be in a 'flat' format, meaning that will not be splited in 2G files and all the disk space is allocated from the start, i use 15G virtual disks. Second, make sure that the network is set on nat. It's important the the virtual machine to be named 'Kubuntu'.
  • Install a Kubuntu CD on the virtual machine.
  • Copy the ssh public key (/etc/lmce-build/builder.key.pub) in the virtual machine and make sure that you can ssh using the private key from you system (/etc/lmce-build/builder.key) into root on you vmware machine.
  • Add an entry in /etc/hosts with hostname 'vmware' and ip of your vmware machine
  • Leave virtual machine powered up and shutdown vmware. This way the next time you start the virtual machine you will be able to login quickly with ssh on it.
  • Copy all the files of the virtual machine in /var/Kubuntu