Difference between revisions of "Installing LinuxMCE on Virtual Machine via KVM"

From LinuxMCE
Jump to: navigation, search
m
(→‎KVM: To boldly go where (almost) no man has gone before)
 
(45 intermediate revisions by 5 users not shown)
Line 1: Line 1:
−
[[Category: Virtualization]]
+
[[Category:Virtualization|KVM]]
−
== '''KVM: To badly go where (almost) no man has gone before''' ==
+
  
 +
{{versioninfo|810Status=relevant|810UpdatedDate=13th December 2010|810UpdatedBy=daballiemo}}
  
−
In this overview I will try to follow the installation path to get LCME 8.10 into a virtual machine which uses PCI(-E) devices from the host. Normally the option to install on a dedicated computer is the best, but as I want to host several systems, including LCME, websites etc and the ease of testing out new stuff combined with the fact that I want to keep it as green as possible lead me to the conclusion that virtualization was the way to go. Over the last 2 years I've been building systems that have failed in doing what I wanted (XEN and KVM based), but with the latest state of soft- and hardware I'll hope I'll have a better change. The below is my queste for the wanted setup up, step by step.
+
 
 +
== '''KVM: To boldly go where (almost) no man has gone before''' ==
 +
 
 +
[[Installing LinuxMCE on VM via KVM on Ubuntu 10.10 Server]]
 +
 
 +
In this overview I will try to follow the installation path to get LCME 8.10 into a virtual machine which uses PCI(-E) devices from the host. Normally the option to install on a dedicated computer is the best, but as I want to host several systems, including LCME, websites etc and the ease of testing out new stuff combined with the fact that I want to keep it as green as possible lead me to the conclusion that virtualization was the way to go. Over the last 2 years I've been building systems that have failed in doing what I wanted (XEN and KVM based), but with the latest state of soft- and hardware I'll hope I'll have a better chance. The below is my quest for the wanted setup up, step by step.
  
 
Virtualization the way I want to do it can only be done with specific hardware/firmware which uses VT-d, a virtualization PCI Passthrough technique embedded in the bios.
 
Virtualization the way I want to do it can only be done with specific hardware/firmware which uses VT-d, a virtualization PCI Passthrough technique embedded in the bios.
Line 14: Line 19:
 
* 1 onboard GbE Nic
 
* 1 onboard GbE Nic
 
* 1 PCI-E GbE Nic
 
* 1 PCI-E GbE Nic
−
* 1 Hauppauge PVR-150 (for the moment)
+
* 1 Hauppauge PVR-150
 +
* 1 Hauppauge PVR-500
  
  
−
  han@obelix:~$ lspci                                                            
+
  root@obelix:~# lspci
−
  00:00.0 Host bridge: Intel Corporation 82Q35 Express DRAM Controller (rev 02)  
+
  00:00.0 Host bridge: Intel Corporation 82Q35 Express DRAM Controller (rev 02)
 
  00:02.0 VGA compatible controller: Intel Corporation 82Q35 Express Integrated Graphics Controller (rev 02)
 
  00:02.0 VGA compatible controller: Intel Corporation 82Q35 Express Integrated Graphics Controller (rev 02)
−
  00:03.0 Communication controller: Intel Corporation 82Q35 Express MEI Controller (rev 02)                
+
  00:03.0 Communication controller: Intel Corporation 82Q35 Express MEI Controller (rev 02)
−
  00:03.2 IDE interface: Intel Corporation 82Q35 Express PT IDER Controller (rev 02)                      
+
  00:03.2 IDE interface: Intel Corporation 82Q35 Express PT IDER Controller (rev 02)
−
  00:03.3 Serial controller: Intel Corporation 82Q35 Express Serial KT Controller (rev 02)                
+
  00:03.3 Serial controller: Intel Corporation 82Q35 Express Serial KT Controller (rev 02)
−
  00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network Connection (rev 02)            
+
  00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network Connection (rev 02)
−
  00:1a.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 02)          
+
  00:1a.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 02)
−
  00:1a.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 02)          
+
  00:1a.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 02)
−
  00:1a.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6 (rev 02)          
+
  00:1a.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6 (rev 02)
 
  00:1a.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 02)
 
  00:1a.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 02)
 
  00:1b.0 Audio device: Intel Corporation 82801I (ICH9 Family) HD Audio Controller (rev 02)
 
  00:1b.0 Audio device: Intel Corporation 82801I (ICH9 Family) HD Audio Controller (rev 02)
 
  00:1c.0 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 1 (rev 02)
 
  00:1c.0 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 1 (rev 02)
−
00:1c.4 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 5 (rev 02)
 
 
  00:1d.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 02)
 
  00:1d.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 02)
 
  00:1d.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 02)
 
  00:1d.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 02)
Line 41: Line 46:
 
  00:1f.5 IDE interface: Intel Corporation 82801I (ICH9 Family) 2 port SATA IDE Controller (rev 02)
 
  00:1f.5 IDE interface: Intel Corporation 82801I (ICH9 Family) 2 port SATA IDE Controller (rev 02)
 
  00:1f.6 Signal processing controller: Intel Corporation 82801I (ICH9 Family) Thermal Subsystem (rev 02)
 
  00:1f.6 Signal processing controller: Intel Corporation 82801I (ICH9 Family) Thermal Subsystem (rev 02)
−
  01:00.0 IDE interface: JMicron Technologies, Inc. JMB368 IDE controller
+
  01:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)
−
02:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)
+
  02:00.0 Multimedia video controller: Internext Compression Inc iTVC16 (CX23416) MPEG-2 Encoder (rev 01)
−
  03:01.0 Multimedia video controller: Internext Compression Inc iTVC16 (CX23416) MPEG-2 Encoder (rev 01)
+
02:01.0 PCI bridge: Hint Corp HB6 Universal PCI-PCI bridge (non-transparent mode) (rev 11)
−
  03:02.0 FireWire (IEEE 1394): Agere Systems FW323 (rev 70)
+
  03:08.0 Multimedia video controller: Internext Compression Inc iTVC16 (CX23416) MPEG-2 Encoder (rev 01)
 +
03:09.0 Multimedia video controller: Internext Compression Inc iTVC16 (CX23416) MPEG-2 Encoder (rev 01)
 +
root@obelix:~#
  
−
Step 1) Install Ubuntu 9.04 server AMD 64 (should be easy enough if you are able to get LMCE up and running :-) )
+
=== Install Ubuntu 9.10 server ===
 +
Step 1) Install Ubuntu 9.10 server AMD 64 (should be easy enough if you are able to get LMCE up and running :-) )
  
  
 +
=== Update ===
 
Step 2) Make the system current:
 
Step 2) Make the system current:
  
Line 55: Line 64:
  
  
 +
=== Install management tools ===
 
Step 3) Add some management tools:
 
Step 3) Add some management tools:
  
−
sudo wget http://prdownloads.sourceforge.net/webadmin/webmin_1.480_all.deb
+
==== Webmin ====
−
sudo apt-get install libnet-ssleay-perl libauthen-pam-perl libio-pty-perl libmd5-perl
+
−
sudo dpkg -i webmin_1.480_all.deb
+
−
sudo apt-get install mc
+
−
 
+
 
I use webmin to give my server a fixed ip address and perform other administrative tasks.
 
I use webmin to give my server a fixed ip address and perform other administrative tasks.
−
MC (Midnight Commander) is a nice tool to move files around the server.
 
  
 +
apt-get install libnet-ssleay-perl libauthen-pam-perl libio-pty-perl libmd5-perl
 +
wget http://downloads.sourceforge.net/project/webadmin/webmin/1.510/webmin_1.510-2_all.deb?r=http%3A%2F%2Fwww.webmin.com%2Fdownload.html&ts=1283088165&use_mirror=netcologne
 +
dpkg -i webmin_1.510-2_all.deb
  
−
Step 4) To be able to do remote monitoring of virtual machines, install virt-manager on your remote machine. If your are using Jaunty on your remote system, install ssh-askpass as per [[https://bugs.launchpad.net/ubuntu/+source/virt-manager/+bug/366057]]
 
  
−
  sudo apt-get install ssh-askpass
+
==== MidNight Commander ====
 +
I use MidNight Commander for file stuff
 +
 +
  apt-get install mc
  
−
or use ssh keys
 
  
 +
=== Getting the right kernel and parameters ===
 +
Step 4) Getting the right kernel and parameters
  
−
Step 5) Get the up to date kernel sources & unpack them
+
Install the required tools and packages, so open a terminal and type the following,
  
−
  sudo apt-get install linux-source
+
  apt-get install fakeroot kernel-wedge build-essential makedumpfile kernel-package libncurses5 libncurses5-dev
−
sudo tar -jxvf linux-2.6.31.tar.bz2 -C /usr/src
+
  
 +
Get the right kernel, at the time of writing 2.6.32.21 was the best kernel source.
  
−
Step 6) Install dependencies for make menuconfig.
+
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.32.21.tar.bz2
 +
tar -jxvf linux-2.6.32.21.tar.bz2 -C /usr/src
 +
 +
Go to the unpack directory
  
−
  sudo apt-get install gcc ncurses* kernel-package fakeroot
+
  cd /usr/src/linux-2.6.32.21
  
  
−
Step 7) Configure the new kernel. (In this case for Karmic 2.6.31)
+
It is a good idea to start with the same .config as the currently running kernel, so type the following,
  
−
  cd /usr/src/linux-2.6.31 # cd into directory
+
  cp -vi /boot/config-`uname -r` .config
  
−
# this will make sure you start of with the right settings from the previous kernel
+
Now we are ready to customize the build and kernel options.
−
cp /boot/config-2.6.31-14-server .config
+
  
−
# here you are being asked for all of the changes and if you want to implement them, my advise: make a note and say no, in the next step it can all be adjusted
+
  make menuconfig
−
  make oldconfig
+
  
−
# here you are able to make the wanted changes, for VT-D to work you will need to enable at least:
+
  * set "Bus options (PCI etc.)" -> "Support for DMA Remapping Devices" to "*"
−
sudo make menuconfig
+
  * set "Bus options (PCI etc.)" -> "Enable DMA Remapping Devices" to "*"
 +
  * set "Bus options (PCI etc.)" -> "PCI Stub driver" to "*"
 +
  This line broke my setup so I took it out * optional setting: set "Bus options (PCI etc.)" -> "Support for Interrupt Remapping" to "*"
  
−
    * set "Bus options (PCI etc.)" -> "Support for DMA Remapping Devices" to "*"
+
Once you have finished, save and exit. It is now time to compile. However to speed up the build if you have a dual/triple/quad core processor type,
−
    * set "Bus options (PCI etc.)" -> "Enable DMA Remapping Devices" to "*"
+
−
    * set "Bus options (PCI etc.)" -> "PCI Stub driver" to "*"
+
  
−
  (This line broke my setup so I took it out * ''optional setting: set "Bus options (PCI etc.)" -> "Support for Interrupt Remapping" to "*"'')
+
  export CONCURRENCY_LEVEL=5
  
 +
The general rule is 1 + the number of processor cores.
  
 +
==== Building the new kernel ====
 +
Step 5) Build the new kernel.
  
−
Step 8) Build the new kernel.
+
  make-kpkg clean
−
 
+
  fakeroot make-kpkg --initrd --append-to-version=-iommu kernel-image kernel-headers
−
  sudo make-kpkg clean
+
−
  sudo fakeroot make-kpkg --initrd --append-to-version=-custom kernel_image kernel_headers
+
  
 
You can grab a coffee, make a hot meal, have a sleep ;-)
 
You can grab a coffee, make a hot meal, have a sleep ;-)
  
 +
==== Instaling the new kernel ====
 +
Step 6) Install The New Kernel
  
−
Step 9) Install The New Kernel
+
After the successful kernel build, you can find two .deb packages in the /usr/src directory.
−
 
+
−
After the successful kernel build, you can find two .deb packages in the /usr/bin directory.
+
 
  cd ..  
 
  cd ..  
 
  ls -l linux*
 
  ls -l linux*
  
 
Install them:
 
Install them:
−
  sudo dpkg -i linux-image-2.6.30.4-custom_2.6.30.4-custom-10.00.Custom_amd64.deb
+
  dpkg -i linux-image-2.6.33-custom_2.6.33-custom-10.00.Custom_amd64.deb
−
  sudo  sudo dpkg -i linux-headers-2.6.30.4-custom_2.6.30.4-custom-10.00.Custom_amd64.deb
+
  dpkg -i linux-headers-2.6.33-custom_2.6.33-custom-10.00.Custom_amd64.deb
 +
 
 +
=== Booting with the new kernel ===
 +
Step 7) Booting with the new kernel
 +
 
 +
Add the options to enable IOMMU in the kernel commandline
 +
 
 +
edit /etc/default/grub
 +
 
 +
new: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash intel_iommu=on,igfx_off,pass-through"
 +
old: #GRUB_CMDLINE_LINUX_DEFAULT="quiet"
 +
 
 +
update the booting menu
  
 +
update-grub
  
−
Step 10) Check that menu.list has been updated with your new kernel
+
and reboot
  
−
cat /boot/grub/menu.lst
 
 
  sudo reboot
 
  sudo reboot
  
−
Step 11) Deinstall networkmanager
+
 
 +
=== Deinstall networkmanager ===
 +
Step 8) Deinstall networkmanager
  
 
  aptitude remove network-manager
 
  aptitude remove network-manager
  
−
Step 12) Setup your network, change /etc/network/interfaces to look like this
+
 
 +
=== Networking setup ===
 +
Step 9) Setup your network, change '''/etc/network/interfaces''' to look like this
  
 
  # This file describes the network interfaces available on your system
 
  # This file describes the network interfaces available on your system
Line 163: Line 192:
 
With the above the host will use dcerouter as its way into the internet and thus the protection offered by the firewall. The host has no direct access to the 10.0.0.0 network.
 
With the above the host will use dcerouter as its way into the internet and thus the protection offered by the firewall. The host has no direct access to the 10.0.0.0 network.
  
−
 
+
=== Install LinuxMCE ===
−
Step 12) Install Linuxmce
+
Step 10) Install LinuxMCE
  
 
Using virt-manager you can now install LinuxMCE. As we are all experts on that I will only give away my xml config file (should be enough ;-))
 
Using virt-manager you can now install LinuxMCE. As we are all experts on that I will only give away my xml config file (should be enough ;-))
  
−
   <name>DCERouter</name>                            
+
<domain type='kvm'>
−
   <uuid>f1a8bb7e-9e07-4f72-1337-d04429d0d2de</uuid>  
+
   <name>DCERouter</name>
−
   <memory>2097152</memory>                          
+
   <uuid>xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx</uuid>
−
   <currentMemory>2097152</currentMemory>            
+
   <memory>2097152</memory>
−
   <vcpu>2</vcpu>                                    
+
   <currentMemory>2097152</currentMemory>
−
   <os>                                              
+
   <vcpu>2</vcpu>
−
     <type arch='i686' machine='pc'>hvm</type>        
+
   <os>
−
     <boot dev='hd'/>                                
+
     <type arch='x86_64' machine='pc-0.11'>hvm</type>
−
   </os>                                              
+
     <boot dev='hd'/>
−
   <features>                                        
+
   </os>
−
     <acpi/>                                          
+
   <features>
−
     <apic/>                                          
+
     <acpi/>
−
     <pae/>                                          
+
     <apic/>
−
   </features>                                        
+
     <pae/>
 +
   </features>
 
   <clock offset='utc'/>
 
   <clock offset='utc'/>
 
   <on_poweroff>destroy</on_poweroff>
 
   <on_poweroff>destroy</on_poweroff>
Line 188: Line 218:
 
   <devices>
 
   <devices>
 
     <emulator>/usr/bin/kvm</emulator>
 
     <emulator>/usr/bin/kvm</emulator>
−
     <disk type='file' device='disk'>
+
     <disk type='block' device='disk'>
−
       <source file='/storage2/100-virtualmachines/DCERouter.img'/>
+
       <source dev='/dev/sdc'/>
 
       <target dev='hda' bus='ide'/>
 
       <target dev='hda' bus='ide'/>
 
     </disk>
 
     </disk>
Line 197: Line 227:
 
     </disk>
 
     </disk>
 
     <interface type='bridge'>
 
     <interface type='bridge'>
−
       <mac address='54:52:00:0a:09:ee'/>
+
       <mac address='xx:xx:xx:xx:xx:xx'/>
 
       <source bridge='bridge_ext'/>
 
       <source bridge='bridge_ext'/>
−
      <target dev='vnet0'/>
 
−
      <model type='virtio'/>
 
 
     </interface>
 
     </interface>
 
     <interface type='bridge'>
 
     <interface type='bridge'>
−
       <mac address='54:52:00:14:87:a4'/>
+
       <mac address='xx:xx:xx:xx:xx:xx'/>
 
       <source bridge='bridge_int'/>
 
       <source bridge='bridge_int'/>
−
      <target dev='vnet1'/>
 
 
       <model type='virtio'/>
 
       <model type='virtio'/>
 
     </interface>
 
     </interface>
 
     <serial type='pty'>
 
     <serial type='pty'>
−
      <source path='/dev/pts/1'/>
 
 
       <target port='0'/>
 
       <target port='0'/>
 
     </serial>
 
     </serial>
−
     <console type='pty' tty='/dev/pts/1'>
+
     <console type='pty'>
−
      <source path='/dev/pts/1'/>
+
 
       <target port='0'/>
 
       <target port='0'/>
 
     </console>
 
     </console>
 
     <input type='tablet' bus='usb'/>
 
     <input type='tablet' bus='usb'/>
 
     <input type='mouse' bus='ps2'/>
 
     <input type='mouse' bus='ps2'/>
−
     <graphics type='vnc' port='5900' autoport='yes' keymap='en-us'/>
+
     <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>
−
     <sound model='es1370'/>
+
     <video>
−
     <hostdev mode='subsystem' type='pci' managed='no'>
+
      <model type='cirrus' vram='9216' heads='1'/>
 +
    </video>
 +
     <hostdev mode='subsystem' type='pci' managed='yes'>
 +
      <source>
 +
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
 +
      </source>
 +
    </hostdev>
 +
    <hostdev mode='subsystem' type='pci' managed='yes'>
 +
      <source>
 +
        <address domain='0x0000' bus='0x03' slot='0x08' function='0x0'/>
 +
      </source>
 +
    </hostdev>
 +
    <hostdev mode='subsystem' type='pci' managed='yes'>
 +
      <source>
 +
        <address domain='0x0000' bus='0x03' slot='0x09' function='0x0'/>
 +
      </source>
 +
    </hostdev>
 +
    <hostdev mode='subsystem' type='usb' managed='yes'>
 +
      <source>
 +
        <vendor id='0x0403'/>
 +
        <product id='0xf2d0'/>
 +
      </source>
 +
    </hostdev>
 +
    <hostdev mode='subsystem' type='pci' managed='yes'>
 
       <source>
 
       <source>
−
         <address domain='0x0000' bus='0x03' slot='0x01' function='0x0'/>
+
         <address domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
 
       </source>
 
       </source>
 
     </hostdev>
 
     </hostdev>
 
   </devices>
 
   </devices>
 
  </domain>
 
  </domain>
−
</domstatus>
 
  
−
Step 13) Enjoying building a test enviroment, inline upgrade etc.
 
  
−
Step 14) I chose to put my media on the host via a Samba share which will be detected as a NAS. This will simplify things when performing upgrades and test. Because I am using LVM also expansion is easy ;-)
+
=== The rest ===
 +
Step 11) Enjoying building a test enviroment, inline upgrade etc.
  
 +
Step 12) I chose to put my media on the host via a Samba share which will be detected as a NAS. This will simplify things when performing upgrades and test. Because I am using LVM also expansion is easy ;-)
  
  
 +
[[Image:Lmce_kvm.png]]
  
  
  
 
'''
 
'''
 +
 +
{{p}}
 +
 
== Hints/Known Issues/Working Hardware ==
 
== Hints/Known Issues/Working Hardware ==
 
'''
 
'''
Line 251: Line 303:
  
 
'''Networking'''<br>
 
'''Networking'''<br>
−
- Step 11) is not needed if the physical network devices are configured as 'auto ethx inet manual'<br>
+
- Step 11) is not needed if the physical network devices are configured as 'auto ethx \ iface ethx inet manual'<br>
 
- The virtual LMCE runs fine with only one NIC provided to it. The setup automatically configures a sub-NIC which works well. I would suggest to have the host provide the internet-access, so vLMCE can't mess it up at the installation.
 
- The virtual LMCE runs fine with only one NIC provided to it. The setup automatically configures a sub-NIC which works well. I would suggest to have the host provide the internet-access, so vLMCE can't mess it up at the installation.
  
Line 284: Line 336:
  
 
- Please note that each change in the .xml-files is activated only after rebooting the libvirtd. You can do this manually by killing and restarting the process "libvirtd" or by a host reboot.
 
- Please note that each change in the .xml-files is activated only after rebooting the libvirtd. You can do this manually by killing and restarting the process "libvirtd" or by a host reboot.
 +
 +
'''Problems with PVR-150 not responding'''
 +
 +
During usage it turned out that sometimes recordings failed having a total size of 0 B. Live TV has not worked stable at all. Checking the mythbackend logs shows that /dev/video0 was not responding as expected.
 +
Furthermore I detected DMA errors in the log of the host.
 +
 +
I am now testing the commandline option iommu=on,igfx_off,pass-through, where pass-through should be able to iron out the DMA errors.
 +
 +
Because the behaviour was not stable I also installed the mercurial ivtv drivers (within the VM ofcourse!!!!):
 +
 +
sudo apt-get install mercurial
 +
cd /usr/src
 +
sudo hg clone http://linuxtv.org/hg/v4l-dvb
 +
cd v4l-dvb
 +
sudo make
 +
sudo make install
 +
sudo make unload
 +
sudo make load
 +
 +
You will have to rebuild after a kernel change.
 +
 +
'''Clicking sound on Audio device: Intel Corporation 82801I (ICH9 Family) HD Audio Controller (rev 02)'''
 +
 +
When passing-through my audio card I noticed hick ups in the MP3's and a message in kern.log;
 +
 +
hda-intel: IRQ timing workaround is activated for card #0. Suggest a bigger bdl_pos_adj.
 +
 +
edit alsa-base.conf;
 +
 +
vim /etc/modprobe.dalsa-base.conf
 +
 +
add the following line at the bottom;
 +
 +
options snd-hda-intel model=3stack-dig
 +
options snd-hda-intel enable_msi=1
 +
options snd-hda-intel single_cmd=1
 +
 +
and reboot (reload alsa does not seem to have the full effect)
 +
 +
Tip from [https://bugs.launchpad.net/ubuntu/+source/alsa-driver/+bug/274424 Bugs Launchpad]

Latest revision as of 04:59, 24 March 2016


Version Status Date Updated Updated By
710 Unknown N/A N/A
810 relevant 13th December 2010 daballiemo
1004 Unknown N/A N/A
1204 Unknown N/A N/A
1404 Unknown N/A N/A
Usage Information


KVM: To boldly go where (almost) no man has gone before

Installing LinuxMCE on VM via KVM on Ubuntu 10.10 Server

In this overview I will try to follow the installation path to get LCME 8.10 into a virtual machine which uses PCI(-E) devices from the host. Normally the option to install on a dedicated computer is the best, but as I want to host several systems, including LCME, websites etc and the ease of testing out new stuff combined with the fact that I want to keep it as green as possible lead me to the conclusion that virtualization was the way to go. Over the last 2 years I've been building systems that have failed in doing what I wanted (XEN and KVM based), but with the latest state of soft- and hardware I'll hope I'll have a better chance. The below is my quest for the wanted setup up, step by step.

Virtualization the way I want to do it can only be done with specific hardware/firmware which uses VT-d, a virtualization PCI Passthrough technique embedded in the bios.

So I will state my machine details first:

  • Motherboard: ASUS P5E VM DO with bios 10.1
  • CPU: Q6600
  • 6 Gb DDR-2 Memory
  • 1 onboard GbE Nic
  • 1 PCI-E GbE Nic
  • 1 Hauppauge PVR-150
  • 1 Hauppauge PVR-500


root@obelix:~# lspci
00:00.0 Host bridge: Intel Corporation 82Q35 Express DRAM Controller (rev 02)
00:02.0 VGA compatible controller: Intel Corporation 82Q35 Express Integrated Graphics Controller (rev 02)
00:03.0 Communication controller: Intel Corporation 82Q35 Express MEI Controller (rev 02)
00:03.2 IDE interface: Intel Corporation 82Q35 Express PT IDER Controller (rev 02)
00:03.3 Serial controller: Intel Corporation 82Q35 Express Serial KT Controller (rev 02)
00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network Connection (rev 02)
00:1a.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 02)
00:1a.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 02)
00:1a.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6 (rev 02)
00:1a.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 02)
00:1b.0 Audio device: Intel Corporation 82801I (ICH9 Family) HD Audio Controller (rev 02)
00:1c.0 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 1 (rev 02)
00:1d.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 02)
00:1d.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 02)
00:1d.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 02)
00:1d.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)
00:1f.0 ISA bridge: Intel Corporation 82801IO (ICH9DO) LPC Interface Controller (rev 02)
00:1f.2 IDE interface: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 4 port SATA IDE Controller (rev 02)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02)
00:1f.5 IDE interface: Intel Corporation 82801I (ICH9 Family) 2 port SATA IDE Controller (rev 02)
00:1f.6 Signal processing controller: Intel Corporation 82801I (ICH9 Family) Thermal Subsystem (rev 02)
01:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)
02:00.0 Multimedia video controller: Internext Compression Inc iTVC16 (CX23416) MPEG-2 Encoder (rev 01)
02:01.0 PCI bridge: Hint Corp HB6 Universal PCI-PCI bridge (non-transparent mode) (rev 11)
03:08.0 Multimedia video controller: Internext Compression Inc iTVC16 (CX23416) MPEG-2 Encoder (rev 01)
03:09.0 Multimedia video controller: Internext Compression Inc iTVC16 (CX23416) MPEG-2 Encoder (rev 01)
root@obelix:~#

Install Ubuntu 9.10 server

Step 1) Install Ubuntu 9.10 server AMD 64 (should be easy enough if you are able to get LMCE up and running :-) )


Update

Step 2) Make the system current: 

 sudo apt-get update
 sudo apt-get upgrade


Install management tools

Step 3) Add some management tools:

Webmin

I use webmin to give my server a fixed ip address and perform other administrative tasks. 

apt-get install libnet-ssleay-perl libauthen-pam-perl libio-pty-perl libmd5-perl
wget http://downloads.sourceforge.net/project/webadmin/webmin/1.510/webmin_1.510-2_all.deb?r=http%3A%2F%2Fwww.webmin.com%2Fdownload.html&ts=1283088165&use_mirror=netcologne
dpkg -i webmin_1.510-2_all.deb


MidNight Commander

I use MidNight Commander for file stuff 

apt-get install mc


Getting the right kernel and parameters

Step 4) Getting the right kernel and parameters

Install the required tools and packages, so open a terminal and type the following, 

apt-get install fakeroot kernel-wedge build-essential makedumpfile kernel-package libncurses5 libncurses5-dev

Get the right kernel, at the time of writing 2.6.32.21 was the best kernel source. 

wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.32.21.tar.bz2
tar -jxvf linux-2.6.32.21.tar.bz2 -C /usr/src

Go to the unpack directory 

cd /usr/src/linux-2.6.32.21


It is a good idea to start with the same .config as the currently running kernel, so type the following, 

cp -vi /boot/config-`uname -r` .config

Now we are ready to customize the build and kernel options. 

make menuconfig

  * set "Bus options (PCI etc.)" -> "Support for DMA Remapping Devices" to "*"
  * set "Bus options (PCI etc.)" -> "Enable DMA Remapping Devices" to "*"
  * set "Bus options (PCI etc.)" -> "PCI Stub driver" to "*"
  This line broke my setup so I took it out * optional setting: set "Bus options (PCI etc.)" -> "Support for Interrupt Remapping" to "*"

Once you have finished, save and exit. It is now time to compile. However to speed up the build if you have a dual/triple/quad core processor type, 

export CONCURRENCY_LEVEL=5

The general rule is 1 + the number of processor cores.

Building the new kernel

Step 5) Build the new kernel. 

make-kpkg clean
fakeroot make-kpkg --initrd --append-to-version=-iommu kernel-image kernel-headers

You can grab a coffee, make a hot meal, have a sleep ;-)

Instaling the new kernel

Step 6) Install The New Kernel

After the successful kernel build, you can find two .deb packages in the /usr/src directory. 

cd .. 
ls -l linux*

Install them: 

dpkg -i linux-image-2.6.33-custom_2.6.33-custom-10.00.Custom_amd64.deb
dpkg -i linux-headers-2.6.33-custom_2.6.33-custom-10.00.Custom_amd64.deb

Booting with the new kernel

Step 7) Booting with the new kernel

Add the options to enable IOMMU in the kernel commandline

edit /etc/default/grub 

new: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash intel_iommu=on,igfx_off,pass-through"
old: #GRUB_CMDLINE_LINUX_DEFAULT="quiet"

update the booting menu 

update-grub

and reboot 

sudo reboot


Deinstall networkmanager

Step 8) Deinstall networkmanager 

aptitude remove network-manager


Networking setup

Step 9) Setup your network, change /etc/network/interfaces to look like this 

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The first bridge interface
auto bridge_ext
iface bridge_ext inet static
address 0.0.0.0 #no external access without going through DCERouter first
netmask 255.255.255.0
broadcast 10.0.0.255 
gateway 10.0.0.138
bridge_ports eth0
bridge_stp off
bridge_maxwait 5
# The second bridge interface
auto bridge_int
iface bridge_int inet static
address 192.168.80.240
netmask 255.255.255.0
broadcast 192.168.80.255
gateway 192.168.80.1
bridge_ports eth1
bridge_stp off
bridge_maxwait 5

With the above the host will use dcerouter as its way into the internet and thus the protection offered by the firewall. The host has no direct access to the 10.0.0.0 network.

Install LinuxMCE

Step 10) Install LinuxMCE

Using virt-manager you can now install LinuxMCE. As we are all experts on that I will only give away my xml config file (should be enough ;-)) 

<domain type='kvm'>
 <name>DCERouter</name>
 <uuid>xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx</uuid>
 <memory>2097152</memory>
 <currentMemory>2097152</currentMemory>
 <vcpu>2</vcpu>
 <os>
   <type arch='x86_64' machine='pc-0.11'>hvm</type>
   <boot dev='hd'/>
 </os>
 <features>
   <acpi/>
   <apic/>
   <pae/>
 </features>
 <clock offset='utc'/>
 <on_poweroff>destroy</on_poweroff>
 <on_reboot>restart</on_reboot>
 <on_crash>restart</on_crash>
 <devices>
   <emulator>/usr/bin/kvm</emulator>
   <disk type='block' device='disk'>
     <source dev='/dev/sdc'/>
     <target dev='hda' bus='ide'/>
   </disk>
   <disk type='file' device='cdrom'>
     <target dev='hdc' bus='ide'/>
     <readonly/>
   </disk>
   <interface type='bridge'>
     <mac address='xx:xx:xx:xx:xx:xx'/>
     <source bridge='bridge_ext'/>
   </interface>
   <interface type='bridge'>
     <mac address='xx:xx:xx:xx:xx:xx'/>
     <source bridge='bridge_int'/>
     <model type='virtio'/>
   </interface>
   <serial type='pty'>
     <target port='0'/>
   </serial>
   <console type='pty'>
     <target port='0'/>
   </console>
   <input type='tablet' bus='usb'/>
   <input type='mouse' bus='ps2'/>
   <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>
   <video>
     <model type='cirrus' vram='9216' heads='1'/>
   </video>
   <hostdev mode='subsystem' type='pci' managed='yes'>
     <source>
       <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
     </source>
   </hostdev>
   <hostdev mode='subsystem' type='pci' managed='yes'>
     <source>
       <address domain='0x0000' bus='0x03' slot='0x08' function='0x0'/>
     </source>
   </hostdev>
   <hostdev mode='subsystem' type='pci' managed='yes'>
     <source>
       <address domain='0x0000' bus='0x03' slot='0x09' function='0x0'/>
     </source>
   </hostdev>
   <hostdev mode='subsystem' type='usb' managed='yes'>
     <source>
       <vendor id='0x0403'/>
       <product id='0xf2d0'/>
     </source>
   </hostdev>
   <hostdev mode='subsystem' type='pci' managed='yes'>
     <source>
       <address domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
     </source>
   </hostdev>
 </devices>
</domain>


The rest

Step 11) Enjoying building a test enviroment, inline upgrade etc.

Step 12) I chose to put my media on the host via a Samba share which will be detected as a NAS. This will simplify things when performing upgrades and test. Because I am using LVM also expansion is easy ;-)


Lmce kvm.png




Hints/Known Issues/Working Hardware


AMD/ATI Hardware
- NVidia Chipset 3600 with Opteron Socket F has full KVM support and works fine with LMCE 8.10.
- ATI-driver 9.8 supports kernel 2.6.30

Keyboard Layout
- As you can't choose the keyboard-layout in virtual-manager, simply change the keymap in the xml-file e.g. from "en-us" to "de".

Networking
- Step 11) is not needed if the physical network devices are configured as 'auto ethx \ iface ethx inet manual'
- The virtual LMCE runs fine with only one NIC provided to it. The setup automatically configures a sub-NIC which works well. I would suggest to have the host provide the internet-access, so vLMCE can't mess it up at the installation.

PCI-Passthrough at KVM
- PCI-Passthrough only works if the full PCI-device is transferred to the virtual machine. So if the PCI-Slot has sub-functions (e.g. Firewire) all of the 'functions' have to be adressed to the virtual machine with the <hostdev> section of Step 12). To look up your PCI-device-tree, use 'lspci -t'.

(Tyan S2915-E)
[0000:00]-+-00.0
          +-01.0
          +-01.1
          +-02.0
          +-02.1
          +-04.0
          +-05.0
          +-05.1
          +-06.0-[0000:01]----04.0                                           (<- PCI-Slot = DVB-card)
                            \-04.1                                           (<- Firewire, either disable in Bios or ->KVM)
          +-06.1
          +-08.0
          +-09.0
          +-0d.0-[0000:02-04]--+-00.0-[0000:03]----04.0
          |                    \-00.1-[0000:04]--
          +-0f.0-[0000:05]--+-00.0
          |                 \-00.1
          +-18.0
          +-18.1
          +-18.2
          +-18.3
          \-18.4

- PCIe-devices do not have this restriction.

- Please note that each change in the .xml-files is activated only after rebooting the libvirtd. You can do this manually by killing and restarting the process "libvirtd" or by a host reboot.

Problems with PVR-150 not responding

During usage it turned out that sometimes recordings failed having a total size of 0 B. Live TV has not worked stable at all. Checking the mythbackend logs shows that /dev/video0 was not responding as expected. Furthermore I detected DMA errors in the log of the host.

I am now testing the commandline option iommu=on,igfx_off,pass-through, where pass-through should be able to iron out the DMA errors.

Because the behaviour was not stable I also installed the mercurial ivtv drivers (within the VM ofcourse!!!!): 

sudo apt-get install mercurial
cd /usr/src
sudo hg clone http://linuxtv.org/hg/v4l-dvb
cd v4l-dvb
sudo make
sudo make install
sudo make unload
sudo make load

You will have to rebuild after a kernel change.

Clicking sound on Audio device: Intel Corporation 82801I (ICH9 Family) HD Audio Controller (rev 02)

When passing-through my audio card I noticed hick ups in the MP3's and a message in kern.log; 

hda-intel: IRQ timing workaround is activated for card #0. Suggest a bigger bdl_pos_adj.

edit alsa-base.conf; 

vim /etc/modprobe.dalsa-base.conf

add the following line at the bottom; 

options snd-hda-intel model=3stack-dig
options snd-hda-intel enable_msi=1
options snd-hda-intel single_cmd=1

and reboot (reload alsa does not seem to have the full effect)

Tip from Bugs Launchpad

Retrieved from "http://wiki.linuxmce.org/index.php?title=Installing_LinuxMCE_on_Virtual_Machine_via_KVM&oldid=36683"