Linuxinternals-qemu-hacking

From eLinux.org
Revision as of 22:21, 5 September 2016 by Joelagnel (talk | contribs) (correctiom)
Jump to: navigation, search

Introduction

This is a guide to quickly setting a Qemu environment, suitable for kernel testing and hacking on an x86 host. The guide assumes that you are running an Ubuntu host.

Install libvirt / virt-manager

The easiest way to install qemu with a user-friendly front end (virt-manager) is to install virt-manager, which will pull all the dependencies including Qemu that you would need.

sudo apt-get install virt-manager

[Then, follow the instructions here to install the latest Ubuntu ISO https://help.ubuntu.com/community/KVM/VirtManager]

Configuration of your libvirt guest

Make sure to add bridged networking to your guest, give it enough CPUs and atleast 1GB RAM. Make sure openssh-server is installed in the guest, this will allow you to scp kernel images to your guest and test them.

Building kernel for the Qemu guest

  • check out Linus's kernel tree on your host:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
cd linux
  • Assuming your host is running a fairly recent version of Ubuntu, run the following
cp /boot/config-`uname -r` .config
  • If you're hacking on PSTORE, follow the section "Additional kernel steps for PSTORE" below and come back here.
  • Configure kernel
make olddefconfig
  • Build the kernel (use the bindeb-pkg as below to build convenient Ubuntu debian packages)
make bindeb-pkg
  • After a successful build, one directory below you'll find a linux-image .deb package (ignore the -dbg package), it should look something like
linux-image-4.7.0+_4.7.0+-1_amd64.deb

Now scp this deb pkg to your Qemu guest, ssh into your qemu guest and run:

dpkg -i  linux-image-4.7.0+_4.7.0+-1_amd64.deb    # or whatever the .deb filename is

Additional kernel build steps for PSTORE

  • If you're hacking on pstore, edit .config and make sure the following options are enabled (=y):
CONFIG_PSTORE=y
CONFIG_PSTORE_FTRACE=y
CONFIG_PSTORE_PMSG=y
CONFIG_PSTORE_RAM=y
  • Also add the following to .config, this will make sure kernel reserves 64KB of RAM starting from 256MB:
CONFIG_CMDLINE_BOOL=y
CONFIG_CMDLINE="memmap=64K$0x10000000 reboot=w"
  • Download the following patch and apply it, it is required for ramoops + pstore to work:
wget https://raw.githubusercontent.com/joelagnel/joel-snips/master/patches/kernel/pstore/ramoops-force-params.patch
git am ramoops-force-params.patch
  • Download the devmem2 patch, its required for devmem2 tool to read and write to all physical memory
wget https://raw.githubusercontent.com/joelagnel/joel-snips/master/patches/kernel/pstore/0001-hack-force-devmem2-to-r-w-from-all-mem.patch
git am 0001-hack-force-devmem2-to-r-w-from-all-mem.patch

Adding Persistent RAM support to Qemu

The following steps add support for persistent memory in Qemu. This is required to make the PSTORE feature work, as it depends on RAM persisting across a reboot. Inorder to persist RAM, Qemu will back all guest RAM with a file. This is possible after successfully following the steps outlined below:

Building Qemu from source

  • First download the sources of Qemu on your Ubuntu host
sudo apt-get source qemu-system-x86_64
  • If you're hacking on pstore project, apply the following PSTORE patch to the source to make all guest RAM backed by a file:
wget https://raw.githubusercontent.com/joelagnel/joel-snips/master/patches/qemu/0001-Force-QEMU-to-persist-memory-to-mem-path-specified-f.patch
git am 0001-Force-QEMU-to-persist-memory-to-mem-path-specified-f.patch
  • Instead of installing the new package deb, its faster to copy the built qemu binary to /usr/bin
cp /usr/bin/qemu-system-x86_64 /usr/bin/qemu-system-x86_64.bak
cp ./debian/qemu-system-x86/usr/bin/qemu-system-x86_64 /usr/bin

Configuring Qemu to use memory backed file

Once you have created a guest in virt-manager and made sure its working fine, power the guest off.

  • Edit /etc/libvirt/qemu.conf and change user and group to root as below:
#
# Some examples of valid values are:
#
#       user = "qemu"   # A user named "qemu"
#       user = "+0"     # Super user (uid=0)
#       user = "100"    # A user named "100" or a user with uid=100
#
user = "root"
# The group for QEMU processes run by the system instance. It can be
# specified in a similar way to user.
group = "root"

The above step is required so that qemu has permissions to open and memory map the file backing the RAM. If you have a better way, let me know ;)

  • Add the memory backing file to qemu command line, for this you have to edit the libvirt configuration for the guest.

Add the following to your guest XML file. It will look like /etc/libvirt/qemu/<guest-name>.xml. (for me, its /etc/libvirt/qemu/xenial.xml)

 <qemu:commandline>
   <qemu:arg value='-mem-path'/>
   <qemu:arg value='/var/lib/libvirt/qemu/ramf'/>
 </qemu:commandline>

This makes Qemu back all RAM for this guest to/from the file /var/lib/libvirt/qemu/ramf

Now power up the guest, all memory should be backed by the above file, any memory writes should persist across reboots.

Verify that all RAM qemu is writing to is file backed

  • Reserve a certain area of RAM using memmap (see instructions in the "Additional kernel build steps for PSTORE" section but change the following).
CONFIG_CMDLINE_BOOL=y
CONFIG_CMDLINE="memmap=64K$0x20000000 reboot=w"

This reserves 64KB at an offset of 256MB from start of the RAM. Rebuild your kernel, copy it to the guest and boot into it.

  • Install devmem2 on the guest
sudo apt-get install devmem2
  • In the guest, write 32-bit word "0xDEAD" to the reserved memory area:
root@joel:~# devmem2 0x20000000 w 0xdead
/dev/mem opened.
Memory mapped at address 0x7f9efae6a000.
Value at address 0x10000000 (0x7f9efae6a000): 0x43474244
Written 0xDEAD; readback 0xDEAD
  • Reboot the guest, then verify that what was written persisted:
root@joel:~# devmem2 0x20000000 w
/dev/mem opened.
Memory mapped at address 0x7fd5db021000.
Value at address 0x10000000 (0x7fd5db021000): 0xDEAD