Linuxinternals-qemu-hacking

Introduction
This is a guide to quickly setting a Qemu environment, suitable for kernel testing and hacking on an x86 host. It also contains instruction on setting up PSTORE if you want to. The guide assumes that you are running an Ubuntu host.

Install libvirt / virt-manager
The easiest way to install qemu with a user-friendly front end (virt-manager) is to install virt-manager, which will pull all the dependencies including Qemu that you would need. sudo apt-get install virt-manager [Then, follow the instructions here to install the latest Ubuntu ISO https://help.ubuntu.com/community/KVM/VirtManager]

Configuration of your libvirt guest
Make sure to add bridged networking to your guest, give it enough CPUs and atleast 1GB RAM. Make sure openssh-server is installed in the guest, this will allow you to scp kernel images to your guest and test them.

Building kernel for the Qemu guest
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git cd linux
 * check out Linus's kernel tree on your host:

cp /boot/config-`uname -r` .config
 * Assuming your host is running a fairly recent version of Ubuntu, run the following


 * If you're hacking on PSTORE, follow the section "Additional kernel steps for PSTORE" below and come back here.

make olddefconfig
 * Configure kernel

make bindeb-pkg
 * Build the kernel (use the bindeb-pkg as below to build convenient Ubuntu debian packages)

linux-image-4.7.0+_4.7.0+-1_amd64.deb
 * After a successful build, one directory below you'll find a linux-image .deb package (ignore the -dbg package), it should look something like

Now scp this deb pkg to your Qemu guest, ssh into your qemu guest and run: dpkg -i linux-image-4.7.0+_4.7.0+-1_amd64.deb    # or whatever the .deb filename is

Additional kernel build steps for PSTORE
CONFIG_PSTORE=y CONFIG_PSTORE_FTRACE=y CONFIG_PSTORE_PMSG=y CONFIG_PSTORE_RAM=y
 * If you're hacking on pstore, edit .config and make sure the following options are enabled (=y):

CONFIG_CMDLINE_BOOL=y CONFIG_CMDLINE="memmap=64K$0x10000000 reboot=w"
 * Also add the following to .config, this will make sure kernel reserves 64KB of RAM starting from 256MB:

wget https://raw.githubusercontent.com/joelagnel/joel-snips/master/patches/kernel/pstore/ramoops-force-params.patch git am ramoops-force-params.patch
 * Download the following patch and apply it, it is required for ramoops + pstore to work:

wget https://raw.githubusercontent.com/joelagnel/joel-snips/master/patches/kernel/pstore/0001-hack-force-devmem2-to-r-w-from-all-mem.patch git am 0001-hack-force-devmem2-to-r-w-from-all-mem.patch
 * Download the devmem2 patch, its required for devmem2 tool to read and write to all physical memory

Adding Persistent RAM support to Qemu
The following steps add support for persistent memory in Qemu. This is required to make the PSTORE feature work, as it depends on RAM persisting across a reboot. Inorder to persist RAM, Qemu will back all guest RAM with a file. This is possible after successfully following the steps outlined below:

Building Qemu from source
sudo apt-get source qemu-system-x86_64
 * First download the sources of Qemu on your Ubuntu host

wget https://raw.githubusercontent.com/joelagnel/joel-snips/master/patches/qemu/0001-Force-QEMU-to-persist-memory-to-mem-path-specified-f.patch git am 0001-Force-QEMU-to-persist-memory-to-mem-path-specified-f.patch
 * If you're hacking on pstore project, apply the following PSTORE patch to the source to make all guest RAM backed by a file:


 * Follow the instructions [here http://www.cyberciti.biz/faq/rebuilding-ubuntu-debian-linux-binary-package/] to rebuild the qemu package. The package name is qemu-system-x86_64 . Note that to save time, pass the option "-nc -j8" to dpkg-buildpackage. This will save time by doing incremental builds and with parallelism.

cp /usr/bin/qemu-system-x86_64 /usr/bin/qemu-system-x86_64.bak cp ./debian/qemu-system-x86/usr/bin/qemu-system-x86_64 /usr/bin
 * Instead of installing the new package deb, its faster to copy the built qemu binary to /usr/bin

Configuring Qemu to use memory backed file
Once you have created a guest in virt-manager and made sure its working fine, power the guest off.

# # # user = "root"
 * Edit /etc/libvirt/qemu.conf and change user and group to root as below:
 * 1) Some examples of valid values are:
 * 1)       user = "qemu"   # A user named "qemu"
 * 2)       user = "+0"     # Super user (uid=0)
 * 3)       user = "100"    # A user named "100" or a user with uid=100

group = "root"
 * 1) The group for QEMU processes run by the system instance. It can be
 * 2) specified in a similar way to user.

The above step is required so that qemu has permissions to open and memory map the file backing the RAM. If you have a better way, let me know ;)

Add the following to your guest XML file. It will look like /etc/libvirt/qemu/.xml. (for me, its /etc/libvirt/qemu/xenial.xml)     This makes Qemu back all RAM for this guest to/from the file /var/lib/libvirt/qemu/ramf
 * Add the memory backing file to qemu command line, for this you have to edit the libvirt configuration for the guest.

Now power up the guest, all memory should be backed by the above file, any memory writes should persist across reboots.

Verify that all RAM qemu is writing to is file backed
CONFIG_CMDLINE_BOOL=y CONFIG_CMDLINE="memmap=64K$0x20000000 reboot=w" This reserves 64KB at an offset of 256MB from start of the RAM. Rebuild your kernel, copy it to the guest and boot into it.
 * Reserve a certain area of RAM using memmap (see instructions in the "Additional kernel build steps for PSTORE" section but change the following).

sudo apt-get install devmem2
 * Install devmem2 on the guest

root@joel:~# devmem2 0x20000000 w 0xdead /dev/mem opened. Memory mapped at address 0x7f9efae6a000. Value at address 0x10000000 (0x7f9efae6a000): 0x43474244 Written 0xDEAD; readback 0xDEAD
 * In the guest, write 32-bit word "0xDEAD" to the reserved memory area:

root@joel:~# devmem2 0x20000000 w /dev/mem opened. Memory mapped at address 0x7fd5db021000. Value at address 0x10000000 (0x7fd5db021000): 0xDEAD
 * Reboot the guest, then verify that what was written persisted: