User:Uli/Acer Chromebook R13 GPU test system

This page gives instructions on how to install a Debian userland system for the Chromebook R13, and how to compile a kernel with GPU support from source.

= Purpose = This will create a dual-booting environment where you can switch between booting Debian Linux ARM and the stock ChromeOS. No changes are made to the internal eMMC drive, and your new Linux install will run completely from external storage.

These instructions are intended to produce a system sufficient for testing GPU operation. There is no focus on the things a typical user would want, such as WiFi and a desktop environment. If you want those, you are on your own.

= Requirements =
 * An Acer Chromebook R13.
 * An SD card.
 * A USB-to-Ethernet adapter (optional, but highly recommended).

You must be running the latest ChromeOS prior to installation.

The instructions for switching the Chromebook to developer mode and for preparing an SD card have been lifted wholesale from the ArchLinux ARM web site.

= Switch to developer mode =
 * 1) Turn off the laptop.
 * 2) To invoke Recovery mode, you hold down the ESC and Refresh keys and poke the Power button.
 * 3) At the Recovery screen press Ctrl-D (there's no prompt - you have to know to do it).
 * 4) Confirm switching to developer mode by pressing enter, and the laptop will reboot and reset the system. This takes about 10-15 minutes.

Note: After enabling developer mode, you will need to press Ctrl-D each time you boot, or wait 30 seconds to continue booting.

= Enable booting from external storage =
 * 1) After booting into developer mode, hold Ctrl and Alt and poke the T key. This will open up the crosh shell.
 * 2) Type shell to get into a bash shell.
 * 3) Type sudo su to become root.
 * 4) Then type this to enable USB booting:  crossystem dev_boot_usb=1 dev_boot_signed_only=0
 * 5) Reboot the system to allow the change to take effect.

= Create a root USB or SD for dual booting = Note: If you are performing this step on the Chromebook, it is highly recommended to use a USB SD card adapter for these steps when targeting a micro SD card. The ChromeOS kernel uses a high speed for the SD slot, which can cause many errors reading and writing to the card. The Linux kernel we build later lowers the speed to allow safe operation in the micro SD slot.

These instructions are written for installing to an SD card. This is recommended as it leaves the USB port free for other things, such as a USB-to-Ethernet adapter. The steps for preparing a USB drive are identical, however; only the device names have to be changed.

 If you are doing this on the Chromebook, get a root shell as described in the previous section. Many Linux systems, including ChromeOS, will automatically mount any partitions they find, so unmount everything now: umount /dev/mmcblk1* Start fdisk to create a GPT partition table: fdisk /dev/mmcblk1 At the fdisk prompt: Partition the micro SD card: cgpt create /dev/mmcblk1 cgpt add -i 1 -t kernel -b 8192 -s 65536 -l Kernel -S 1 -T 5 -P 10 /dev/mmcblk1 To create the rootfs partition, we first need to calculate how big to make the partition using information from cgpt show. Look for the number under the start column for Sec GPT table which is 15633375 in this example: localhost / # cgpt show /dev/mmcblk1 start       size    part  contents 0          1          PMBR 1          1          Pri GPT header 8192      65536      1   Label: "Kernel" Type: ChromeOS kernel UUID: E3DA8325-83E1-2C43-BA9D-8B29EFFA5BC4 Attr: priority=10 tries=5 successful=1 15633375         32          Sec GPT table 15633407          1          Sec GPT header Replace the xxxxx string in the following command with that number to create the root partition: cgpt add -i 2 -t data -b 73728 -s `expr xxxxx - 73728` -l Root /dev/mmcblk1 Tell the system to refresh what it knows about the disk partitions:</li> partx -a /dev/mmcblk1 Format the root partition:</li> mkfs.ext4 /dev/mmcblk1p2 </ol>
 * 1) Type g. This will create a new empty GPT partition table.
 * 2) Write the partition table and exit by typing w.

= Install a Debian armhf userland system =

The ChromeOS system runs a 32-bit hardfloat userspace, and we need the binary-only libraries for GPU operation, so we need a compatible system.

These instructions assume that you are running a Debian-derived distribution.

<ol>Install the packages <tt>debootstrap</tt> and <tt>qemu-user-static</tt>.</li> Create a directory that you want to contain your new Debian system.</li> Run debootstrap:</li> qemu-debootstrap --arch armhf sid $TARGET_DIRECTORY http://deb.debian.org/debian/ Wait.</li> </ol>

Once the process has finished, you will have a very bare-bones Debian armhf system in <tt>$TARGET_DIRECTORY</tt>. Mount the root partition you have created on the SD card and rsync the contents of <tt>$TARGET_DIRETORY</tt> there.

Edit the file <tt>/etc/shadow</tt> and remove the asterisk in the line starting with <tt>root</tt>. That will allow you to log in as root on the console.

= Build a kernel with GPU support =

Requirements
<ol>If you are not running an ARM64 workstation, you need an ARM64 (aka aarch64) cross-toolchain.</li> You need the u-boot <tt>mkimage</tt> tool and the ChromeOS verified boot utilities:</li> apt-get install u-boot-tools vboot-kernel-utils Note: In older Debian versions, the vboot-kernel-utils package was built without support for non-native architectures, so if you are running a non-ARM system, you may have to manually install a version of this package from a later distribution. A number of files are required that can be found at the ArchLinux ARM PKGBUILDs repository in the subdirectory <tt>core/linux-oak</tt>:</li> 0009-Downgrade-mmc1-speed.patch cmdline config kernel.its kernel.keyblock kernel_data_key.vbprivk </ol>

Build Procedure
<ol>Clone the ChromeOS kernel sources:</li> git clone https://chromium.googlesource.com/chromiumos/third_party/kernel r13-kernel Switch to the chromeos-3.18 branch:</li> cd r13-kernel git checkout chromeos-3.18 Apply the MMC patch. This patch is very important. Without it, corruption of the root filesystem on the SD card is guaranteed.</li> git am 0009-Downgrade-mmc1-speed.patch Set the cross-compiling environment variables.</li> export ARCH=arm64 export CROSS_COMPILE=aarch64-linux-gnu- You may have to amend the latter, depending on what toolchain you use. Configure the kernel. Copy the ArchLinux <tt>config</tt> file to <tt>.config</tt> (with a dot), then run .</li> Enable the GPU driver. The config option is <tt>CONFIG_DRM_POWERVR_ROGUE_1_6</tt>. Build kernel, modules and device trees:</li> make -j$VIEL Image modules dtbs Create a kernel partition image:</li> mkimage -D "-I dts -O dtb -p 2048" -f kernel.its vmlinux.uimg dd if=/dev/zero of=bootloader.bin bs=512 count=1 vbutil_kernel --pack vmlinux.kpart --version 1 --vmlinuz vmlinux.uimg --arch aarch64 --keyblock kernel.keyblock --signprivate kernel_data_key.vbprivk --config cmdline --bootloader bootloader.bin </ol>

Install the kernel and modules
<ol> <li>Install the generated kernel partition image to partition 1 of your SD card:</li> dd if=vmlinux.kpart of=/dev/mmcblk1p1 <li>Install the kernel modules to the root file system on your SD card:</li> make modules_install INSTALL_MOD_PATH= </ol>

= Booting the system on the Chromebook =
 * 1) Unmount the SD card and remove it from your workstation.
 * 2) With the Chromebook turned off, insert the SD card.
 * 3) Turn the Chromebook on.
 * 4) At the "OS verification disabled" screen, press Ctrl-U. (Works for both USB drives and SD cards.)

After a few seconds, you should be greeted with a login screen (and some random kernel messages).

= Install development tools =

In order to install more software, you need a network connection, so connect the USB-to-Ethernet adapter.

<ol> <li>Run .</li> <li>Run .</li> <li>Install a number of packages: <ol><li>Development essentials:</li> apt-get install gcc libgles2-mesa-dev libgbm-dev git autoconf automake libtool pkgconf make <li>Convenience:</li> apt-get install net-tools rsync less $YOUR_EDITOR </ol> </ol>

= Rip the binary blobs from ChromeOS =

<ol><li>Mount the ChromeOS system partition:</li> mount -o ro /dev/mmcblk0p3 /mnt <li>Copy the firmware blobs for various devices:</li> rsync -a /mnt/lib/firmware/ /lib/firmware/ A reboot may be required to make sure the new firmware files are loaded correctly. <li>Copy the DRI driver:</li> mkdir /usr/lib/dri cp -p /mnt/usr/lib/dri/pvr_dri.so /usr/lib/dri <li>Rsync the userspace libraries:</li> mkdir /pvrlibs rsync -a /mnt/usr/lib/lib{EGL,GLESv2,dbm,gbm,glslcompiler,minigbm,nettle,pvr_dri_support,srv_um,usc}.so* /pvrlibs/ </ol>

This allows you to use the GPU by running your program like this: LD_LIBRARY_PATH=/pvrlibs

Alternatively, you can copy the libraries to <tt>/usr/lib</tt> and do away with the explicit library path, but that may get you in trouble when updating packages.

Another possibility is not to copy the files and to simply point the library path to the ChromeOS partition. That works with trivial examples, but is likely to break with anything substantial.

= Mainline kernel =

A non-working forward-port of the R13-specific drivers to the mainline kernel can be found at https://github.com/uli/kernel/tree/elm-broken. It currently freezes with a blank screen at an unknown point. Hints on how to get a serial console on this device would be appreciated.

Low-level debugging
There is an on-board USB hub to which the external USB 3.0 port is connected, and it can be enabled and disabled using GPIO 118. (The firmware source code says "reset", but similar chips by the same manufacturer do not have a reset pin, so I suspect the GPIO just switches the power supply on and off.)

The firmware turns the hub on, which pulls the D- pin of the USB 3.0 port low. The kernel does not touch pin 118, so the hub remains in that state. When the hub is turned off, D- is pulled to 3.3V. So if one adds code anywhere in the kernel that writes 0x40 to 0x10005478, D- will go high if it has been executed, and remain low otherwise.

This behavior is independent of the state of the other USB hardware, and the method works just as well in head.S as it does in any driver.

There is a substantial delay between turning the hub on again (write 0x40 to 0x10005474) and D- going low, which is what makes me suspect that the reset is not a "real" reset but a power cycle. For practical purposes, this makes it more or less impossible to bit-bang serial data out of D-, but a slow (1 Hz) blink is possible.