EBC Exercise 22 Cross-Compiling and Finding the Right Kernel - bitbake

This class is about developing software for embedded Linux. So far we have been doing all of our development on the Beagle. This works well for small (and not so small) programs. However, we are now moving into kernel development and that's best done on a more powerful host computer. In EBC Exercise 08 Installing Development Tools you learned how to download and install the cross-compilers and the source for kernel and u-boot. Now we'll use those tools.

First we'll check everything by compiling the Hello World program, then we'll try the kernel and u-boot.

Cross-compiling Hello World
This shows how to setup the paths to use the compiler loaded by bitbake.

Listing 2-4 on page 29 of the text is an embedded version of Hello World. If you've set up your git repository you will find it in helloWorld.c when you do a git pull. Compile and run it on your host to be sure it works.

host$ gcc helloWorld.c host$ ./a.out Hello, World! Main is executing at 0x400524 This address (0x7fff8260bdf8) is in our stack frame This address (0x601038) is in our bss section This address (0x601020) is in our data section

Now that you know it's working, let's cross compile it. First set the paths to find the cross-compilers. Put the following in a file, call it ~/.oe/crossCompileEnvBitbake.sh. Make sure the path is correct for your system. This is for a 32-bit linux.

export ARM_TOOLCHAIN_PATH=~/BeagleBoard/oe/build/tmp-angstrom_v2012_05-eglibc/sysroots/i686-linux/usr/bin/armv7a-angstrom-linux-gnueabi PATH=$PATH:$ARM_TOOLCHAIN_PATH/.. PATH=$ARM_TOOLCHAIN_PATH:$PATH export ARCH=arm export CROSS_COMPILE=arm-angstrom-linux-gnueabi-
 * 1) add cross tools to your path

Here it is for 64-bit

export ARM_TOOLCHAIN_PATH=~/BeagleBoard/oe/build/tmp-angstrom_v2012_05-eglibc/sysroots/x86_64-linux/usr/bin/armv7a-angstrom-linux-gnueabi PATH=$PATH:$ARM_TOOLCHAIN_PATH/.. PATH=$ARM_TOOLCHAIN_PATH:$PATH export ARCH=arm export CROSS_COMPILE=arm-angstrom-linux-gnueabi-
 * 1) add cross tools to your path

Now source the file and compile again. (Note: you only have to source once per terminal session.) host$ source ~/.oe/crossCompileEnvBitbake.sh host$ ${CROSS_COMPILE}gcc helloWorld.c host$ file a.out a.out: ELF 32-bit LSB executable, ARM, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.16, not stripped

The file command tells what's in the file. In this case we have an ARM executable. Success! Now copy to your Beagle and run

host$ scp a.out root@beagle:. host$ ssh root@beagle ./a.out Hello, World! Main is executing at 0x8374 This address (0xbeb32d4c) is in our stack frame This address (0x10650) is in our bss section This address (0x10648) is in our data section

The scp copies a.out to the beagle and the ssh runs the a.out on the beagle. Notice the address are very different from the host version.

Finding the Kernel and Installing It
After doing EBC Exercise 08 Installing Development Tools - bitbake you have a Beagle kernel on your host computer. Let's see if it works.

Once compiled the kernel is put in a file called uImage. You may have seen it in the FAT partition on your SD card. Let's find it on your host. One way to find it is

host$ cd ~/BeagleBoard/oe host$ '''find. -name uImage''' ./build/tmp-angstrom_v2012_05-eglibc/work/beagleboard-angstrom-linux-gnueabi/linux-mainline-3.2.18-r121b/git/arch/arm/boot/uImage ./build/tmp-angstrom_v2012_05-eglibc/work/beagleboard-angstrom-linux-gnueabi/linux-mainline-3.2.18-r121b/sysroot-destdir/kernel/uImage ./build/tmp-angstrom_v2012_05-eglibc/work/beagleboard-angstrom-linux-gnueabi/linux-mainline-3.2.18-r121b/image/kernel/uImage ./build/tmp-angstrom_v2012_05-eglibc/sysroots/beagleboard/kernel/uImage

It looks like I've compile the 3.2.18 version of the kernel. What version is the beagle running?

beagle$ uname -a Linux yoder-bone 3.2.25 #1 Fri Aug 10 10:33:12 CEST 2012 armv7l GNU/Linux

The bone is running 3.2.25. Here's how to get our sources to match those used to compile the 3.2.25 kernel.

beagle$ cat /etc/angstrom-build-info Configured Openembedded layers: meta-angstrom    = angstrom-v2012.05-yocto1.2:a029f5f5d19f788e2ab17227f35350ff5b6ca125 meta-oe toolchain-layer meta-efl meta-gpe meta-gnome meta-xfce meta-initramfs meta-multimedia  = denzil:b4488f4c012a64604b28bd2b6e93de411c34466d meta-opie        = master:efa3892b20a4ef80274e56e5633ebd62c16f9731 meta-java        = master:3386ea6c96096f107f43f282f654e5afa456109e meta-browser     = master:c47f59df2e723495679c751cbdf6a8c6adec4b6a meta-mono        = master:83f8233b0498aadb18bf7605c3ba6c71d9e13a3a meta-kde         = master:5b0882d951cfd71886d423c190faaa7c7f932333 meta-ti          = angstrom-staging:5364d0795bb27c2deb5a4e2df7d2f01ffb394cca meta-efikamx     = master:2c09a3a780b23448e8a6ca964256ff7f5ccba65d meta-nslu2       = master:3d9fc951b05b4df476374b6fc3085ebac7f293ee meta-htc meta-nokia meta-openmoko meta-palm        = master:2b106be01228f64298d6cb338f93088806594344 meta-handheld    = master:1f05a15aceb4c3a19fa070463b58125b5658b2a9 meta-raspberrypi = denzil:34eef2ea4f5f24630dbb73b386861430167b8431 meta-intel meta-sugarbay meta-crownbay meta-emenlow meta-fishriver meta-fri2 meta-jasperforest meta-n450        = master:f75f9b6f68473eb0efac802409608f8389be0030 meta             = denzil:b052427a6678b1b696f8c64cf0fdc55e6af86c48

This file shows the various repositories that were used to compile the kernel (and the root file system) for the given SD image (the bone A6A image in my case). Yes, the kernel comes from several git repositories. How many? host$ cd ~/BeagleBoard/oe host$ '''find. -name .git''' ./.git ./build/tmp-angstrom_v2012_05-eglibc/buildhistory/.git ./build/tmp-angstrom_v2012_05-eglibc/work/beagleboard-angstrom-linux-gnueabi/u-boot-2011.12-r8/git/.git ./build/tmp-angstrom_v2012_05-eglibc/work/beagleboard-angstrom-linux-gnueabi/linux-mainline-3.2.18-r121b/git/.git ./build/tmp-angstrom_v2012_05-eglibc/work/armv7a-angstrom-linux-gnueabi/mtd-utils-1.4.9-r1/git/.git ./sources/meta-xilinx/.git ./sources/meta-intel/.git ./sources/bitbake/.git ./sources/meta-efikamx/.git ./sources/meta-angstrom/.git ./sources/openembedded-core/.git ./sources/meta-handheld/.git ./sources/meta-kde/.git ./sources/meta-nslu2/.git ./sources/meta-openembedded/.git ./sources/meta-raspberrypi/.git ./sources/meta-mono/.git ./sources/meta-java/.git ./sources/meta-ti/.git ./sources/meta-ettus/.git ./sources/meta-openpandora/.git ./sources/meta-smartphone/.git ./sources/meta-opie/.git ./sources/meta-browser/.git

The first .git is the repository you cloned to do the initial bitbake. The next is the u-boot repository. The next one is the kernel's repository. We'll be using that repository so much you should set up a symbolic link to it so you can find it easily.

host$ ln -s ~/BeagleBoard/oe/build/tmp-angstrom_v2012_05-eglibc/work/beagleboard-angstrom-linux-gnueabi/linux-mainline-3.2.18-r121b/git/ ~/...

Notice several .gits are in the sources directory. Note they have many names in common with the contents of /etc/angstrom-build-info. These are the repositories where we need to checkout the commits that correspond to the tags above. Here's how, take the first repository with a tag and cd to it.

meta-angstrom
host$ cd sources/meta-angstrom/ /home/beagle/BeagleBoard/oe/sources/meta-angstrom host$ git status nothing to commit (working directory clean) host$ git remote update Fetching origin remote: Counting objects: 132, done. remote: Compressing objects: 100% (57/57), done. remote: Total 105 (delta 53), reused 96 (delta 45) Receiving objects: 100% (105/105), 16.95 KiB, done. Resolving deltas: 100% (53/53), completed with 18 local objects. From git://github.com/Angstrom-distribution/meta-angstrom 140671a..784d76b master     -> origin/master
 * 1) On branch angstrom-v2012.05-yocto1.2

Now checkout the corresponding tag host$ git checkout a029f5f5d19f788e2ab17227f35350ff5b6ca125 Note: checking out 'a029f5f5d19f788e2ab17227f35350ff5b6ca125'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b new_branch_name HEAD is now at a029f5f... base-files bbappend: move /var/run handling to systemd proper host$ git status nothing to commit (working directory clean)
 * 1) Not currently on any branch.

Don't worry that your HEAD is detached. That just means your not on a branch. However it is the commit that was used to create the kernel we want.

meta-opie
Now, repeat this for each of the hash tags in /etc/angstrom-build-info. I'll get you started, you finish it.

host$ cd ../meta-opie /home/beagle/BeagleBoard/oe/sources/meta-opie host$ git remote update Fetching origin host$ git checkout efa3892b20a4ef80274e56e5633ebd62c16f9731 HEAD is now at efa3892... opie-packagemanager: fix a couple more quoting issues

and so on...
Keep going until you get an error.

meta-ti
This repository pulls from more than one place so the checkout will fail. Here's what to do. host$ cd ../meta-ti host$ git remote update host$ git checkout 5364d0795bb27c2deb5a4e2df7d2f01ffb394cca fatal: reference is not a tree: 5364d0795bb27c2deb5a4e2df7d2f01ffb394cca

So what repository has the tag 5364d0795bb27c2deb5a4e2df7d2f01ffb394cca? Try google. With a bit of googling, I was able to find https://github.com/Angstrom-distribution/meta-ti/commit/5364d0795bb27c2deb5a4e2df7d2f01ffb394cca. Add this to the remote repositories. host$ git remote add test https://github.com/Angstrom-distribution/meta-ti host$ git remote update host$ git checkout 5364d0795bb27c2deb5a4e2df7d2f01ffb394cca

Now continue on with the rest of the repositories.

Once all the repositories are all up to date
Now it's time to bitbake again and the 3.2.25 kernel should appear.

host$ cd ~/BeagleBoard/oe host$ source ~/.oe/environment-angstromv2012.05 host$ time bitbake virtual/kernel

Finding the Kernel for the ETC 2012 image
We are looking for uImage, but we only want the 2.6.32 version. The last grep file the file in the boot directory. Several lines are printed, but the one I'm interested in is: ./build/tmp-angstrom_2010_x-eglibc/work/beagleboard-angstrom-linux-gnueabi/linux-omap-psp-2.6.32-r110b+gitr5fc29e7b2a76a64a739f857858ef0b98294aa155/git/arch/arm/boot/uImage

The first part of the path (in italics) is the path to the kernel. Check it out.

host$ cd build/tmp-angstrom_2010_x-eglibc/work/beagleboard-angstrom-linux-gnueabi/linux-omap-psp-2.6.32-r110b+gitr5fc29e7b2a76a64a739f857858ef0b98294aa155/git host$ ls -F arch/          drivers/   Kbuild       modules.order   samples/    usr/ block/         firmware/  kernel/      Module.symvers  scripts/    virt/ COPYING        fs/        lib/         net/            security/   vmlinux* CREDITS        include/   MAINTAINERS  patches/        sound/      vmlinux.o crypto/         init/      Makefile     README          System.map Documentation/ ipc/       mm/          REPORTING-BUGS  tools/

We'll be learning what's in many of these over the next couple of weeks. Remember this location. Let's find uImage.

host$ cd arch/arm/boot; ls -shF total 13M 4.0K bootp/      6.3M Image*      4.0K Makefile  3.0M zImage* 4.0K compressed/ 4.0K install.sh  3.0M uImage host$ ls -l uImage -rw-r--r-- 1 beagle beagle 3144300 2011-12-08 01:02 uImage

It should have the date that you did the bitbake. Let's see if it runs on the Beagle host$ scp uImage root@beagle:. host$ ssh root@beagle beagle$ cd /boot; ls -F total 18M 4.0K MLO@ 372K Module.symvers-2.6.32 1.4M System.map-2.6.32 80K config-2.6.32 4.0K u-boot.bin@ 0 uImage@ 3.1M uImage-2.6.32 8.9M vmlinux-2.6.32 beagle$ ls -l uImage lrwxrwxrwx 1 www-data www-data 13 May 12 2011 uImage -> uImage-2.6.32

So the uImage that is there is a symbolic link to uImage-2.6.32. Let's save the working uImage in a difference file and copy our new uImage in.

beagle$ mv uImage-2.6.32 uImage-2.6.32.orig beagle$ mv ~/uImage uImage-2.6.32

And then reboot. beagle$ shutdown -r now

After a couple of minutes your should be back and running again. Check and see if you are really running the new kernel. beagle$ uname -a Linux beagleboard 2.6.32 #3 PREEMPT Thu Dec 8 01:02:13 EST 2011 armv7l GNU/Linux

It worked! That's the date I compiled mine on.

Failed Kernel
Here's what I get when trying to run the 3.2.18 kernel on the A6A bone image.

reading u-boot.img reading u-boot.img

U-Boot 2011.09-00053-gb423c52 (Aug 10 2012 - 11:26:55)

I2C:  ready DRAM: 256 MiB WARNING: Caches not enabled No daughter card present NAND: HW ECC Hamming Code selected No NAND device found!!! 0 MiB MMC:  OMAP SD/MMC: 0, OMAP SD/MMC: 1
 * Warning - readenv failed, using default environment

Net:  cpsw Hit any key to stop autoboot: 0 SD/MMC found on device 0 reading uEnv.txt

27 bytes read Loaded environment from uEnv.txt Importing environment from mmc ... Loading file "/boot/uImage" from mmc device 0:2 xxa2 4385656 bytes read Image Name:  Linux-3.2.18 Image Type:  ARM Linux Kernel Image (uncompressed) Data Size:   4385592 Bytes = 4.2 MiB Load Address: 80008000 Entry Point: 80008000 Verifying Checksum ... OK  XIP Kernel Image ... OK OK
 * 1) Booting kernel from Legacy Image at 80007fc0 ...

Starting kernel ...

Installing a New U-boot
While we're at it, let's install a new U-boot. Note: The new U-boot runs for me, but doesn't boot the kernel, yet.

host$ cd ~/BeagleBoard/oe/build/tmp-angstrom_2010_x-eglibc/work/beagleboard-angstrom-linux-gnueabi/ host$ ls -F linux-3.0.9-r110a/ linux-omap-psp-2.6.32-r110b+gitr5fc29e7b2a76a64a739f857858ef0b98294aa155/ shadow-4.1.4.3-r5/ u-boot-2011.09-r4/

There's the U-boot directory.

host$ cd u-boot-2011.09-r4/git/; ls -F api/       COPYING    fs/          mkconfig*     post/            tools/ arch/      CREDITS    include/     mmc_spl/      README           u-boot* board/     disk/      lib/         nand_spl/     rules.mk         u-boot.bin boards.cfg doc/       MAINTAINERS  net/          snapshot.commit  u-boot.lds common/    drivers/   MAKEALL*     onenand_ipl/  spl/             u-boot.map config.mk  examples/  Makefile     patches/      System.map       u-boot.srec

The file we want is right on the top level.

host$ scp u-boot.bin root@beagle:. host$ ssh -X root@beagle beagle$ cd /media/mmcblk0p1/; ls -F MLO*        u-boot.bin.broken*  uEnv.txt* U-BOOT.BIN* UIMAGE*             uEnv/

Be sure you are logged into the Beagle via the serial port, rather than using ssh. Otherwise you won't see the boot sequence that appears before the kernel is running.

Here we've changed to the FAT partition which is where u-boot lives. Back it up and install the new one.

beagle$ mv U-BOOT-BIN u-boot-bin.orig beagle$ mv ~/.u-boot-bin . beagle$ shutdown -r now

You should now see the new u-boot running if you are logged in via the serial port.

Compile via make
When you use bitbake it sets up all the paths to use the correct cross compilers. You can also build the kernel or u-boot by using make if you set the paths like we did above.

If you haven't already, source the file and cd to the kernel directory and try a make.

host$ source ~/.oe/crossCompileEnv.sh host$ cd ~/BeagleBoard/oe/build/tmp-angstrom_2010_x-eglibc/work/beagleboard-angstrom-linux-gnueabi/linux-omap-psp-2.6.32-r110b+gitr5fc29e7b2a76a64a739f857858ef0b98294aa155/git host$ make xconfig

(EBC Exercise 13 Configuring the Kernel has details on configuring the kernel.)

To make the kernel run

host$ make uImage

Now, follow the instructions above to install your freshly compiled kernel.

Getting the A6A Kernel
beagle$ cat /etc/angstrom-build-info Configured Openembedded layers: meta-angstrom    = angstrom-v2012.05-yocto1.2:a029f5f5d19f788e2ab17227f35350ff5b6ca125 meta-oe toolchain-layer meta-efl meta-gpe meta-gnome meta-xfce meta-initramfs meta-multimedia  = denzil:b4488f4c012a64604b28bd2b6e93de411c34466d meta-opie        = master:efa3892b20a4ef80274e56e5633ebd62c16f9731 meta-java        = master:3386ea6c96096f107f43f282f654e5afa456109e meta-browser     = master:c47f59df2e723495679c751cbdf6a8c6adec4b6a meta-mono        = master:83f8233b0498aadb18bf7605c3ba6c71d9e13a3a meta-kde         = master:5b0882d951cfd71886d423c190faaa7c7f932333 meta-ti          = angstrom-staging:5364d0795bb27c2deb5a4e2df7d2f01ffb394cca meta-efikamx     = master:2c09a3a780b23448e8a6ca964256ff7f5ccba65d meta-nslu2       = master:3d9fc951b05b4df476374b6fc3085ebac7f293ee meta-htc meta-nokia meta-openmoko meta-palm        = master:2b106be01228f64298d6cb338f93088806594344 meta-handheld    = master:1f05a15aceb4c3a19fa070463b58125b5658b2a9 meta-raspberrypi = denzil:34eef2ea4f5f24630dbb73b386861430167b8431 meta-intel meta-sugarbay meta-crownbay meta-emenlow meta-fishriver meta-fri2 meta-jasperforest meta-n450        = master:f75f9b6f68473eb0efac802409608f8389be0030 meta             = denzil:b052427a6678b1b696f8c64cf0fdc55e6af86c48

host$ cd BeagleBoard/oe/sources/meta-ti host$ git checkout master