EBC Exercise 08 Installing Development Tools - bitbake - Revision history

Yoder: /* Install development environment v1 */

2012-10-29T19:56:53Z

‎Install development environment v1

Yoder: /* Install development environment v2 */ Moved to own page

2012-10-22T12:14:24Z

‎Install development environment v2: Moved to own page

Yoder: /* Installing on a Remote Machine */ Moved to own page

2012-10-22T12:13:57Z

‎Installing on a Remote Machine: Moved to own page

Yoder: Splitting off from git version.

2012-10-22T11:54:19Z

Splitting off from git version.

New page

[[Category:ECE497]]
{{YoderHead}}

Early in the class most of the exercises we will do will all run on the BeagleBoard. You'll be able to edit ([http://projects.gnome.org/gedit/ gedit]), compile ([http://gcc.gnu.org/ gcc]) and run all on the Beagle. Later, when we start compiling the kernel [http://www.kernel.org/] or the boot loader, ([http://www.denx.de/wiki/U-Boot U-boot]) you will need to cross compile on a Linux machine and copy the results to the Beagle.

The purpose of this exercise is to install all the tools needed for compiling on your host so they will be ready when you need them.

Instructions for building Ångström are given [http://www.angstrom-distribution.org/building-angstrom here]; however there are a few changes you have to make. Here's what I did.

'''Tip:''' Run this exercise using a wired connection if you can. The Ubuntu wireless driver can be finicky, and if it stops working you'll have to restart some of this.

== Install development environment v2 ==
Follow these v2 instructions. They work for the 3.2.25 kernel.

=== Using Jason's instructions ===

These are notes on following instructions [https://github.com/beagleboard/kernel/blob/6682025752d0b807119c1e363a0b1b9bfe2ab453/README.md Beagleboard kernel git site].

host$ '''sudo apt-get install -y git lzop gcc-arm-linux-gnueabi uboot-mkimage'''

25 seconds

host$ '''git clone git://github.com/beagleboard/kernel.git'''

2.5 seconds

host$ '''./patch.sh'''

38.5 minutes

host$ '''make -j9'''

4 minutes

host$ '''make uImage'''

18 seconds

host$ '''make uImage'''
host$ '''mkdir ~/kernel/rootfs'''
host$ '''make INSTALL_MOD_PATH=~/kernel/rootfs modules_install'''
host$ '''cd'''

12 seconds

host$ '''scp kernel/kernel/arch/arm/boot/uImage root@beaglebone.local:/boot/uImage-3.2.25+
10 seconds

host$ '''cd kernel/rootfs'''
host$ '''find -H -depth | cpio -o -H crc | ssh root@beaglebone.local 'cd /; cpio -id' '''

1 minute 16 seconds

host$ '''cd'''
host$ '''ssh root@beaglebone.local 'cd /boot; rm uImage' '''
host$ ssh root@beaglebone.local 'cd /boot; ln -s uImage-3.2.25+ uImage'
host$ ssh root@beaglebone.local 'mount /dev/mmcblk0p1 /mnt'
host$ ssh root@beaglebone.local 'cp /boot/uImage-3.2.25+ /mnt/uImage'

== Install development environment v1 ==

These are older instructions. Use at your own risk.

The step is to get the crosscompilers, etc. installed. Generally you do this install on your local copy of Linux. Some of these installs can take a long time (one took me 12 hours, slow download times), so you may want to install on a remote server that's always connected to the internet. If so, see the tips [[ | below]]

=== Get the setup scripts ===

First install <code>git</code> by running the following on your host computer.

host$ '''sudo apt-get install git-core'''

(Here is a nice [http://progit.org/book/ git tutorial]. Take a look at it if you want to understand the commands that follow. We'll explore it in more depth in a later exercise.)

Then run the following to load the meta data.

I have everything in a <code>~/BeagleBoard</code>, so here is how I setup things:

host$ '''cd'''
host$ '''mkdir -p BeagleBoard'''
host$ '''cd BeagleBoard'''
host$ '''git clone git://github.com/Angstrom-distribution/setup-scripts.git oe'''
host$ '''cd oe'''

(The git took 4.5 seconds.)
This creates a directory for open embedded (<code>oe</code>) and gets the script to download Ångström. What files do you see?

=== Setting up for the BeagleBoard ===

Now let's setup <code>local.conf</code> for our needs:
$ gedit conf/local.conf
Add the following to the '''end''' of the file.
MACHINE ?= "beagleboard"
Look at this block of lines:
<pre>
# Make use of SMP and fast disks
PARALLEL_MAKE = "-j2"
BB_NUMBER_THREADS = "2"
</pre>
Here you can tell it how many parallel threads to run. If you have several cores on your machine, make this number big. If you have only one core, you might be better performance setting it to 1. If you have no clue which value to pick, Gentoo users suggest 1 more than the number of cores in your computer. More details are [http://wiki.openembedded.org/index.php/Advanced_configuration here].
* PARALLEL_MAKE sets the number "gcc" threads (same as make -j4 at compile time
* BB_NUMBER_THREADS sets the number of bitbake threads, (one thread can be downloading, while another compiles)

To save you a lot of time, it is useful to disable locale generation for all but the one you need. Add this to local.conf
GLIBC_GENERATE_LOCALES = "en_US.UTF-8 en_GB.UTF-8 de_DE.UTF-8 fr_FR.UTF-8 pt_BR.UTF-8 es_ES.UTF-8 kn_IN.UTF-8 ml_IN.UTF-8 ta_IN.UTF-8"

Now run:

host$ '''./oebb.sh config beagleboard'''

I got some errors; some things are missing
host$ '''ls -ls /bin/sh'''
0 lrwxrwxrwx 1 root root 4 Mar 29 14:53 /bin/sh -> dash
host$ '''sudo rm /bin/sh'''
host$ '''sudo ln -s /bin/bash /bin/sh'''
host$ '''ls -ls /bin/sh'''
0 lrwxrwxrwx 1 root root 9 Aug 30 10:38 /bin/sh -> /bin/bash
host$ '''sudo apt-get install gawk'''

Now try again
host$ '''./oebb.sh config beagleboard'''
host$ '''./oebb.sh update'''

This will bring in all the tools you need. The config took about 3.5 minutes on my machine, update took about 1 minute.

=== Build a small program ===

Before the first time you run bitbake you need to do the following to set up the environmental variables:
host$ '''source ~/.oe/environment-angstromv2012.05'''

Take a look in the file and see what it is doing.
host$ '''less ~/.oe/environment-angstromv2012.05'''

To see if everything is ready, run the following

host$ '''bitbake nano'''

nano is a small editor. We really don't need nano, but if it can be compiled we know we have everything in place. I get the following error when I run the bitbake:
<pre>
Pseudo is not present but is required, building this first before the main build
.
.
.
ERROR: Poky's config sanity checker detected a potential misconfiguration.
Either fix the cause of this error or at your own risk disable the checker (see sanity.conf).
Following is the list of potential problems / advisories:

Please install following missing utilities: diffstat,texi2html,makeinfo,cvs,svn,chrpath

ERROR: Execution of event handler 'check_sanity_eventhandler' failed
</pre>
So, some programs are missing. Install them with:
host$ '''sudo apt-get install diffstat texi2html texinfo subversion chrpath gettext'''
This took just a few minutes.

Now run:
host$ '''bitbake nano'''
This will take a while. Mine first said it has 77 tasks to do. Once those were done it had 1113 tasks.

Below is a table of the times from last year. Add your bitbake time to this year's table.
{| style="color:green; background-color:#ffffcc;" cellspacing="0" border="1" cellpadding="5"
|+ Initial <tt>bitbake nano</tt> Winter 2011-2012 runtimes
! Processor
! Settings
! Run Time
|-
| 8 Core virtual machine
| PARALLEL_MAKE = "-j8" BB_NUMBER_THREADS = "8" 
| 2.5 hours
|-
| Intel Core 2 Duo T7800, 2.60 GHz
| PARALLEL_MAKE = "-j2" BB_NUMBER_THREADS = "2" 
| 1:55
|-
| AMD Phenom(tm) II X3 720 Processor, 2.80 GHz
| PARALLEL_MAKE = "-j3" BB_NUMBER_THREADS = "3" 
| 1:44
|}
{| border="1" cellspacing="0" cellpadding="5"
|+ Initial <tt>bitbake nano</tt> 2011 runtimes
! Processor
! Settings
! Run Time
|-
| Dell E521 with AMD Athlon 64 dual-core
| PARALLEL_MAKE = "-j4" BB_NUMBER_THREADS = "4" Psyco? Yes.
| 1.5 hours
|-
| Intel Core 2 Duo T7800, 2.60 GHz
| PARALLEL_MAKE = "-j1" BB_NUMBER_THREADS = "1" Psyco? No.
| 1:46
|-
| Intel Core 2 Duo T7800, 2.60 GHz
| PARALLEL_MAKE = "-j4" BB_NUMBER_THREADS = "3" Psyco? Yes. Ubuntu 10.04 Running in VMWare Workstation 7.1.3 Under Windows 7 32-bit
| 1:42
|}

Once nano is done building you can find it with:
host$ '''find . -name nano'''
Can you figure out which one is the nano that will run on your Beagle? I found it here:

build/tmp-angstrom_v2012_05-eglibc/work/armv7a-angstrom-linux-gnueabi/nano-2.2.5-r2.0/package/usr/bin/nano

== Download and Compile the Kernel ==

The next step is to download and compile the kernel.

We want to keep the kernel source code around so we can experiment with it. Do this:
host$ '''gedit conf/local.conf'''
Find the line near the '''top''' that says <code>INHERIT += " rm_work "</code> and comment it out. This will save the source code.
# INHERIT += " rm_work

Exit gedit and do the following:

host$ '''source ~/.oe/environment-angstromv2012.05'''
host$ '''cd ~/BeagleBoard/oe'''
host$ '''bitbake virtual/kernel'''

Mine says it has 769 tasks to do. It took about 1 hour to do them. Once done you should get a kernel that will work.

If you accidentally run bitbake without commenting out the above line, after it runs you can comment out the INHERIT line and then run

host$ '''bitbake -c clean virtual/kernel'''
host$ '''bitbake -f -c compile virtual/kernel'''

This second run should take less time (one timing: first run took 3.5 hours; the recompiling took 25 minutes).

=== Finding the Correct Kernel ===

When you do the '''bitbake virtual/kernel''' you get the most recent kernel. How do you know which version you got? Look in the source directory.

host$ '''cd ~/BeagleBoard/oe; ls -F'''
build/ conf/ oebb.sh* pseudodone scripts/ sources/

Take a look in the '''sources''' directory.

host:~/BeagleBoard/oe$ '''ls -F sources'''
bitbake/ meta-ettus/ meta-nslu2/ meta-ti/
downloads/ meta-handheld/ meta-openembedded/ meta-xilinx/
layers.txt meta-intel/ meta-openpandora/ openembedded-core/
meta-angstrom/ meta-java/ meta-opie/
meta-browser/ meta-kde/ meta-raspberrypi/
meta-efikamx/ meta-mono/ meta-smartphone/

Here you see directories that contain information about various repositories that were used in pulling together all the sources used to build the kernel, u-boot, and the cross-compilers. Look around a bit an see what you can learn.

Let's see what was built when we made the kernel.

host:~/BeagleBoard/oe$ '''cd build; ls -F'''
sstate-cache/ tmp-angstrom_v2012_05-eglibc/

We are interested in the second directory. This is where the Angstrom build is kept, it's the 2012 version. Open Embedded can support other distributions and the software is set up so you can work with more than one distribution as the time. We're only working with Angstrom though.

The '''-eglibc''' tells us what libraries were used in making the build. We'll talk about libraries later. Look further:

host:~/BeagleBoard/oe/build$ '''cd tmp-angstrom_v2012_05-eglibc/; ls -F'''
abi_version ccache/ qa.log stamps/ work-shared/
buildhistory/ deploy/ saved_tmpdir sysroots/
cache/ pkgdata/ sstate-control/ work/

I'm not sure what all of these are for, but I know the cross-compilers are kept in '''sysroots'''. Go and explore. See if you can find them. We'll work with them in a later exercise. What we want is in the '''work''' directory:

host:~/BeagleBoard/oe/build/tmp-angstrom_v2012_05-eglibc$ '''cd work; ls -F'''
all-angstrom-linux/ beagleboard-angstrom-linux-gnueabi/
armv7a-angstrom-linux-gnueabi/ x86_64-linux/

These directories contain tools based on what they work on. '''x86_64-linux''', for example, has the tools that run on your host computer. '''armv7a-angstrom-linux-gnueabi''' has things that run on all arm processors. '''beagleboard-angstrom-linux-gnueabi''' has things that are Beagle specific, and that's where we find the kernel.

host:~/BeagleBoard/oe/build/tmp-angstrom_v2012_05-eglibc/work$ '''cd beagleboard-angstrom-linux-gnueabi/; ls -F'''
linux-mainline-3.2.18-r121b/

It looks like we have the 3.2.18 version of the kernel. Here's how you find out what kernel version have we've been running on the Beagle. On the Beagle run:

beagle$ '''uname -a'''
Linux beaglebone 3.2.5+ #1 Mon Feb 13 19:22:44 CET 2012 armv7l GNU/Linux

It looks like we're running the 3.2.5+ kernel. But what if you don't want the most recent kernel? You need to find the bitbake recipe for the kernel you are running. Here's how I did it, back to the host:

host$ '''cd ~/BeagleBoard/oe'''
host$ '''find . -name "*2.6.32*" | grep recipe'''
./sources/meta-ti/recipes-kernel/linux/linux-omap-psp-2.6.32
./sources/meta-ti/recipes-kernel/linux/linux-omap-psp_2.6.32.bb

The '''find''' command looks for files and directories that contain the string '''2.6.32''', the '''grep''' command then displays only those that contain the string '''recipe''' in the name. What you find is the the name of the recipe for building the kernel we're interested in. Now all you have to do is bitbake it. Be sure the '''conf/local.conf''' file is set as described earlier so that the sources aren't deleted.

Note: '''Don't do this just yet. I'm not sure which version we need.'''

host$ '''source ~/.oe/environment-angstromv2012.05'''
host$ '''bitbake linux-omap-psp-2.6.32'''

I think mine took a couple of hours.

== Download and Compile U-boot ==

While were' at it, let's get the boot loader we'll be using...

host$ '''bitbake u-boot'''

Mine took about 8 minutes.

Once installed you are ready for kernel work.

== Installing on a Remote Machine ==

Installing the cross development tools and the kernel on a laptop is nice, but sometimes the downloads are too long for such a portable device. I've had a bitbake run some 12 hours. Another option is to use a remote machine. In my case our CSSE department has created a virtual machine with Ubuntu 12.04 installed on it.

To install on a remote machine are the same as above; However here's a couple of tips to make it easier.

First, ssh to the remote machine. Assume the machine is called '''csse''' and your login name is '''beagle'''.

local$ ssh -CX beagle@csse

You can leave the '''beagle@''' off if you have the same login on both machine. The '''-C''' says to compress everything that's moved between the machines over the network. This is good for slow connections. The '''X''' says to pass the X11 display information to the remote machine. This way you can run graphical programs, such as gedit, on the remote machine and the graphics will display on your local machine (assuming you are running X11 on your local machine).

If you are '''ssh'''ing a lot check out the tip [[ECE497_Tips_and_Tricks#Suspending_ssh | here]] about suspending ssh and the tip [[ECE497_Tips_and_Tricks#Authorizing_ssh | here]] about authorizing ssh so you don't have to enter your password every time.

Once logged on to the removed machine run

remote$ sudo apt-get install byobu

This installs '''byobu''' which is a program that lets you connect to the same shell from multiple machines. Once installed run

remote$ byobu

You'll see something like:

[[File:Screenshot-beagle@ubu-may_-_byobu.png | 300px]]

So what? Run a couple of commands, like '''ls''' or '''who''', then hit '''F6'''. This will suspend your session. Now run '''byobu''' again. You'll be back in the same session. The session keeps running, even when you aren't connected.

Do you see the use?
* Fire up byobu
* Start a long bitbake
* Once you are sure it running OK, hit F6 and go home.
* From home fire up byobu and you'll see your bitbake (probably still running).
* You can F6 and check on it later.

Try opening another terminal and running byobu in both terminals. You'll see the same thing in both.

Run

$ man byobu

to see what it can do. Here's a list of shortcuts:

F2 - Create a new window
F3 - Move to previous window
F4 - Move to next window
F5 - Reload profile
F6 - Detach from this session
F7 - Enter copy/scrollback mode
F8 - Re-title a window
F9 - Configuration Menu
F12 - Lock this terminal
Ctrl-a $ - show detailed status
Ctrl-a R - Reload profile
Ctrl-a ! - Toggle key bindings on and off
Ctrl-a k - Kill the current window

But there is much more. Go and explore.

{{YoderFoot}}

← Older revision		Revision as of 12:14, 22 October 2012
Line 9:		Line 9:

	'''Tip:''' Run this exercise using a wired connection if you can. The Ubuntu wireless driver can be finicky, and if it stops working you'll have to restart some of this.		'''Tip:''' Run this exercise using a wired connection if you can. The Ubuntu wireless driver can be finicky, and if it stops working you'll have to restart some of this.
−
−	~~== Install development environment v2 ==~~
−	~~Follow these v2 instructions. They work for the 3.2.25 kernel.~~
−
−	~~=== Using Jason's instructions ===~~
−
−	~~These are notes on following instructions [https://github.com/beagleboard/kernel/blob/6682025752d0b807119c1e363a0b1b9bfe2ab453/README.md Beagleboard kernel git site].~~
−
−	~~host$ '''sudo apt-get install -y git lzop gcc-arm-linux-gnueabi uboot-mkimage'''~~
−
−	~~25 seconds~~
−
−	~~host$ '''git clone git://github.com/beagleboard/kernel.git'''~~
−
−	~~2.5 seconds~~
−
−	~~host$ '''./patch.sh'''~~
−
−	~~38.5 minutes~~
−
−	~~host$ '''make -j9'''~~
−
−	~~4 minutes~~
−
−	~~host$ '''make uImage'''~~
−
−	~~18 seconds~~
−
−	~~host$ '''make uImage'''~~
−	~~host$ '''mkdir ~/kernel/rootfs'''~~
−	~~host$ '''make INSTALL_MOD_PATH=~/kernel/rootfs modules_install'''~~
−	~~host$ '''cd'''~~
−
−	~~12 seconds~~
−
−	~~host$ '''scp kernel/kernel/arch/arm/boot/uImage root@beaglebone.local:/boot/uImage-3.2.25+~~
−	~~10 seconds~~
−
−	~~host$ '''cd kernel/rootfs'''~~
−	~~host$ '''find -H -depth \| cpio -o -H crc \| ssh root@beaglebone.local 'cd /; cpio -id' '''~~
−
−	~~1 minute 16 seconds~~
−
−	~~host$ '''cd'''~~
−	~~host$ '''ssh root@beaglebone.local 'cd /boot; rm uImage' '''~~
−	~~host$ ssh root@beaglebone.local 'cd /boot; ln -s uImage-3.2.25+ uImage'~~
−	~~host$ ssh root@beaglebone.local 'mount /dev/mmcblk0p1 /mnt'~~
−	~~host$ ssh root@beaglebone.local 'cp /boot/uImage-3.2.25+ /mnt/uImage'~~

	== Install development environment v1 ==		== Install development environment v1 ==

← Older revision		Revision as of 12:13, 22 October 2012
Line 299:		Line 299:

	Once installed you are ready for kernel work.		Once installed you are ready for kernel work.
−
−	~~== Installing on a Remote Machine ==~~
−
−	Installing the cross development tools and the kernel on a laptop is nice, but sometimes the downloads are too long for such a portable device. I've had a bitbake run some 12 hours. Another option is to use a remote machine. In my case our CSSE department has created a virtual machine with Ubuntu 12.04 installed on it.
−
−	~~To install on a remote machine are the same as above; However here's a couple of tips to make it easier.~~
−
−	~~First, ssh to the remote machine. Assume the machine is called '''csse''' and your login name is '''beagle'''.~~
−
−	~~local$ ssh -CX beagle@csse~~
−
−	You can leave the '''beagle@''' off if you have the same login on both machine. The '''-C''' says to compress everything that's moved between the machines over the network. This is good for slow connections. The '''X''' says to pass the X11 display information to the remote machine. This way you can run graphical programs, such as gedit, on the remote machine and the graphics will display on your local machine (assuming you are running X11 on your local machine).
−
−	If you are '''ssh'''ing a lot check out the tip [[ECE497_Tips_and_Tricks#Suspending_ssh \| here]] about suspending ssh and the tip [[ECE497_Tips_and_Tricks#Authorizing_ssh \| here]] about authorizing ssh so you don't have to enter your password every time.
−
−	~~Once logged on to the removed machine run~~
−
−	~~remote$ sudo apt-get install byobu~~
−
−	~~This installs '''byobu''' which is a program that lets you connect to the same shell from multiple machines. Once installed run~~
−
−	~~remote$ byobu~~
−
−	~~You'll see something like:~~
−
−	~~[[File:Screenshot-beagle@ubu-may_-_byobu.png \| 300px]]~~
−
−	So what? Run a couple of commands, like '''ls''' or '''who''', then hit '''F6'''. This will suspend your session. Now run '''byobu''' again. You'll be back in the same session. The session keeps running, even when you aren't connected.
−
−	~~Do you see the use?~~
−	* Fire up byobu
−	* Start a long bitbake
−	* Once you are sure it running OK, hit F6 and go home.
−	* From home fire up byobu and you'll see your bitbake (probably still running).
−	* You can F6 and check on it later.
−
−	~~Try opening another terminal and running byobu in both terminals. You'll see the same thing in both.~~
−
−	~~Run~~
−
−	~~$ man byobu~~
−
−	~~to see what it can do. Here's a list of shortcuts:~~
−
−	~~F2 - Create a new window~~
−	~~F3 - Move to previous window~~
−	~~F4 - Move to next window~~
−	~~F5 - Reload profile~~
−	~~F6 - Detach from this session~~
−	~~F7 - Enter copy/scrollback mode~~
−	~~F8 - Re-title a window~~
−	~~F9 - Configuration Menu~~
−	~~F12 - Lock this terminal~~
−	~~Ctrl-a $ - show detailed status~~
−	~~Ctrl-a R - Reload profile~~
−	~~Ctrl-a ! - Toggle key bindings on and off~~
−	~~Ctrl-a k - Kill the current window~~
−
−	~~But there is much more. Go and explore.~~

	{{YoderFoot}}		{{YoderFoot}}

@@ Line 10: / Line 10: @@
 '''Tip:''' Run this exercise using a wired connection if you can. The Ubuntu wireless driver can be finicky, and if it stops working you'll have to restart some of this.
-== Install development environment v1 ==
+== Install development environment ==
-These are older instructions.  Use at your own risk.
+These are the bitbake instructions.  Use at your own risk.
 The step is to get the crosscompilers, etc. installed. Generally you do this install on your local copy of Linux.  Some of these installs can take a long time (one took me 12 hours, slow download times), so you may want to install on a remote server that's always connected to the internet. If so, see the tips [[ | below]]