eLinux.org - User contributions [en]

Category:Resource Management

2008-02-14T13:28:34Z

Glenn: -linkspam

[[Category:Linux]]

Category:Resource Management

2008-02-13T21:41:43Z

Glenn: -linkspam

[[Category:Linux]]

Tool Chain

2008-02-12T09:19:38Z

Glenn: /* Code Sourcery */ -start space

This page has links to various tool chains and tool chain resources, which might
be of interest.

== Toolchain sites ==
=== General ===
==== Crosstool ====
Jim Wilson said the following on LKML:
I recommend Dan Kegel's page for anyone trying to build a cross compiler
to linux. See
http://kegel.com/crosstool
This isn't very hard to follow, and it gives you a properly configured
and built gcc/glibc for the target.

The University of Szeged has been doing benchmarking of GNU tools
across processor architectures, but including ARM. There's a site at
http://www.inf.u-szeged.hu/gcc-arm/

==== Debian ====
Debian has packages which support cross-compilation.
The following source packages can be used to build cross-compilers.
** http://packages.debian.org/testing/devel/toolchain-source
** http://packages.debian.org/testing/devel/toolchain-source-gdb
** http://packages.debian.org/testing/devel/toolchain-source-newlib

There are also (binary) library/headers packages, from
Debian packages for the target architectures, which support cross-compilation.

dpkg-cross is a tool for installing libraries and headers for cross compiling
in a way similar to dpkg. See:
** http://packages.debian.org/testing/utils/dpkg-cross

Please see /usr/share/doc/toolchain-source/README from the toolchain-source
package for more information.

=== Architecture-specific ===
==== ARM ====
You may wish to examine the following mailing list: linux-arm-toolchain@lists.arm.linux.org.uk

Several ARM toolchain combo's have been discussed often in the recent
past. List archives are at: http://lists.arm.linux.org.uk/pipermail/linux-arm-toolchain/

===== Building GCC 4.0 from scratch =====
* How to build a gcc 4.0 ARM toolchain from scratch: http://linux.omap.com/pipermail/linux-omap-open-source/2005-November/005665.html

===== [[Code Sourcery]] =====
You can now download ARM GNU tool chains (source and pre-built releases) from http://www.codesourcery.com where you will have regular builds of the tool chain, integrating support for new core functionality. The current release include support for [[ARM v6]] cores (binutils) and VFP support.

===== handhelds.org reference =====
Kristian S�rensen wrote:

We have had success using the ARM toolchain specified by handhelds.org. There
are both a binary available and a script for building your own.

A description of the toolchain and how to use it on a HP iPAQ is available in
our Master's Thesis in Appendix A (p 92). This may be downloaded here:
http://prdownloads.sourceforge.net/umbrella/Umbrella.pdf?download

Jamey Hicks wrote:
The handhelds.org toolchain binaries, sources, and build script are at ftp://ftp.handhelds.org/pub/linux/arm/toolchain/

The build script is actually crosstool 0.27, with slight changes I made to it for the particular selection of binutils, gcc, and glibc versions.

===== Greg Ungerer ARM multi-lib toolchain build =====
Greg writes:

Maybe this is interresting to some.
This is the instructions I put together for building a gcc-3.3.4
based ARM toolchain that is multi-lib-ed to be able to build for
all of big and little endian targets, and using either soft
or hard float.

http://ftp.snapgear.org/pub/snapgear/tools/arm-linux/build-arm-linux-3.4.4

Its nice just having one tool chain for all those varients
(I generate code for both little and big endian targets on a
daily basis, and everything from small non-mmu ARM7 cores to
xscale, all with the same tool chain).

== Toolchain downloads ==
Some member companies have provided sources and binaries for toolchains
they are using for forum work.

/\ These are all provided on terms of: "use at your own risk".

=== ARM and MIPS toolchains from Sony ===
- ''Note that this information is now out of date (for linux-2.4)''.
- How to install - [http://tree.celinuxforum.org/toolchains/INSTALL.txt INSTALL.txt]

- ARM toolchain - [http://tree.celinuxforum.org/toolchains/armv4tl-celf-linux-toolchain.tar.gz arvv4tl-celf-linux-toolchain.tar.gz] (22 MB)
- MIPS toolchain - [http://tree.celinuxforum.org/toolchains/mipsel-celf-linux-toolchain.tar.gz mipsel-celf-linux-toolchain.tar.gz] (25 MB)

- Sources:
- [http://tree.celinuxforum.org/toolchains/gcc-3.2.3.tar.gz gcc-3.2.3.tar.gz] (27 MB)
- [http://tree.celinuxforum.org/toolchains/gcc-patch.tar.gz gcc-patch.tar.gz] (29 KB)
- [http://tree.celinuxforum.org/toolchains/binutils-2.12.1.tar.bz2 binutils-2.12.1.tar.bz2] (9 MB)
- [http://tree.celinuxforum.org/toolchains/binutils-patch.tar.gz binutils-patch.tar.gz] (14 KB)

=== i386 and SH toolchains from Lineo Solutions ===
- i386 toolchain [http://tree.celinuxforum.org/toolchains/tools_i686_RPMS.tar.gz tools_i686_RPMS.tar.gz] (28 MB)
- SH toolchain [http://tree.celinuxforum.org/toolchains/tools_sh4_RPMS.tar.gz tools_sh4_RPMS.tar.gz] (24 MB)

- Sources:
- [http://tree.celinuxforum.org/toolchains/tools_i686_SRPMS.tar.gz tools_i686_SRPMS.tar.gz] (43 MB)
- [http://tree.celinuxforum.org/toolchains/tools_sh4_SRPMS.tar.gz tools_sh4_SRPMS.tar.gz] (40 MB)

- Packages for user-space programs:
- [http://tree.celinuxforum.org/toolchains/Userland_i686_RPMS.tar.gz Userland_i686_RPMS.tar.gz] (41 MB)
- [http://tree.celinuxforum.org/toolchains/Userland_sh4_RPMS.tar.gz Userland_sh4_RPMS.tar.gz] (47 MB)
- [http://tree.celinuxforum.org/toolchains/Userland_SRPMS.tar.gz Userland_SRPMS.tar.gz] (123 MB)

=== ARM/Thumb toolchain from Panasonic ===
- Software information regarding this package - [http://tree.celinuxforum.org/toolchains/PMCinfomation.txt PMCinformation.txt]
- How to install - [http://tree.celinuxforum.org/toolchains/PMCinstall.txt PMCinstall.txt]
- Sources:
- [http://tree.celinuxforum.org/toolchains/cross-gcc-3.3.1-7.0.24.0500655.src.rpm cross-gcc-3.3.1-7.0.24.0500655.src.rpm](31 MB)
- [http://tree.celinuxforum.org/toolchains/PMC.patch PMC.patch] (8 KB)
- Binaries:
- [http://tree.celinuxforum.org/toolchains/mvlcee3.1_thumb.for_export_050531.tgz mvlcee3.1_thumb.for_export_050531.tgz] (388 MB)

== Distributions ==
Some distributions provide toolchains as part of their offerings.
The following free/unsupported distributions are available.

==== Snap Gear ====
[http://www.snapgear.org SnapGear] provides a [[UCLinux]] distribution CD. On the web page, there is source and binaries for
toolchains for several architectures.

See http://www.uclinux.org/pub/uClinux/dist

==== ELDK ====
The Embedded Linux Development Kit (ELDK)is produced by
Wolfgang Denk. His company, Denx Software Engineering just released, as of November 2004,
a new version of the ELDK (release 3.1) for [[PowerPC]], ARM and MIPS systems. Allthough
it doesn't actively use 2.6 kernels yet, the toolchain can
be used for these, too.

See http://www.denx.de/twiki/bin/view/DULG/ELDK

==== [[Open Embedded]] ====
There is a project called OpenEmbedded which aims at making a free
embedded distribution (with build system).

See http://openembedded.org and the [http://www.openembedded.org/wiki OE wiki]

==== [[Tux Screen]] ====
TuxScreen and uClibc projects have build systems that generate complete toolchains, jffs2 filesytem images, bootloader etc.
* http://TuxScreen.net/
* http://cvs.sourceforge.net/viewcvs.py/tuxscreen/buildroot-tux/

==== uClibc ====
* http://uClibc.org/
* http://buildroot.uclibc.org/

== Tool Chains ==

The first thing you need for an [[Embedded Linux]] system is a [[Tool Chain]].

Here are random pointers to some resources:

* [http://www.kegel.com/crosstool/ CrossTools package]
* ftp://ftp.arm.linux.org.uk/pub/linux/arm/toolchain/
* http://www.emdebian.org/crossdev.html
* http://www.lart.tudelft.nl/lartware/compile-tools/
* http://handhelds.org/z/wiki/Toolchains
* ftp://ftp.handhelds.org/pub/linux/arm/toolchain/
* ftp://ftp.handhelds.org/pub/linux/arm/toolchain/source/
* ftp://ftp.handhelds.org/pub/linux/arm/toolchain/jacques
* ftp://ftp.handhelds.org/pub/linux/arm/toolchain/monmotha/
* http://www.armlinux.org/docs/toolchain/toolchHOWTO/x43.html
* http://buildroot.uclibc.org/
* http://www.denx.de/re/ELDK.html
* http://cvs.gentoo.org/~zwelch/xcompile.html

[[Category:Development Tools]]

Tool Chain

2008-02-12T09:17:37Z

Glenn: merged Tool Chains in Tool Chain

This page has links to various tool chains and tool chain resources, which might
be of interest.

== Toolchain sites ==
=== General ===
==== Crosstool ====
Jim Wilson said the following on LKML:
I recommend Dan Kegel's page for anyone trying to build a cross compiler
to linux. See
http://kegel.com/crosstool
This isn't very hard to follow, and it gives you a properly configured
and built gcc/glibc for the target.

The University of Szeged has been doing benchmarking of GNU tools
across processor architectures, but including ARM. There's a site at
http://www.inf.u-szeged.hu/gcc-arm/

==== Debian ====
Debian has packages which support cross-compilation.
The following source packages can be used to build cross-compilers.
** http://packages.debian.org/testing/devel/toolchain-source
** http://packages.debian.org/testing/devel/toolchain-source-gdb
** http://packages.debian.org/testing/devel/toolchain-source-newlib

There are also (binary) library/headers packages, from
Debian packages for the target architectures, which support cross-compilation.

dpkg-cross is a tool for installing libraries and headers for cross compiling
in a way similar to dpkg. See:
** http://packages.debian.org/testing/utils/dpkg-cross

Please see /usr/share/doc/toolchain-source/README from the toolchain-source
package for more information.

=== Architecture-specific ===
==== ARM ====
You may wish to examine the following mailing list: linux-arm-toolchain@lists.arm.linux.org.uk

Several ARM toolchain combo's have been discussed often in the recent
past. List archives are at: http://lists.arm.linux.org.uk/pipermail/linux-arm-toolchain/

===== Building GCC 4.0 from scratch =====
* How to build a gcc 4.0 ARM toolchain from scratch: http://linux.omap.com/pipermail/linux-omap-open-source/2005-November/005665.html

===== [[Code Sourcery]] =====
- You can now download ARM GNU tool chains (source and pre-built releases) from http://www.codesourcery.com where you will have regular builds of the tool chain, integrating support for new core functionality. The current release include support for [[ARM v6]] cores (binutils) and VFP support.

===== handhelds.org reference =====
Kristian S�rensen wrote:

We have had success using the ARM toolchain specified by handhelds.org. There
are both a binary available and a script for building your own.

A description of the toolchain and how to use it on a HP iPAQ is available in
our Master's Thesis in Appendix A (p 92). This may be downloaded here:
http://prdownloads.sourceforge.net/umbrella/Umbrella.pdf?download

Jamey Hicks wrote:
The handhelds.org toolchain binaries, sources, and build script are at ftp://ftp.handhelds.org/pub/linux/arm/toolchain/

The build script is actually crosstool 0.27, with slight changes I made to it for the particular selection of binutils, gcc, and glibc versions.

===== Greg Ungerer ARM multi-lib toolchain build =====
Greg writes:

Maybe this is interresting to some.
This is the instructions I put together for building a gcc-3.3.4
based ARM toolchain that is multi-lib-ed to be able to build for
all of big and little endian targets, and using either soft
or hard float.

http://ftp.snapgear.org/pub/snapgear/tools/arm-linux/build-arm-linux-3.4.4

Its nice just having one tool chain for all those varients
(I generate code for both little and big endian targets on a
daily basis, and everything from small non-mmu ARM7 cores to
xscale, all with the same tool chain).

== Toolchain downloads ==
Some member companies have provided sources and binaries for toolchains
they are using for forum work.

/\ These are all provided on terms of: "use at your own risk".

=== ARM and MIPS toolchains from Sony ===
- ''Note that this information is now out of date (for linux-2.4)''.
- How to install - [http://tree.celinuxforum.org/toolchains/INSTALL.txt INSTALL.txt]

- ARM toolchain - [http://tree.celinuxforum.org/toolchains/armv4tl-celf-linux-toolchain.tar.gz arvv4tl-celf-linux-toolchain.tar.gz] (22 MB)
- MIPS toolchain - [http://tree.celinuxforum.org/toolchains/mipsel-celf-linux-toolchain.tar.gz mipsel-celf-linux-toolchain.tar.gz] (25 MB)

- Sources:
- [http://tree.celinuxforum.org/toolchains/gcc-3.2.3.tar.gz gcc-3.2.3.tar.gz] (27 MB)
- [http://tree.celinuxforum.org/toolchains/gcc-patch.tar.gz gcc-patch.tar.gz] (29 KB)
- [http://tree.celinuxforum.org/toolchains/binutils-2.12.1.tar.bz2 binutils-2.12.1.tar.bz2] (9 MB)
- [http://tree.celinuxforum.org/toolchains/binutils-patch.tar.gz binutils-patch.tar.gz] (14 KB)

=== i386 and SH toolchains from Lineo Solutions ===
- i386 toolchain [http://tree.celinuxforum.org/toolchains/tools_i686_RPMS.tar.gz tools_i686_RPMS.tar.gz] (28 MB)
- SH toolchain [http://tree.celinuxforum.org/toolchains/tools_sh4_RPMS.tar.gz tools_sh4_RPMS.tar.gz] (24 MB)

- Sources:
- [http://tree.celinuxforum.org/toolchains/tools_i686_SRPMS.tar.gz tools_i686_SRPMS.tar.gz] (43 MB)
- [http://tree.celinuxforum.org/toolchains/tools_sh4_SRPMS.tar.gz tools_sh4_SRPMS.tar.gz] (40 MB)

- Packages for user-space programs:
- [http://tree.celinuxforum.org/toolchains/Userland_i686_RPMS.tar.gz Userland_i686_RPMS.tar.gz] (41 MB)
- [http://tree.celinuxforum.org/toolchains/Userland_sh4_RPMS.tar.gz Userland_sh4_RPMS.tar.gz] (47 MB)
- [http://tree.celinuxforum.org/toolchains/Userland_SRPMS.tar.gz Userland_SRPMS.tar.gz] (123 MB)

=== ARM/Thumb toolchain from Panasonic ===
- Software information regarding this package - [http://tree.celinuxforum.org/toolchains/PMCinfomation.txt PMCinformation.txt]
- How to install - [http://tree.celinuxforum.org/toolchains/PMCinstall.txt PMCinstall.txt]
- Sources:
- [http://tree.celinuxforum.org/toolchains/cross-gcc-3.3.1-7.0.24.0500655.src.rpm cross-gcc-3.3.1-7.0.24.0500655.src.rpm](31 MB)
- [http://tree.celinuxforum.org/toolchains/PMC.patch PMC.patch] (8 KB)
- Binaries:
- [http://tree.celinuxforum.org/toolchains/mvlcee3.1_thumb.for_export_050531.tgz mvlcee3.1_thumb.for_export_050531.tgz] (388 MB)

== Distributions ==
Some distributions provide toolchains as part of their offerings.
The following free/unsupported distributions are available.

==== Snap Gear ====
[http://www.snapgear.org SnapGear] provides a [[UCLinux]] distribution CD. On the web page, there is source and binaries for
toolchains for several architectures.

See http://www.uclinux.org/pub/uClinux/dist

==== ELDK ====
The Embedded Linux Development Kit (ELDK)is produced by
Wolfgang Denk. His company, Denx Software Engineering just released, as of November 2004,
a new version of the ELDK (release 3.1) for [[PowerPC]], ARM and MIPS systems. Allthough
it doesn't actively use 2.6 kernels yet, the toolchain can
be used for these, too.

See http://www.denx.de/twiki/bin/view/DULG/ELDK

==== [[Open Embedded]] ====
There is a project called OpenEmbedded which aims at making a free
embedded distribution (with build system).

See http://openembedded.org and the [http://www.openembedded.org/wiki OE wiki]

==== [[Tux Screen]] ====
TuxScreen and uClibc projects have build systems that generate complete toolchains, jffs2 filesytem images, bootloader etc.
* http://TuxScreen.net/
* http://cvs.sourceforge.net/viewcvs.py/tuxscreen/buildroot-tux/

==== uClibc ====
* http://uClibc.org/
* http://buildroot.uclibc.org/

== Tool Chains ==

The first thing you need for an [[Embedded Linux]] system is a [[Tool Chain]].

Here are random pointers to some resources:

* [http://www.kegel.com/crosstool/ CrossTools package]
* ftp://ftp.arm.linux.org.uk/pub/linux/arm/toolchain/
* http://www.emdebian.org/crossdev.html
* http://www.lart.tudelft.nl/lartware/compile-tools/
* http://handhelds.org/z/wiki/Toolchains
* ftp://ftp.handhelds.org/pub/linux/arm/toolchain/
* ftp://ftp.handhelds.org/pub/linux/arm/toolchain/source/
* ftp://ftp.handhelds.org/pub/linux/arm/toolchain/jacques
* ftp://ftp.handhelds.org/pub/linux/arm/toolchain/monmotha/
* http://www.armlinux.org/docs/toolchain/toolchHOWTO/x43.html
* http://buildroot.uclibc.org/
* http://www.denx.de/re/ELDK.html
* http://cvs.gentoo.org/~zwelch/xcompile.html

[[Category:Development Tools]]

Kernel Trace Systems

2008-02-11T21:17:15Z

Glenn: +cat

Here are some links to information about different kernel tracing systems:

== General Purpose tracing systems ==

Some major Linux general-purpose tracing systems are:
*ptrace - ability to trace syscall entry and exit, and signal delivery, to a process (also used for debugging a process)
** see "man ptrace" and "man strace"
* [[System Tap]] - System Tap is a system for building and executing tracing and sampling systems that can be applied to a running Linux system
*LTTng - [[Linux Trace Toolkit]], next generation
*LKST - [[Linux Kernel State Tracer]]

== Special Purpose tracing systems ==

There are some other notable special-purpose kernel tracing systems:
* KFT - [[Kernel Function Trace]] - traces functions to show function durations and call graphs
*latency trace - RT-preempt tool for measuring interrupt and mutex latency
**The latency tracer is embedded in the RT-preempt patch - see [[Realtime Preemption]] and [http://lwn.net/Articles/97811/ RT-preempt Article]
*block tracer (blktrace) - allows you to see exactly what is going on in the block layer for a given queue
**Introduction by Jens Axboe: [http://lwn.net/Articles/148761/ Introduction]
**Execellent presentation: [http://www.gelato.org/pdf/apr2006/gelato_ICE06apr_blktrace_brunelle_hp.pdf blktrace.pdf]
**Guide to using is at: [https://secure.engr.oregonstate.edu/wiki/CS411/index.php/Blktrace_Guide blktrace guide]
**This appears to have been mainlined as of 2.6.17
**Timeline utility (blktrace post-processing tool): [http://www.nabble.com/NEW%3A-btt---blktrace-timeline-utility%3A-analyze-I-Os-collected-with-blktrace.-tf1644874.html blktrace timeline utility]
* delay accounting patches - collect statistics about the delays that are experienced by each task on the system
** see [http://lkml.org/lkml/2006/5/2/30 delay accounting patches]

== Trace Infrastructure ==

*KProbes - grew out of dprobes, with information at: [http://dprobes.sourceforge.net/ dprobes]
**see an excellent tutorial at: [http://www-users.cs.umn.edu/~boutcher/kprobes/ kprobes]
**The mainline version of the KProbes supports x86,Alpha and PPC64 architectures. A MIPS implementation has been completed on the 2.6.16 kernel and tested on the Toshiba TX49 platform. Patch is available in the [[Patch Archive]].

* [would be nice to have some djprobe stuff here]

== Sampling Systems ==

Note that profile systems (or "sampling systems") are slightly different, in that they involve sampling instead of event tracing. Some major ones for Linux are:
* top - provides a dynamic real-time view of a running system, including processes
** see "man top"
** see also ksysguard, [http://freshmeat.net/projects/gnome-system-monitor/ Gnome system Monitor]
* OProfile - system-wide profiler for Linux systems
** see [http://oprofile.sourceforge.net/about/ oprofile]
** also: [http://www-128.ibm.com/developerworks/linux/library/l-oprof.html oprofile at IBM]
* !BootChart - samples bootup and provides visualization of process startup and system utilization
** see [[Boot Chart]]

== Related facilities ==

* in-kernel statistics infrastructure - proposal for a generic implementation of statistics facilities inside the kernel
** see [http://lkml.org/lkml/2006/5/19/106 inkernel stats]
* perfmon2 - interfaces to hardware performance monitoring features of the CPU
** see [http://perfmon2.sourceforge.net/ perfmon]
* inotify - [http://www-128.ibm.com/developerworks/linux/library/l-inotify.html inotify]

== Other Systems ==

Here are some systems I haven't classified yet:
* Datastreams - a system for creating and monitoring tracepoints - see [http://kusp.ittc.ku.edu/wiki/index.php/Main_Page datastreams]

== Collaboration Efforts ==

Some trace system project leaders are trying to collaborate: see [[Tracing Collaboration Project]]

[[Category:Linux tracing]]

Benchmark Programs

2008-02-11T21:16:24Z

Glenn: +Category:Development Tools

Here are some different programs for performing benchmarking.

<div style="margin:0; margin-top:10px; margin-right:10px; border:1px solid #dfdfdf; padding:0 1em 1em 1em; background-color:#ffffcc; align:right; ">
''Note: It is important to recognize that benchmarks between systems may be misleading.
Benchmarks should primarily be used to determine differences in performance for different
software configurations on the '''same''' hardware system.''
</div>
== Unix Bench ==
FYI, the URL to the UnixBench is as follows;

http://www.tux.org/pub/tux/benchmarks/System/unixbench/

UnixBench contains 9 kinds of tests:
# Dhrystone 2 using register variables
# Double-Precision Whetstone
# Execl Throughput
# File Copy
# Pipe Throughput
# Pipe-based Context Switching
# Process Creation
# Shell Script
# System Call Overhead

Unix Bench is preferred over lmbench since (according to reports) lmbench
cannot be easily cross-compiled.

I have cross-compiled lmbench - it's not too hard.
My target didn't have "make" or a compiler, but it did have bash.

Here is my recipe:
# Obtain lmbench. Make sure to get it from sourceforge (I used 3.0-a4), not bitmover, because the bitmover package is slightly tangled with a bitkeeper file "bk.ver". It's
# relatively easy to debug and disentangle it, but someone's already done that and put it up on sourceforge.
# Unpack lmbench in your build system.
# cd to lmbench-3.0-a4/src. Note that this is not the top level, which does have a Makefile.
# do "make OS=armv5EJl-linux-gnu CC=arm-linux-gcc" (or whatever your os and cross-compiler are)
# After everything is compiled, transport the whole directory tree to the target.
# cd to lmbench-3.0-a4/scripts
# Do ./config.run and answer some questions. This creates a config file that you can reuse.
# Do ./results
# Inside lmbench-3.0-a4/results will be a folder named armv5EJl-linux-gnu or something similar; inside that folder is a text file named imx21_199.0 or something similar.
# This is the raw lmbench results. Transport it back to civilization.
# When lmbench tries to save the results, it increments the last part ".0" until it finds an unused name. Therefore, you can rerun lmbench many times with a simple one-line
# bash script "for i in {1..100}; do ./results; done", and the result files will not overlap.
# You can get various kinds of summary postprocessing from lmbench. The "getsummary" script was sufficient for my purposes.
# To figure out which binary is generating which measurement, it may be useful to read the "lmbench" shell script in parallel with the raw results file.

== lmbench ==
The [[LMBench]] home page is at: http://www.bitmover.com/lmbench/
The sourceforge project page is at: http://sourceforge.net/projects/lmbench

[[Category:Development Tools]]

Linux Kernel Resources

2008-02-11T21:16:01Z

Glenn: +Category:Development Tools

This page has references to various kernel resources (web sites and mailing lists) for developers.
Most of this information was gathered over a year ago, and may not be accurate.

/\ ''Note: You should always look at the kernel MAINTAINERS file for up-to-date information''

== main (x86) kernel ==
- web site = http://www.kernel.org/
- [http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=summary Linus' Git Repository]
- kernel cross-reference online: http://glide.stanford.edu/lxr/source/ (only some version available)

=== Mailing List (lkml) ===
{|
|- bgcolor="80c0c0"
| align="center" | '''Item'''
| align="center" | '''Link'''
|-
| List E-mail Address
| linux-kernel@vger.kernel.org
|-
| Subscriptions (List Home)
| http://vger.kernel.org/majordomo-info.html
|-
| Archives (Aims Group)
| http://marc.theaimsgroup.com/?l=linux-kernel
|-
| Archives (lkml.org)
| http://lkml.org/
|-
| Archives (as a Google group)
| http://www.google.com/groups?hl=en&lr=&ie=UTF-8&group=linux.kernel
|-
| Archives (at Indiana University)
| http://www.ussg.iu.edu/hypermail/
|-
| Google search of Ind. Univ. archives of LKML
| [http://www.uwsg.iu.edu/search/?scope=kernel Search]
|-
| Summaries - (Kernel Traffic)
| http://kt.zork.net/kernel-traffic/index.html
|-
| Latest summary
| [http://kt.zork.net/kernel-traffic/latest.html latest kernel-traffic issue]
|}

=== Repository access ===
{|
|- bgcolor="80c0c0"
| align="center" | '''Item'''
| align="center" | '''Link'''
|-
| www.kernel.org
| http://www.kernel.org/git/
|}

=== News ===
- Linux Weekly News, Kernel page - http://lwn.net/Kernel/
=== Changelog ===
* Comprehensible changelog of the linux kernel
** http://wiki.kernelnewbies.org/LinuxChanges

* Announcemnts from linus
** 2.6.3 http://lwn.net/Articles/71670/
** 2.6.10 http://lwn.net/Articles/117188/
** 2.6.12 http://lwn.net/Articles/140441/
** 2.6.13 http://lwn.net/Articles/149480/
** 2.6.14 http://lwn.net/Articles/157474/
** 2.6.15 http://lwn.net/Articles/166131/

* LWN atricles for spcific releases
** 2.6.3 http://lwn.net/Articles/71669/
** 2.6.10 http://lwn.net/Articles/117187/
** 2.6.12 http://lwn.net/Articles/140165/
** 2.6.15 http://lwn.net/Articles/166130/

* LWN aricles on 2.6 API changes
** 2.6 API changes http://lwn.net/Articles/2.6-kernel-api/
** 2.6.12 API changes http://lwn.net/Articles/140164/

== Architecture Sites ==
=== MIPS ===
- web site = http://www.linux-mips.org/wiki/Main_Page
- mailing list = http://www.linux-mips.org/wiki/Net_Resources#Mailing_lists
- Maintainer = Ralph Baechle

- there's an alternate site on [[Source Forge]]
- the site is: http://sourceforge.net/projects/linux-mips
- Note that this is used for experimental stuff that hasn't been merged
into the official mips tree by Ralph Baechle

=== ARM ===
- web site = http://www.arm.linux.org.uk/
- cvs access = http://cvs.arm.linux.org.uk/
- mailing list = http://www.arm.linux.org.uk/armlinux/mailinglists.php
- wiki = http://www.linux-arm.org/
- Maintainer = Russell King

=== [[PowerPC]] ===
- web site = http://penguinppc.org/
- mailing lists = http://penguinppc.org/about/community.php#lists
- Git repository = kernel.org:/pub/scm/linux/kernel/git/paulus/powerpc.git
- Maintainer = Paul Mackerras
- [[Power Macintosh]] Maintainer = Benjamin Herrenschmidt

- cross-compiler mini-howto: http://penguinppc.org/embedded/cross-compiling/

See the following for information on different linuxppc source trees available:
http://www.penguinppc.org/dev/kernel.shtml
=== SuperH (SH) ===
- web site = http://www.linux-sh.org/
- Git repository = kernel.org:/pub/scm/linux/kernel/git/lethal/sh-2.6.git
- mailing list address = linux-sh@vger.kernel.org
- mailing list page = http://vger.kernel.org/vger-lists.html#linux-sh
- mailing list archives = http://news.gmane.org/gmane.linux.ports.sh.devel
- wiki = http://linux-sh.org/shwiki/FrontPage
- Maintainer = Paul Mundt

== Documentation ==
- Linux Device Drivers - 3rd Edition, online - http://lwn.net/images/pdf/LDD3/
- Rusty Russell's "Unreliable Guide to Locking" - http://kernelbook.sourceforge.net/kernel-locking.html

== Cross-reference / code online ==
* http://www.linux-m32r.org/lxr/http/source/
* http://lxr.free-electrons.com/
* http://sosdg.org/~coywolf/lxr/source/
* http://lxr.linux.no/source/

[[Category:Development Tools]]

Kernel Debugging Tips

2008-02-11T21:15:30Z

Glenn: +Category:Development Tools

Here are some miscellaneous tips for debugging a kernel:

== Accessing the printk buffer after a silent hang on boot ==
Quinn Jensen writes:

Something I've found handy when the console is
silent is to dump the printk buffer from the boot
loader. To do it you have to know how your boot
loader maps memory compared to the kernel. Here's
what I do with Redboot on i.MX31:

fgrep printk_buf System.map

this shows the linked address of the printk_buf, e.g.:

c02338f0 b printk_buf.16194

The address "c02338f0" is in kernel virtual, which,
in the case of i.MX31 ADS, redboot will have mapped
to 0x802338f0. So, after resetting the target board,
but without letting it try to boot again, at the redboot
prompt,

dump -b 0x802338f0 -l 10000

And you see the printk buffer that never got flushed
to the UART. Knowing what's there can be _very_
useful in debugging your console.

[[Category:Development Tools]]

Category:TCT-Hammer

2008-02-11T21:10:47Z

Glenn: catspec

TinCanTool Hammer Board related pages

[[category:TinCanTools]]
[[Category:ARM Development Boards]]

UsingJtagTools

2008-02-10T21:52:26Z

Glenn: +cat

'''Note''': This assumes that you use the default locations for the jtag tools 0.6 install (HEAD from cvs). Please adjust your paths accordingly

== Add Support for the Juicebox ==
=== 0.5.1 ===
here is a [attachment:jtag-jb.diff patch] against the 0.5.1 release. it would be nice if someone made the changes against the cvs repo head and submit it to the openwince folks. There's been some version slippage, and you'll need v0.3.2 of the OpenWinCE Includes to compile jtag 0.5.1.

=== 0.6 ===
here is an [attachment:jtag-jb-cvs-alpha.diff alpha patch] against version 0.6 (cvs head). please try this out and post/fix issues you see with it.

* Install OpenWinCE Includes
* Get the latest code from cvs
* Apply the patch
* Run "autogen.sh"
* make && make install
Get the latest Jtag Tools from the OpenWinCE project CVS Repo - http://openwince.sourceforge.net/jtag/

=== Manually Add Support ===
'''Note:''' This is by far the least desirable method

Using the attached BSDL on ["JuiceBoxJTAG"] file you can create the necessary files for jtag tools using the included '''bsdl2jtag'''.

<pre>
#Convert the BSDL file by piping in the
#bsdl file and piping the stdout to a file
bsdl2jtag < s3c44b0x.bsdl > 0001
#Modify the MANUFACTURERS file
echo 11110000111 juicebox juicebox >> /usr/local/share/jtag/MANUFACTURERS
#Create the juicebox device
mkdir -p /usr/local/share/jtag/juicebox/s3c44b0x
mv 0001 /usr/local/share/jtag/juicebox/s3c44b0x/
echo 1111000011110000 s3c44b0x S3C44B0X > /usr/local/share/jtag/juicebox/PARTS
echo 0001 0001 0001 > /usr/local/share/jtag/juicebox/s3c44b0x/STEPPINGS
</pre>
I am having issues doing anything useful since I do not have the correct bus driver, or don't know which to use. If you are using jtag tools with the juicebox please post some details on the bus driver.

== Using Jtag Tools ==
You can now start jtag tools (/usr/local/bin/jtag) and detect the device

This example uses a Xilinx Parallel Cable III (Model DLC5)

<pre>
jtag> cable parallel 0x378 DLC5
jtag> detect
jtag> print chain
</pre>
Print out the current pin values

<pre>
jtag> instruction SAMPLE/PRELOAD
jtag> shift ir
jtag> shift dr
jtag> dr
</pre>
Initialize the Bus (to peek/poke/copy memory and the other good stuff) and ID the cart

<pre>
jtag> initbus s3c44b0x
*no cart*
jtag> peek 0x00590000
bus_read(0x00590000) = 0x0000CE0C (52748)
*video cart*
jtag> peek 0x00590000
bus_read(0x00590000) = 0x00001880 (6272)
</pre>
'''Note:''' Here I am assuming that the cart id is only 32 bits but I don't know that to be the case

== Useful Scripts for Jtag Tools ==
This script will set up a Xilinx Parallel Cable III (DLC5), detect the juicebox, print out the jtag chain, and initialize the juicebox jtag driver. The help screen is just to show you what other commands are available.

'''init'''

<pre>
cable parallel 0x378 DLC5
detect
print
initbus s3c44b0x
help
</pre>
This script is kinda useless but I wanted to show that you can call one script from another. This probably can be done with "include" as well but I haven't that command yet. '''id_cart'''

<pre>
script /home/antics/jb/init
peek 0x00590000
</pre>

[[Category:Development Tools]]

MMC

2008-02-10T21:49:53Z

Glenn: catcorr

Projects using MMC Cards:

* http://homepage.ntlworld.com/seanellis/mmcserial.htm
* http://instruct1.cit.cornell.edu/courses/ee476/FinalProjects/s2000/peterdan/final.htm
* http://www.compsys1.com/workbench/On_top_of_the_Bench/MMC_Project/mmc_project.html
* http://www.captain.at/electronics/pic-mmc/
* http://www.microchipc.com/sourcecode/#mmc
* http://www.msx.org/forumtopic3754.html

Documents:

* mmc specification from sandisk [[Media:mmc_spec.pdf]]
* mmc i/o example from sharp [[Media:mmc_example.pdf]]
* web site dedicated to mmc/sd information http://mmc.drzeus.cx/wiki/

Prototype mmc sockets:

* [[JuiceBox MMC]]
* http://www.futurlec.com/Mini_SC.shtml
* http://www.sparkfun.com
[[Image:mmc-breakout.jpg]]

[[Category:Hardware]]

MMC

2008-02-10T21:49:36Z

Glenn: +Category:Community

IrDA

2008-02-10T21:48:53Z

Glenn: +cat

Infrared Data Association

Infrared Data Association (IrDA) defines physical specifications communications protocol standards for the short-range exchange of data over infrared light, for uses such as personal area networks (PANs).

[[Category:Hardware]]

CELF Forum Doc Resources

2008-02-10T21:47:29Z

Glenn: +Category:Community

Here are some logos and doc templates.

== Logos ==
* [[Media:final_CELinuxForumLogo_long.eps]] - scalable [[CELinux]] forum title with penguin (long and narrow)
* [[Media:final_CELinuxForumLogo.eps]] - scalable [[CELinux]] forum title with penguin (basically square)
* [[Media:final_penguin_only.eps]] - scalable [[CELinux]] penguin logo (square, but with no wording)
* [[Media:final_CELinuxForumLogo_b_w.eps]] - scalable [[CELinux]] penguin logo (square, one-color)

== Document templates ==
* [[Media:CELinuxForumBlank.ppt]] - blank PowerPoint file

== Advertisements ==
* [[Media:CELinuxForum_AD1.eps]] - Advertisement for Ottawa Linux Symposium program

[[Category:Community]]

Community Participation Guides

2008-02-10T21:45:47Z

Glenn: +Category:Community

Here are some excellent resources for working more effectively in the open source community:
* [http://www.catb.org/~esr/faqs/smart-questions.html How To Ask Questions The Smart Way] - a guide by Eric S. Raymond about asking questions in community forums

[[Category:Community]]

Board and Chip Vendors

2008-02-10T21:44:35Z

Glenn: catspec

This page has a list of the companies or organizations that make processors or boards for
embedded products. If you are looking for companies who sell Linux software or Linux-related services,
see the [[Vendors]] page. If you are looking for companies who sell end-user products based on
Linux, see [[Companies]]

== F ==
* [http://www.freescale.com/ Freescale Semiconductor] - [http://en.wikipedia.org/wiki/Freescale_Semiconductor Wikipedia entry]
** Freescale makes the MX31 ARM11-based processor (and associated development boards), among others.
** Freescale makes several PPC-based processors (and associated development boards) as well.

== R ==
* [http://www.renesas.com/ Renesas Technology] - [http://en.wikipedia.org/wiki/Renesas_Technology Wikipedia entry]
** Renesas makes the SuperH, M32R, and H8 RISC CPUs, the RX CISC CPUs, and others.

[[Category:Hardware]]

Compact Flash

2008-02-10T21:44:03Z

Glenn: +cat

CompactFlash (CF) was originally developed as a type of data storage device used in portable electronic devices. For storage, CompactFlash typically uses flash memory in a standardized enclosure. This form was first specified and produced by SanDisk in 1994. The physical format is now used for a variety of devices.

[[Category:Hardware]]

Filesystem Information

2008-02-10T21:42:44Z

Glenn: catspec Category:File Systems

== Boot Time with various file systems ==
Noboru Wakabayashi of Hitachi provided the following report.

On the OMAP Innovator he built the following file systems:
*CRAMFS
*CRAMFS with XIP
*ROMFS
*JFFS2
*ext2 over RAM disk

He measured the time to initialize these file systems by logic analyzer. This was done by modifying the busybox init program to make LED turn on red. When innovator power on, the LED lights up green. So the time from the light turning from green to red was measured. Also, he measured the time using KFI (from start_kernel() to to_usrspace()).

Results are shown in the following table.
(The result is average of 10 trials for each configuration.)

{| border="1" cellpadding="5" cellspacing="0" align="center"
! style="background:#80C0c0;"|'''Configuration/Filesystem'''
! style="background:#80C0c0;"|'''logic analyzer(sec)'''
! style="background:#80C0c0;"|'''KFI(usec)'''
|-
|CRAMFS* || 3.638850 || 842789.1
|-
|CRAMFS with XIP || 2.788076 || 764141.3
|-
|CRAMFS with XIP and PLPJ || 2.583988 || 551735.5
|-
|ROMFS || 3.510876 || 839078.2
|-
|JFFS2* || 4.822642 || 1241068.4(log full)
|-
|ext2 over RAM disk || cannot measure || 2952081.6(log full)
|}

*JFFS2: JFFS2 required much time at first boot time, so he measured from the 2nd starting.

*CRAMFS: At first, he also measured the time with CONFIG_KFI by logic analyzer. The result is 4.324660 sec. It costs more than without CONFIG_KFI. So, he measured file systems without CONFIG_KFI when he used logic analyzer.

The attached zip file has the kfi logfiles for these different tests:
no zip found: kfi-results-omap-filesystems.zip

Next he remeasured the time to initialize "CRAMFS with XIP and PLPJ" using the "quiet" option.
The result is 280676 usec from start_kernel() to to_userspace().
The above result is 551735.5 usec.The time is reduced about 50%.

The following table shows output from 'kd' on the kfi logfile.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|+ output from 'kd' on the kfi logfile
! style="background:#D0E0FF;"|'''Function'''
! style="background:#D0E0FF;"|'''Count'''
! style="background:#D0E0FF;"|'''Time'''
! style="background:#D0E0FF;"|'''Average'''
! style="background:#D0E0FF;"|'''Local'''
! style="background:#D0E0FF;"|'''Max-sub'''
! style="background:#D0E0FF;"|'''Ms count'''
|-
|do_basic_setup||1||114068||114068||509||do_initcalls||1
|-
|mem_init||1||110376||110376||490||free_all_bootmem_node||1
|-
|free_all_bootmem_node||1||109378||109378||12||free_all_bootmem_core||1
|-
|free_all_bootmem_core||1||109366||109366||109366|| - ||0
|-
|schedule||10||84482||8448||34||do_schedule||10
|-
|do_schedule||10||84448||8444||574||switch_to||9
|-
|do_initcalls||1||84159||84159||3831||device_init||1
|-
|switch_to||9||83874||9319||83874|| - ||0
|-
|register_proc_table||22||39161||1780||13079||register_proc_table||18
|-
|device_register||11||22297||2027||415||device_add||11
|-
|device_add||11||21882||1989||1439||kobject_add||11
|-
|device_init||1||20633||20633||30||net_dev_init|| 1
|-
|tifb_init||1||18759||18759||844||register_framebuffer||1
|-
|register_framebuffer||1||13092||13092||88||take_over_console||1
|-
|take_over_console||1||13004||13004||819||redraw_screen||1
|-
|kobject_add||15||12996||866||738||create_dir||15
|-
|setup_arch||1||12542||12542||131||paging_init||1
|-
|paging_init||1||12411||12411||386||free_area_init_node||1
|-
|create_dir||15||12258||817||3625||populate_dir||9
|-
|free_area_init_node||1||12025||12025||30||free_area_init_core||1
|-
|free_area_init_core||1||11995||11995||7496||__alloc_bootmem_node||1
|-
|rs_init||1||11794||11794||377||printk||3
|-
|inet_init||1||11696||11696||1718||ip_init||1
|-
|redraw_screen||2||11247||5623||871||do_update_region||1
|-
|printk||18||10870||603 ||10870||-||0
|-
|net_dev_init||1||10334||10334||3102||ethif_probe||1
|}

= Probe times for various file systems =
As part of work supported by Sony/Matsushita, Todd Poynor got the following numbers using KFI on a 200MHz IBM 405GP "Walnut" evaluation board with a 100MHz core bus and 33MHz PCI bus. A Seagate Barracuda ATA IV 60GB disk drive is cabled to one of the two IDE interfaces on a Promise Ultra66 PCI-IDE bridge card (PDC20262 chipset). All of the drivers for PCI, IDE, PCI-IDE disk, and EXT2 filesystem are built into the kernel.

Boot execution time of IDE/PCI-IDE/MS-DOS partition drivers: 262 msecs. This includes the time to probe and identify the IDE drive device and read the disk partition information from the drive. We booted the kernel with option hdf=none to turn off the slave device on interface
ide2, so that it would not be probed. We also modified the kernel to turn off probes of the second IDE interface on the Promise card. (This was prior to fixing the "ide<x>=noprobe" option bug. If you don't turn off probing the second empty IDE interface then probling takes 1.3 seconds on both a PPC 405GP and a MIPS ITE8172!)

About 250 msecs of the time is spent during the bus probe in repeated calls to ide_delay_50ms() during probe and drive identification, which busywaits (in order to let the IDE controller make progress before polling for status or to allow previous operations to complete). Reading capacity info, etc. also blocks using a wait_for_completion(). The MSDOS partition code also locks pages, which can call schedule() to wait for locks.

If the IDE drivers are made modules for delayed initialization, allowing concurrent module initialization with application execution, then kernel preemption is turned off for about 252 msecs during init of the ide-probe-mod module, which could significantly interfere with real-time response of other threads. (This was verified using the CONFIG_PREEMPT_TIMES feature that gives preemption lock times in /proc/latencytimes, which is also supported in the CELF kernel.)
Because the Big Kernel Lock (BKL) is held during module initialization, preemption is disabled while the module init routines runs and uses busywait calls, but preemption is allowed when CPU-yielding wait calls are employed (the linux scheduler drops and reacquires the BKL in this
case).

So we changed the ide_delay_50ms() busywaits to call schedule_timeout() instead (this is also in the CELF kernel; select CONFIG_IDE_PREEMPT), resulting in a 9.68 msec maximum preempt off time. Note that if you're not using modules but are instead building the drivers statically into the kernel, then the CPU-yielding calls do add some amount of time to the total execution time due to context-switch overhead, etc.

My coworker Dave Singleton also did some analysis and improvement of IDE on the MIPS ITE8172 (again for Sony/Matsushita). He found that with his 7200RPM Maxtor drive, he could reduce the 50ms probe delays to 1ms with no problem. With this, plus some context switch removal and the other optimizations given above, the following boot times were observed, by filesystem type:

{{{
ext2: 167 milliseconds
ext3: 457 milliseconds
xfs: 236 milliseconds
}}}

He explains: "Both EXT3 and XFS file systems cause a log replay at boot/mount time. To improve this time the log recovery feature was by passed in the case of XFS. The log was not replayed and the root file system was mounted readonly. The first init script after booting remounted the root file system readwrite and replayed the log to ensure file system integrity. No such changes were made to EXT3, which is the reason it had the slowest boot times of all 3 file system types."

[[Category:File Systems]]

ELinuxWiki:Glossary

2008-02-10T21:40:34Z

Glenn: +Category:Community

This is a glossary of terms used in embedded Linux products, and links to existing glossary pages:

Here are pages that have list of terms for specific technology areas:
* [[Boot-up Time Definition Of Terms]] - terms related to the Linux boot-up process
* [[Power Management Definition Of Terms]] - Definition of Terms for the CELF Power Management working group
* [[Real Time Terms]] - terms related to systems with real-time performance
* [[Security Terms]] - terms related to Linux security and security frameworks

[[Category:NeedsEditing]]
[[Category:Community]]

ELinuxWiki:Irc

2008-02-10T21:40:21Z

Glenn: +Category:Community

= IRC Channels =
==Freenode irc.freenode.net==
{|
|-valign="top"
| #eLinux|| IRC for the eLinux.org community ([http://ibot.rikers.org/%23elinux/ Archives]).
|-valign="top"
|#edev||Discussion for small embedded device development and hacking.
|-valign="top"
|#openezx||Discussion about Motorola's EZX phone platform, such as the A780, E680 and E680i Linux based GSM phones.
|-valign="top"
|#dlinkhacking||This is the place to discuss software and hardware hacking of ALL DLink products
|-valign="top"
|#linux-sh||Linux on SuperH(SH) Devices.
|-valign="top"
|#mipslinux||Linux on MIPS Devices.
|-valign="top"
|##electronics||General electronics discussion.
|-valign="top"
|#oe||Channel for discussion of OpenEmbedded development environment [http://www.openembedded.org OE Homepage].
|-valign="top"
| #davinci||Discussions regarding the TI Davinci platform and EVM related hardware.
|-valign="top"
| #openmoko||Open Source Telephony
|-valign="top"
| #maemo||Nokia Internet Tablet OS
|-valign="top"
| #neuros|| Neuros OSD Hacking
|-valign="top"
| #usdtv|| USDTV/Hisense HDTV Receiver Hacking
|-valign="top"
| #u-boot||U-boot bootloader
|-valign="top"
| #openjtag||where the openocd folks hang out
|-valign="top"
| #fedora-arm||Fedora on ARM based processors
|}

==OFTC irc.debian.org==
{|
|-valign="top"
| #emdebian||Debian-based embedded and crosstools stuff, and emdebian distribution
|-valign="top"
| #debian-arm||Debian ARM port
|}

[[Category:Community]]

Help:Contents

2008-02-10T21:40:04Z

Glenn: +Category:Community

==Policy and Content Licensing==

'''Policy'''<br>
* This site is wholly intended as a public site created by and for the Embedded Linux Community. Content posted herein will comply with the following policies:<br>

* A wiki author shall retain rights and ownership to any material posted to this wiki that they currently hold the rights and ownership for.<br>

* It is the responsibility of a content author to understand the legalities of the content they are posting. The Editor(s) reserve the right to remove content that is found to not comply with any of the policies, guidelines or intents found herein. This may include content that is deemed offensive, illegal, infringing on copyrighted material, or out of scope of the topic of Embedded Linux.<br>

* This wiki is considered a Neutral location for content. This means that no sponsoring interest can or will retain rights over any content with the exception of content to which the party already retained rights.<br>
'''Licensing'''<br>
* Unlicensed content is any post to the wiki that does not either explicitly (via declaration) or implicitly (through inheritance) carry a license already.<br>
* Unlicensed content will default to one of the following licenses depending on it's nature:<br>
** The GNU Free Document License (GFDL) - This license applies to all unlicensed content that is NOT code.<br>
** The GNU General Public License, Version 2 (GPLv2) - This license will apply to all unlicensed code or code snippets.<br>
* Prior Licensed Content<br>
** Prior licensed Code posted to the wiki must carry a license in compliance with the Open Source Initiative's (OSI) definition of an open source license.<br>
**Code patches posted to the wiki will carry the license of the code to which the patch applies.<br>

==Addition Inherited Policy==
<br>
NOTE: The following eLinux Wiki Policies and Guidelines borrow heavily from the Wikipedia Policies and Guidelines.

You don't need to read every eLinux Wiki policy before you contribute. However, the following policies are particularly important to the project, and the sooner you understand and use them, the better:
# '''eLinux Wiki is not an encyclopedia.''' Although content of an encyclopedic nature may appear here on occasion, it will generally be in support of other content. Content of an encyclopedic nature should be added to Wikipedia and linked back here.
# '''Respect other contributors.''' eLinux Wiki contributors come from many different countries and cultures, and have widely different views. Treating others with respect is key to collaborating effectively in building an this resource.
# '''Don't infringe copyrights.''' eLinux Wiki is a free developer resource licensed under the terms of the [http://www.gnu.org/copyleft/fdl.html GNU Free Documentation License]. Submitting work which infringes copyrights threatens our objective to build a truly free resource that anyone can redistribute and could lead to legal problems.
# '''eLinux Wiki encourages original research.''' As the central resource for leading edge Embedded Linux work, you are encouraged to submit your research, test results, etc. to the Embedded Linux community through this wiki.

== Guidelines ==
There is a [[Quickstart Guide]] for using and contributing to this site.

In addition to the generally accepted policies listed above, a very large number of [http://en.wikipedia.org/wiki/Wikipedia:List_of_guidelines Wikipedia Guidelines] have been adopted by eLinux Wikipedians. These are used to provide guidance in various situations that arise on Wikipedia. They cover everything from naming conventions and sensitive terms that should be avoided to how to get along, and why not to bite the newcomers.

The eLinux Wiki adheres to Wikipedia guidelines, with the exception of the following: ??

[[Category:Community]]

ELinuxWiki:Mailing List

2008-02-10T21:39:51Z

Glenn: +Category:Community

If you want to discuss this wiki with its primary maintainers and editors, please join the list below:
* [http://tree.celinuxforum.org/mailman/listinfo/elinux-wiki elinux-wiki]

Archives of this list are available at:
* http://tree.celinuxforum.org/pipermail/elinux-wiki/

[[Category:Community]]

Help:Editing

2008-02-10T21:39:40Z

Glenn: +Category:Community

This Editing Overview has a lot of examples. You may want to keep this page open in a separate browser window for reference while you edit. If you want the super-quickie help, see the [[Quickstart Guide]].

Each of the topics covered here is covered somewhere else in more detail. See box at right for that
== Editing basics ==
;Start editing
:To start editing a [[w:MediaWiki|MediaWiki]] page, click the '''Edit this page''' (or just '''edit''') link at one of its edges. This brings you to the '''edit page''': a page with a text box containing the ''[[w:wikitext|wikitext]]'': the editable source code from which the server produces the webpage. ''If you just want to experiment, please do so in the [[{{ns:4}}:Sandbox|sandbox]], not here''.

;Type your changes
:You can just type your text. However, also using basic wiki markup (described in the next section) to make links and do simple formatting adds to the value of your contribution.

;Summarize your changes
:Write a short [[Help:Edit summary|edit summary]] in the small field below the edit-box. You may use shorthand to describe your changes, as described in the [[{{ns:4}}:Edit summary legend|legend]].

;Preview before saving
:When you have finished, click '''[[Help:Show preview|Show preview]]''' to see how your changes will look -- '''before''' you make them permanent. Repeat the edit/preview process until you are satisfied, then click '''Save page''' and your changes will be immediately applied to the article.

{{:Help:Wikitext quick reference}} 

== Minor edits ==

A [[Help:Logging in|logged-in]] user can mark an edit as "minor". [[Help:Minor edit|Minor edit]]s are generally spelling corrections, formatting, and minor rearrangement of text. Users may choose to ''hide'' minor edits when viewing [[Help:Recent changes|Recent Changes]].

Marking a significant change as a minor edit is considered bad Wikiquette. If you have accidentally marked an edit as minor, make a [[Help:Dummy edit|dummy edit]], verify that the "<small>'''[ ] This is a minor edit'''</small>" check-box is unchecked, and explain in the edit summary that the previous edit was not minor.

==See also==
*[[Help:Editing FAQ]]
*[[Help:Automatic conversion of wikitext]]
*[[Help:Calculation]]
*[[Help:Editing toolbar]]
*[[Help:HTML in wikitext]]
*[[Help:Administration#Page protection|Protecting pages]]
*[http://wikipedia.org/wiki/Wikipedia:Cheatsheet Wikipedia:Cheatsheet]—a listing of the basic editing commands.
*[[Help:Starting a new page]]
*[[Help:Variable]]
*[[w:HTML element|HTML elements]].
*[[Wikipedia:Manual of Style]]
{{h:f|enname=Editing}}

[[Category:Community]]

Help:About

2008-02-10T21:39:26Z

Glenn: +Category:Community

This is the embedded Linux wiki.

The purpose of this wiki is to preserve and present information about using Linux in embedded systems.
The primary target audience is developers, engineers, managers, hobbyists and others who are involved
with putting Linux into embedded systems.

Please utilize this information in your own development efforts, and please provide back
information when you can. See [[Volunteer editor tasks]] to get started, if you don't know where to begin.

== Sponsors ==
This site is sponsored by the CE Linux Forum, with hosting provided by Movial. [[User:Wmat|Bill Traynor]]
is the primary administrator and maintainer of this site.

== History ==
This site was created in late 2006 and early 2007, and opened to the public in June 2007.
The major initial content for this site came from merging content from two previous sites:
* the original elinux.org site
* the CELF public wiki

The original elinux.org site had a lot of content about putting Linux onto existing
products (often, products which were not shipped with Linux or designed to support it).

The CELF public wiki has a collection of resources, including research results, collections
of links to interesting articles, and many presentations from CELF and other conferences.

In 2006, it was decided to create a larger site with a combination of the two sets of resources.
CELF hired a part-time administrator to work on this, and Movial provided hosting for the site.

[[Category:Community]]

Category:HOWTOs

2008-02-10T21:38:45Z

Glenn: new

[[Category:Categories]]

Squash FS Howto

2008-02-10T21:38:22Z

Glenn: catchg

==SquashFS HOWTO==

Artemiy I. Pavlov

Artemio.net

<artemio (at) artemio (dot) net>

2004-06-07
Revision History
Revision 1.5 2004-06-07
Changes according to SquashFS release 2.0 alpha. Lots of
description improvements and clarifications. Split
instructions for Linux kernels of 2.6.x (new) and 2.4.x
series.
Revision 1.1 2004-05-22
Changes according to SquashFS release 1.3r3.
Revision 1.0 2004-02-19
Initial Release, reviewed by LDP.
Revision 0.2 2003-12-08
Text corrections, license added.
Revision 0.1 2003-11-24
Initial version. Instructions for SquashFS release 1.3r2.

Abstract

This HOWTO describes the usage of SquashFS - a highly-compressed read-only file system for Linux, which is intended for use in tiny-sized and embedded systems, and
anywhere else you'd want to use a compressed file system. With this document, you'll learn how to prepare a SquashFS-ready Linux kernel, create a sqaushed file system and happily use
it.

Home of this HOWTO

The SquashFS HOWTO lives at http://artemio.net/projects/linuxdoc/squashfs. There you will
always find the latest version of the document, and will be able to send your feedback.

===What is SquashFS===

====Introduction====

When creating tiny-sized and embedded Linux systems, every byte of the storage device (floppy, flash disk, etc.) is very important, so compression is used everywhere possible. Also, compressed file systems are frequently needed for archiving purposes. For huge public archives, as well as for personal media archives, this is essential.

SquashFS brings all this to a new level. It is a read-only file system that lets you compress whole file systems or single directories, write them to other devices/partitions or
to ordinary files, and then mount them directly (if a device) or using a loopback device (if it is a file). The modular, compact system design of SquashFS is bliss. For archiving
purposes, SquashFS gives you a lot more flexibility and performance speed than a .tar.gz archive.

SquashFS is distributed as a Linux kernel source patch (which enables SquashFS read support in your kernel), and the mksquashfs tool, which creates squashed file systems (in a
file or on a block device).

The latest SquashFS release tree is 2.x, the former one was 1.x. This document describes both these releases with proper notes given. For example, if some feature or parameter is
different in these release trees, it will be written as follows: new value (2.x) or old value (1.x)

====Overview of SquashFS====

* Data, inodes and directories are compressed
* SquashFS stores full uid/gids (32 bits), and file creation time
* Files up to 2^32 bytes are supported; file systems can be up to 2^32 bytes
* Inode and directory data are highly compacted, and packed on byte boundaries; each compressed inode is on average 8 bytes in length (the exact length varies on file type, i.e. regular file, directory, symbolic link, and block/character device inodes have different sizes)
* SquashFS can use block sizes up to 32 Kb (1.x) and 64Kb (2.x), which achieves greater compression ratios than the normal 4K block size
* SquashFS 2.x inroduced the concept of fragment blocks: an ability to join multiple files smaller than block size into a single block, achieving greater compression ratios
* File duplicates are detected and removed
* Both big and little endian architectures are supported; SquashFS can mount file systems created on different byte-order machines

====Making it clear====

Now let's make sure any further discussions will be clearer for you to understand. The procedure of getting SquashFS working, basically, consists of the following steps:
#Patching and recompiling the target Linux kernel to enable SquashFS support
#Compiling the mksquashfs tool
#Creating a compressed file system with mksquashfs
#Testing: mounting a squashed file system to a temporary location
#Modifying the /etc/fstab or startup scripts of your target Linux system to mount the new squashed file system when needed

===Getting ready for SquashFS===

====Acquiring SquashFS====

The SquashFS home site is located at http://squashfs.sourceforge.net/ - it contains news for the latest release and it's changelog, as well as general information about SquashFS. You can grab the latest version at the SqaushFS project page at SourceForge.

====Preparing a SquashFS-capable kernel====

In order to read SquashFS, you need it supported in your kernel - just as if it was a reiserfs or ext3 file system. You have to make sure there is an appropriate patch for your kernel version - it should be located in linux-2.x.y subdirectory of the SquashFS source tree. Also, remember that in most cases you will need a clean (original) Linux kernel source from kernel.org. If your kernel source is from a distro vendor, it may be already pre-patched with custom vendor patches, and patching with a SquashFS patch will almost surely not work, as SquashFS patches are made against original Linux kernels.

====Patching the kernel source====

With a kernel source and a proper SquashFS patch present, all you have to do is (we'll assume that you have your Linux kernel source in /usr/src/linux and that you have the SquashFS
source in /usr/src/squashfs):

Change to the SquashFS source directory and copy the kernel patch (we'll assume it's named squashfs-patch) to /usr/src/linux.
bash# cd /usr/src/squashfs
bash# cp linux-2.x.y/squashfs-patch /usr/src/linux

Go to the linux kernel source directory /usr/src/linux:
bash# cd /usr/src/linux

Note: please remember that we will not be leaving this directory during all further kernel-related procedures, and all paths will be given relative to /usr/src/linux.

Now patch the source with the SquashFS patch:
bash# patch -p1 < squashfs-patch

====Compiling a 2.6.x kernel====

Cleanup and prepare the kernel source:
bash# make distclean
bash# make mrproper

Configure the kernel using your favourite method (config/menuconfig/xconfig/gconfig):
bash# make menuconfig

#In the "File systems" section, "Miscellaneous file systems" subsection, enable the "Squashed filesystem" option, whether as module or bundled with the kernel. It is only obligatory to compile SquashFS inside the kernel if you plan using squashed initial RAM disks (initrd).
#If you would like to use a squashed initial RAM disk, enable the "Initial RAM disk support" in the "Device drivets" section, "Block devices" subsection. If you want to be able to mount the squashed file system via a loopback device in future, you should enable
#"Loopback device support" in the "Device drivers" section,
#"Block devices" subsection.

Now you may compile the kernel and modules:
bash# make

====Compiling a 2.4.x kernel====

Configure the kernel:
bash# make menuconfig

#In the "File systems" section, enable the "Squashed filesystem" option, whether as module or bundled with the kernel. It is only obligatory to compile SquashFS inside
the kernel if you plan using squashed initial RAM disks (initrd).
#If you would like to use a squashed initial RAM disk, enable the "Initial RAM disk support" in the "Block devices" section.
#If you want to be able to mount the squashed file system via a loopback device in future, you should enable "Loopback device support" in the "Block devices" section.

Now you may compile the kernel and modules:
bash# make dep
bash# make bzImage
bash# make modules

==== Installing and testing the kernel ====

It's time to install your new SquashFS-enabled kernel. The instructions below are for installing and booting the kernel on the host machine. You may want to install and test it on the target system.

We assume that the kernel was compiled for a x86 architecture, and the compressed kernel image is located in the arch/i386/boot/ subdirectory of the kernel tree. Now copy the
kernel to the /boot directory (and name it bzImage-sqsh for convenience, if you like):
bash# cp arch/i386/boot/bzImage /boot/bzImage-sqsh

Don't forget to install the kernel modules if you have any:
bash# make modules_install

Modify your boot loader's configuration file to include your new kernel and install (update) the boot loader. Now you may reboot with your new kernel. When it boots, check that everything went fine:
bash# cat /proc/filesystems

Or, if you built SquashFS support as a kernel module:
bash# insmod squashfs
bash# cat /proc/filesystems

If you see the squashfs line among other file systems, this means you have successfully enabled SquashFS in your kernel.

===Compiling the mksquashfs tool===

Now you need to compile mksquashfs - the tool for creating squashed file systems.
bash# cd /usr/src/squashfs/squashfs-tools

Compile and install mksquashfs:
bash# make
bash# cp mksquashfs /usr/sbin

If everything went fine, typing mksquashfs at the shell prompt should print it's "usage" message.

== The mksquashfs tool, exposed==

=== Using mksquashfs===

mksquashfs is the tool for creating new squashed file systems, and for appending new data to existing squashed file systems. The general command-line format for mksquashfs is:
bash# mksquashfs source1 source2 ... destination [options]

* source1, source2, etc.: files and directories to be added to the resulting filke system, given with relative and/or absolute paths
* destination: a regular file (filesystem image file), or a block device (such as /dev/fd0 or /dev/hda3) where you want to have your squashed file system

Notes for default mksquashfs behavior:
* When the new files are added to the new file system or appended to an existing one, mksquashfs will automatically rename files with duplicate names: if two or more files named text will appear in the same resulting directory, the second file will be renamed to text_1, third one to text_2 and so on.
* Duplicate files will be removed, so there will be only one physical instance (with SquashFS 2.x, you can disable the detection/rtemoval of the duplicates with the -no-duplicates option).
* If destination has a pre-existing SquashFS file system on it, by default, the new source items will be appended to the existing root directory. Examine the options table below to force mksquashfs to overwrite the whole destination and/or change the way new source items are added. Please note that it is not possible to append to a file system created with mksquashfs 1.x using mksquashfs 2.x. You will need to mount the SquashFS-1.x file system and copy the files to some location, and then join them with other needed files to create a SquashFS-2.x file system.
* If a single source file or directory is given, it becomes the root in a newly created file system. If two or more source files and/or directories are given, they will all become sub-items in the root of the new file system.
* The resulting filesystem will be padded to a multiple of 4Kb: this is required for filesystems to be used on block devices. If you are very sure you don't ned this, use the -nopad option to disable this operation.

See the next section for more details about all possible options.

=== Command-line options===

All possible options for mksquashfs are shown in the table below.

{| border="1" cellpadding="5" cellspacing="0" align="center"
|+ Table 1. Command-line options of the mksquashfs tool
!style="background:#80cccc;"|'''Option'''
!style="background:#80cccc;"|'''Description'''
|-
| -always-use-fragments|| divide all files greater than block size into fragments (2.x only, will result in greater compression ratios)
|-
| -b [block size] ||use [block size] filesystem block size (32Kbytes default) - this can be either 512, 1024, 2048, 4096, 8192, 16384 or 32768
|-
| -be or -le ||force a big or little endian file system, respectively
|-
| -check-data|| enable additional file system checks
|-
| -e [file1] ( [file2] ... )|| specify which files and/or directories to omit from the new file system that is to be created
|-
| -ef [file]|| specify a file which contains the list of files/directories to exclude
|-
| -info|| print files, their original size and compression ratio, as they are added to the file system
|-
| -keep-as-directory|| if the source is a single directory, force this directory to be a subdirectory of the root in the created file system
|-
| -noappend|| if the destination file/device already contains a squashed file system, overwrite it, rather than append the new data to an existing file system
|-
| -no-duplicates|| do not detect/remove duplicate file names
|-
| -noD or -noDataCompression|| do not compress the data
|-
| -noF or -noFragmentCompression|| do not compress the fragments (2.x only)
|-
| -no-fragments|| do not generate fragment blocks (2.x only, this will produce almost the same filesystem as 1.x did)
|-
| -noI or -noInodeCompression|| do not compress the inode table
|-
| -nopad|| do not pad the resulting file system to a multiple of 4KBytes
|-
| -root-becomes [name]|| can be used while appending to a pre-existing squashed file system: it will make a new root, and [name] directory will contain all pre-existing files/directories
|-
| -version|| print the version, copyright and license message
|}

In most cases, you should leave all compression/block options by default, as they allow mksquashfs to achieve the best possible compression ratios.

== Creating and using squashed file systems==

===Basic steps===

In order to create a squashed file system out of a single directory (say, /some/dir), and output it to a regular file (thus, producing a file system image), you need to say only
one magic phrase:
bash# mksquashfs /some/dir dir.sqsh

mksquashfs will perform the squashing and print the resulting number of inodes and size of data written, as well as the average compression ratio. Now you have your /some/dir
directory image in the dir.sqsh file. You can now use the mount command to mount it using a loopback device:
bash# mkdir /mnt/dir
bash# mount dir.sqsh /mnt/dir -t squashfs -o loop

To check if you have what's expected:
bash# ls /mnt/dir

If you want to output the file system directly into a device (say, your floppy at /dev/fd0):
bash# mksquashfs /some/dir /dev/fd0

Then just mount the device:
bash# mount /dev/fd0 /mnt/floppy -t squashfs

And check if it's okay:
bash# ls /mnt/floppy

=== Squashing file systems===

Operations described here correspond to most cases where a read-only compressed file system can be used, whether you want it to be on a block device or in a file. This could be
anything from large FTP/HTTP-served archives that don't change often, to having a squashed /usr partition and anything alike with these.

==== Example 1====

Let's suppose you have a /var/arch directory with lots of files and that you want to turn it into a squashed file system and keep it on your root partition as a file (it will be a
file system image that you will mount via a loopback device). The operations needed to perform are as follows.

Squash the directory, then mount it via loopback to test it:
bash# mksquashfs /var/arch /var/arch.sqsh
bash# mkdir /mnt/tmp
bash# mount /var/arch.sqsh /mnt/tmp -t squashfs -o loop
bash# ls /mnt/tmp

If everything is as expected, make this file system mount automatically at boot time by adding this line to your /etc/fstab:
/var/arch.sqsh /var/arch squashfs ro,defaults 0 0

Unmount the file system from the temporary mount point, and mount using it's fstab entry:
bash# umount /mnt/tmp
bash# mount /var/arch

Now just ensure that everything works fine:
bash# ls /var/arch

==== Example 2====

Say you have two hard disk partitions, /dev/hda6 (which is empty) and /dev/hda7 (which is bigger than /dev/hda6, mounted at /var/arch, contains some data and is full). Now, say you
want to squash the /dev/hda7 file system and move it to /dev/hda6, then use /dev/hda7 for some other purposes. We will suppose you have the following line in /etc/fstab (reiserfs is
just an example file system used on /dev/hda7):
/dev/hda7 /var/arch reiserfs defaults 0 0

In the same fashion as with the previous example:
bash# mksquashfs /var/arch /var/arch.sqsh
bash# mkdir /mnt/tmp
bash# mount /var/arch.sqsh /mnt/tmp -t squashfs -o loop
bash# ls /mnt/tmp

If everything went fine, unmount /dev/hda7 and use dd to copy
/var/arch.sqsh to /dev/hda6:
bash# umount /dev/hda7
bash# dd if=/var/arch.sqsh of=/dev/hda7

Now change the line in /etc/fstab for /dev/hda7 to:
/dev/hda6 /var/arch squashfs ro,defaults 0 0

Mount the new file system and check to see if all went fine:
bash# mount /var/arch
bash# ls /var/arch

Don't forget to erase the unneeded file system image:
bash# rm /var/arch.sqsh

=== Creating tiny/embedded systems===

By saying "tiny/embedded", I mean Linux systems that are being built for booting from floppy disks, IDE/USB flash disks, iso9660 CD-ROMs, small-sized hard drives and the like. Whether you want to have your whole root file system on a single media (a single partition, a single floppy), or have a modular system (several floppies or disk partitions), the procedure is almost identical. Creating such Linux systems themselves is out of scope of this HOWTO - there are dedicated HOWTOs and guides for this (like the Bootdisk HOWTO and Linux From Scratch - visit www.tldp.org to retrieve these documents).

==== Squashed file systems on floppy/flash/hard disks ====

In order to use SquashFS for creating Linux systems on small disks, you just have to follow the usual steps for creating a minimal system, performing the following operations at
respective points:
#When developing a kernel for your system, make sure you enable SquashFS support so it can mount squashed file systems
#Use mksquashfs for creating read-only initial ram disks and/or root and/or other file systems
#Don't forget to set file system types to squashfs in /etc/fstab and/or the startup scripts of your system for mounting squashed file systems

Floppy example. Let's say you have your floppy system tree at /home/user/floppylinux and you want to place the root file system on one floppy and /usr on another. What you should do
is:
bash# cd /home/user
bash# mksquashfs floppylinux root.sqsh -e usr
bash# mksquashfs floppylinux/usr usr.sqsh

Note 1: you can see here how we use the -e option to exclude the /usr directory for root file system's image.

Note 2: don't forget to specify squashfs in your root disk's /etc/fstab or startup scripts when mounting the /usr file system.

Insert a root disk in your 3.5" floppy drive (I assume you have a lilo or grub on it, and, thus, a file system exists on this floppy, and the root file system will reside under the
/boot directory of this file system):
bash# mount /mnt/floppy
bash# cp root.sqsh /mnt/floppy/boot

When done, unmount the root floppy, change the floppy to a /usr disk and use dd to transfer the usr file system:
bash# dd if=usr.sqsh of=/dev/fd0

==== Squashed file systems on CD-ROMs====

With SquashFS, you can compress large file systems that will be used in live CDs (just as an example).
#Enable SquashFS in the linux kernel of the target system
#Create a squashed root file system
#Modify the /etc/fstab or startup scripts of the target system to mount the squashd file system when you need it

If you create a root file system out of a running Linux system, use the -e option for mksquashfs to exclude all pseudo-filesystems such as /proc, /sys (on linux kernels after
2.5.x) and /dev (when using DevFS). Also, don't forget to add the file system image itself that is being created with mksquashfs (I think you know the reasons for these
exclusions).

== Acknowledgements==

I would like to express my sincere thanks and immeasurable
respect to:
* Phillip Lougher - for his brilliant work under squashfs, for creating an exculsive patch for linux-2.4.18, for his help with polishing this howto and answers to my mails
* Tabatha Marshall at TLDP for helping me with bringing this HOWTO to the final 1.0 release
* Everybody at The Linux Documentation Project for their great work under all the HOWTOs and guides that helped me a lot with exploring and hacking Linux
* All those at the TLDP mailing lists who helped me with getting started
* Endless thanks and respect to everybody who develops open-source software

== License ==

This document may be used and distributed under the terms and conditions set forth in the Open Content licence. In short, this means that you can freely modify and re-distribute the
HOWTO under the main condition that you keep the author and copyright the article along. The full text of the licence is available at http://www.opencontent.org/opl.shtml

[[Category:File Systems]]
[[Category:HOWTOs]]

File Systems

2008-02-10T21:37:28Z

Glenn: topsort

This page has information about file systems which are of interest to CE Linux Forum members.

== Embedded Filesystems ==
Most embedded devices use [http://en.wikipedia.org/wiki/Flash_memory flash memory] as storage media.
Also, size and bootup time are very important in many consumer electronics products. Therefore,
special file systems are often used with differrent features, such as enhanced compression, or
the ability to execute files directly from flash.

Here are some filesystems designed for and/or commonly used in embedded devices:
=== JFFS2 ===
* [http://sourceware.org/jffs2/ JFFS2] - The Journalling Flash File System, version 2. This is the most commonly used flash filesystem.
** The maximum size of JFFS2 is 128MB.
** http://sourceforge.net/projects/mtd-mods has some patches by Alexey Korolev for improvements to JFFS2
*** See the presentation on Alexey's patches at:

=== CramFS ===
*[http://en.wikipedia.org/wiki/Cramfs CRAMFS] - A compressed read-only file system for Linux. The maximum size of CRAMFS is 256MB.
** "Linear Cramfs" is the name of a special feature to use uncompressed file, in a linear block layout with the Cramfs file system. This is useful for storing files which can be executed in-place. For more information on Linear Cramfs, see [[Application XIP]]

=== SquashFS ===
*[[Squash Fs]] - A (more) compressed read-only file system for Linux. This file system has better compression than JFFS2 or CRAMFS.

=== YAFFS2 ===
*[http://www.aleph1.co.uk/yaffsoverview YAFFS] - Yet Another Flash File System - a file system designed specifically for NAND flash
** Presentation on YAFFS2 by Wookey at ELC Europe 2007: [http://tree.celinuxforum.org/CelfPubWiki/ELCEurope2007Presentations?action=AttachFile&do=get&target=yaffs.pdf yaffs.pdf]
** Presentation from CELF Jamboree 17 comparing YAFFS and JFFS2 on 2.6.10: [http://tree.celinuxforum.org/CelfPubWiki/JapanTechnicalJamboree17?action=AttachFile&do=view&target=celf_flashfs.pdf celf_flash.pdf]

==== YAFFS vs. JFFS2 mount time comparisons for 2.6.10 ====
Here are some core results for mount times. (See the Toshiba Jamboree17 presentation for details.)

* hardware: MIPS, 333 MHZ CPU, with 64 MB NAND Flash.
* kernel: 2.6.10 +EBS patch +YAFFS (20061128 version).
** JFFS2 compression option is disabled.
* Key:
** “Initial”: Time for mounting when the mount is just after launching “flash_eraseall”.
** "1000 files”: Time for mounting after creating 1000 files (one file size is 33554 bytes.)
** “JFFS2+EBS” needs to check EBS, and then it does start to scan the blocks normally. Therefore, “Initial” mount time is a little bit slow.

{|border="1" cellpadding="5" cellspacing="0"
|-bgcolor="#0090ff"
! !! JFFS2 !! JFFS2+EBS !! YAFFS
|-
| Initial || 0.93 sec || 1.12 sec || 0.27 sec
|-
| 1000 files|| 7.34 sec || 1.06 sec || 2.52 sec
|-
|}

=== LogFS ===
*[http://logfs.org/logfs/ logfs] - LogFS is a scalable flash filesystem. It is aimed to replace
JFFS2 for most uses, but focuses more on the large devices.

Matt Mackall writes (in July of 2007):

LogFS is a filesystem designed to support large volumes on FLASH. It
uses a simple copy-on-write update process to ensure consistency (the
"log" in the name is a historical artifact). It's easily the most
modern and scalable open-source FLASH filesystem available for Linux
and it's well on its way to being accepted in the mainline tree.

Scott Preece writes:

The big win for LogFS (in my limited knowledge of it) is that it stores
its tree structure in the media, rather than building it in memory at
mount time. This significantly reduces both startup time and memory
consumption. This becomes more important as the size of the flash device
increases. Read more in LWN (http://lwn.net/Articles/234441) and
linux.com (http://www.linux.com/articles/114295).

Some newer flash memory, like MLC (multi-level cell), are not well supported.

=== AXFS ===
*[[AXFS]] - Advanced XIP File System
** This file system is designed specifically to support Execute-in-place operations

== Special-purpose Filesystems ==
=== ABISS ===
The Active Block I/O Scheduling System is a file system designed to be able to provide real-time
features for file system I/O activities.

See [http://abiss.sourceforge.net/ ABISS]

=== UnionFS ===
Sometimes it is handy to be able to overlay file systems on top of each other.
For example, it can be useful in embedded products to use a compressed read-only
file system, mounted "underneath" a read/write file system. This give the
appearance of a full read-write file system, while still retaining the
space savings of the compressed file system, for those files that won't
change during the life of the product.

UnionFS is a proj3ect to provide such a system (providing a "union" of multiple
file systems).

See http://www.filesystems.org/project-unionfs.html

See also union mounts, which are described at http://lkml.org/lkml/2007/6/20/18
(and also in Documentation/union-mounts.txt in the kernel source tree (or will be, when this feature
is merged.)

== Other projects ==
=== Multi-media file systems ===
* XPRESS file system - [See OLS 2006 proceedings, presentation by Joo-Young Hwang]
** I found out at ELC 2007 that this FS project was recently suspended internally at Samsung

=== flash logical volume management ===
* UBI http://www.linux-mtd.infradead.org/faq/ubi.html manages multiple logical volumes on a single flash device, specifically supporting NAND flash devices. UBI provides a flexible partitioning concept which still allows for wear-levelling across the whole flash device.

[[Category:File Systems| ]]

Category:File Systems

2008-02-10T21:36:57Z

Glenn: new

[[Category:Linux]]

Multimedia

2008-02-10T21:36:26Z

Glenn: topsort

Here are some miscellaneous resources related to audio, video and graphics systems under Linux:

Also see the section on [[User Interfaces]].

== CELF 2.0 Specification for AVG ==
(more like a set of recommendations rather than a specification)
*{{pdf|CelfAudioVideoGraphicsSpec2 accepted 20060606.pdf|AVG Spec V2}}

== Audio Video Working Group ==

Please see the CELF wiki for more information:
[http://tree.celinuxforum.org/pubwiki/moin.cgi/AudioVideoGraphicsWorkingGroup Audio Video Graphics Working Group]

== DirectFB study ==
=== What is DirectFB, How Does DirectFB Work ===
[[DirectFB]]

=== Sample Implementation of DirectFB on an embedded Linux platform ===
[[Porting DirectFB]]

=== Some DirectFB benchmark on embedded Linux platform ===
[[Benchmark DirectFB]]

== Related Projects ==
=== Graphics/Video out ===
====Framebuffer====
*http://www.kernel.org/ (1) KD26/fb
*http://linuxconsole.sourceforge.net/fbdev/HOWTO/
*http://www.tldp.org/HOWTO/Framebuffer-HOWTO.html
====DirectFB====
*http://www.directfb.org/
*http://www.directfb.org/documentation/DirectFB_overview_V0.2.pdf
====NanoX====
*http://www.microwindows.org/
====SDL====
*http://www.libsdl.org/
====Gstreamer====
*http://www.gstreamer.net/
====OpenGL (OpenML)====
*http://www.opengl.org/
*http://www.khronos.org/opengles/

=== Video in ===
====V4L[2]====
*http://www.kernel.org/ (1) KD26/video4linux
*http://bytesex.org/v4l/
====OpenML====
*http://www.khronos.org/openml/
====LinuxTV (DVB API)====
*http://www.linuxtv.org

=== Audio in/out ===
====OSS====
*http://www.kernel.org/ (1) KD26/sound/oss
*http://www.4front-tech.com/oss.html
====ALSA====
*http://www.kernel.org/ (1) KD26/sound/alsa
*http://www.alsa-project.org
====OpenAL====
*http://www.openal.org/

=== Users of AVG ===
====Video Lan====
*http://www.videolan.org
====Freevo====
*http://freevo.sourceforge.net
====LinuxTV====
*http://www.linuxtv.org/
====MythTV====
*http://www.mythtv.org/
====DVR ====
*http://dvr.sourceforge.net/html/main.html
====OpenPVR====
*http://www.funktronics.ca/openpvr/
*http://sourceforge.net/projects/openpvr/

=== Other ===
====TV Linux Alliance====
*http://www.tvlinuxalliance.com/
====TV Anytime====
*http://www.tv-anytime.org/
====Digital Home Working Group====
*http://www.dhwg.org/
====Boot Splash====
*[http://www.BootSplash.org/ www.bootsplash.org]

====Free Type====
*http://freetype.sourceforge.net/freetype2/
====ARIB architecture====
*http://www.arib.or.jp/english/html/overview/ov/std_b24.html

'''Note (1)''' - KD26 refers to the [http://www.kernel.org/pub/linux/kernel/v2.6/ Linux 2.6.X kernel] tree, which has a "Documentation" sub-directory.

[[Category:Multimedia| ]]

Memory Management

2008-02-10T21:35:57Z

Glenn: catchg

This page has information about various memory management projects and activities which are of interest to embedded Linux developers.

== Areas of Interest ==

Most of these areas have wider reaching implications, but are of relatively simpler use in the embedded case, largely thanks to not having to contend with swap and things of that nature. This as well as vendors not afraid of deviation from mainline for product programs makes for an excellent playground for experimenting with new things in the memory management and virtual memory space.

=== Huge/large/superpages ===

*This applies to both transparent large page usage as well as the more static usage models, primarily relating to work outside of the hugetlb interface/libhugetlbfs.
*Embedded systems suffer from very small TLBs generally using PAGE_SIZE'd pages (4kB) for coverage. In most cases this places the system under very heavy pressure for any kind of userspace work, and very visibly degrading performance, with most applications taking anywhere from 5-40% of their time on the CPU servicing page faults.
*Preliminary discussion on this subject as well as links to additional information is happening through the wiki here: [http://linux-mm.org/ Huge Pages]
=== Page cache compression ===

*This relates to using various compression algorithms for performing run-time compression and decompression of page cache pages, specifically aimed at both reducing memory pressure as well as helping performance in certain workloads.
*More information can be found on the wiki here [http://linux-mm.org/CompressedCaching CompressedCaching] as well as at the [http://linuxcompressed.sourceforge.net SF Compressed Caching] home page.

=== Reserving (and accessing) the top of memory on startup ===
A quote from Todd's email on how to use the reserved physical memory in "mem=".

----

Given that you have a fixed address for your memory, and is already
reserved, the easier way to use it is by calling mmap() over the /dev/
mem device, use 0 as the start address, and the physical address of
the reserved memory as the offset. The flags could be MAP_WRITE|
MAP_READ. That will return you a pointer on user space for your
memory mapped by the kernel. For example

If your SDRAM base address is 0x80000000 and your memory is of 64MB,
but you use the cmdline mem=60M to reserve 4MB at the end. Then your
reserved memory will be at 0x83c00000, so all you need to do is

<pre>
int fd;
char *reserved_memory;

fd = open("/dev/mem",O_RDWR);
reserved_memory = (char *) mmap(0,4*1024*1024,PROT_READ| PROT_WRITE,MAP_SHARED,fd,0x83c00000);
</pre>
----

== Additional Resources/Mailing Lists ==
*[http://linux-mm.org LinuxMM] - links to various sub-projects, and acts as a centralized point for discussion relating to memory management topics ([mailto:majordomo@kvack.org linux-mm] mailing list and [http://marc.theaimsgroup.com/?l=linux-mm archives]).

[[Category:Linux]]

WiFi

2008-02-10T21:23:21Z

Glenn: +cat

[[WiFi]] is an acronym for IEEE 802.11 wireless networking. New laptops have often this capability built-in. Coffee shops and cafes offer this so you can connect to the Internet.

The [[ZipIt]] uses built-in [[WiFi]] to access IM (Instant Messaging) servers via the Internet.

[[Category:Hardware]]

Wanted

2008-02-10T21:22:40Z

Glenn: +Category:Community

This article lists pages that are requested to be added to this wiki.
The page specifically lists pages that have been requested by users.
(Another way to do this is to create a stub page for the desired page, and add the tag
for Category:NeedsEditing.) However, this page lists pages that people would like
to see, but have insufficient knowledge or expertise to create.

An automatically generated list of pages that are referenced, but do not exist, is at
[[Special:Wantedpages]]

== User-requested pages ==
Put your request for a page or article you would like to see on this wiki in the list below:

; Realtime testing best practices : I'd like to see a document collecting existing information about realtime testing, with links to test programs and information about already-performed tests (with pros and cons for each approach). [[User:TimBird|TimBird]]

[[Category:Community]]

Main Page

2008-02-10T21:20:56Z

Glenn: +cats

{|
| nowrap style="vertical-align: top; font: bold xx-large sans-serif;" |
Embedded Linux Wiki
|}

The purpose of this wiki is to preserve and present information about the use of Linux in embedded systems.
The primary target audience is developers, engineers, managers, hobbyists and others who are involved
with putting Linux into embedded systems.

To use this wiki, click on one of the portal links below. The top box of (small) icons are for
general information about this site, and related resources. The other portal links will take you to lists
of resources, or collections of information, you can use to tackle problems in the particular area referred to. For example, if you have a problem with boot up time of your embedded Linux system, click on "Boot Time".
You can also see a list of [[Special:Allpages|all the pages on this site]].

We hope this information is helpful in your development tasks.

If you see something wrong, please change it. If you know something more about an issue, please
add it.

<div style="margin-left: 50px;">
{|style="width: 100%; margin:0; padding:0; border-collapse: collapse;"
|style="width: 25%;"|
[[image:mp_info.png|16px]] [[Help:About|About]]<br/>
[[image:mp_help.png|16px]] [[Help:Contents|Site Policy]]
|style="width: 25%;"|
[[image:mp_man.png|16px]] [[Help:Editing|Editing Help]]<br/>
[[image:mp_wanted.png|16px]] [[Wanted|Wanted Pages]]
|style="width: 25%;"|
[[image:mp_mail.png|16px]] [[eLinuxWiki:Mailing List|Mailing List]]<br/>
[[image:mp_irc.png|16px]] [[eLinuxWiki:Irc|IRC]]
|style="width: 25%;"|
[[image:mp_admin.png|16px]] [[eLinuxWiki:Glossary|Glossary]]
|}
</div>

{| width="100%"
|-
|style="vertical-align:top" |

<div style="margin: 0; margin-top:10px; margin-right:10px; border: 1px solid #dfdfdf; padding: 0 1em 1em 1em; align:right;">
'''Development Portals'''
<div style="margin-left: 25px;">
{|style="width: 100%; margin:0; padding:0; border-collapse: collapse;"
|style="width: 25%;"|
[[image:Welly1.jpg]] [[Boot Time]]<br /><br />
[[image:Multimedia.png]] [[Multimedia]]<br /><br />
[[image:Filecab.jpg]] [[File Systems]]
|style="width: 25%;"|
[[image:cfcard.jpg]] [[Memory Management]]<br /><br />
[[image:Power.png]] [[Power Management]]<br /><br />
[[image:clockface.jpg]] [[Real Time]]
|style="width: 25%;"|
[[image:Padlock2.jpg]] [[Security]]<br /><br />
[[image:Blimp.jpg]] [[System Size]]<br /><br />
[[image:Board2.jpg]] [[Resource Management]]
|style="width: 25%;"|
[[image:Event.jpg]] [[Events]]<br /><br />
[[image:Skull.jpg]] [[Hardware Hacking]]<br /><br />
[[image:Toolbox.jpg]] [[Toolbox]]
|}
</div>
</div>

Please [[volunteer editor tasks|help to extend]] this wiki. Thank you!

<div style="margin:0; margin-top:10px; margin-right:10px; border:1px solid #dfdfdf; padding:0 1em 1em 1em; background-color:#ccffff; align:right; ">
'''Monthly Feature'''

In this space each month will be featured a topic relevant to Embedded Linux Development.
</div>

<div style="margin: 0; margin-top:10px; margin-right:10px; border: 1px solid #dfdfdf; padding: 0 1em 1em 1em; align:right;">
'''Embedded Linux Information'''
<div style="margin-left: 25px;">
{|style="width: 100%; margin:0; padding:0; border-collapse: collapse;"
|style="width: 20%;"|[[image:Productsico.jpg]] [[Products]]
|style="width: 20%;"|[[image:Companysico.jpg]] [[Companies]]
|style="width: 20%;"|[[image:Vendorsico.jpg]] [[Vendors]]
|style="width: 20%;"|[[image:Processorsico.png]] [[Processors]]
|-
|[[image:communityico.jpg]] [[Community]]
|[[image:communityico.jpg]] [[Experts]]
|[[image:at-work.jpg]] [[Jobs]]
|[[image:Companysico.jpg]] [[Board and Chip Vendors]]
|}
</div>
</div>

To experiment with this wiki try [[Sandbox]]. See the [http://meta.wikipedia.org/wiki/MediaWiki_User's_Guide User's Guide] for usage and configuration help.

[[Category:Community| ]]
[[Category:Categories| ]]

Vendors

2008-02-10T21:20:09Z

Glenn: +cat

== Overview ==

This page lists companies who offer services around embedded Linux, such as training, support, etc. We also list companies who provide Linux distributions specifically targeting embedded uses. If you are looking for companies who build and sell devices with Linux as their operating system, please see the [[Companies]] page.

* [http://www.katdc.com KAT Digital Corp.]
* [http://www.sidebranch.com Sidebranch], The Netherlands.
* [http://www.mvista.com/ MontaVista]
* [http://www.timesys.com/ TimeSys]
* [http://www.windriver.com/ Wind River]
* [http://www.intellimetrix.us/ Intellimetrix]
* [http://www.linutronix.de/ Linutronix], Germany
* [http://www.lineo.co.jp/eng/index.html Lineo Solutions], Japan
* [http://www.denx.de/ DENX Software Engineering]
* [http://www.drivingdevices.com/ Driving Devices Limited], United Kingdom
* [http://www.wmltd.co.uk/ William Matthew Limited], United Kingdom
* [http://www.esfnet.co.uk/ Embedded Software Foundry Limited], Sheffield, UK
* [http://www.koansoftware.com/ KOAN sas], Bergamo, Italy

[[Category:Companies]]

Tux Screen

2008-02-10T21:18:00Z

Glenn: +Category:Hardware

[[Image:tuxscreen.jpg]]

the Shanon aka TuxScreen was built as part of a R&D joint venture between Lucent and Philips. 4000 built - 2000 were used for R&D the project was cancelled due to dissolving of the lucent/ philips joint venture. the surplus units where then purchase by [[Tim Riker]] who then resold the units as an inexpensive embedded linux development platform.

http://www.tuxscreen.net

[[Category:Hardware]]

Category:Candidates for speedy deletion

2008-02-10T21:17:15Z

Glenn: new

[[Category:Community]]

Category:Templates

2008-02-10T21:16:56Z

Glenn: new

[[Category:Community]]

Template:Delete

2008-02-10T21:16:34Z

Glenn: new

<includeonly>{{{category|[[Category:Candidates for speedy deletion]]}}}</includeonly>
<noinclude>
Template for marking pages to be deleted.

[[Category:Templates]]

</noinclude>

TomoyoLinux

2008-02-10T21:13:33Z

Glenn: +cat

http://tomoyo.sourceforge.jp/tomoyo.png

== Overview ==

TOMOYO Linux is an implementation of Mandatory Access Control for Linux.

*[http://tomoyo.sourceforge.jp/wiki-e/?WhatIs What is TOMOYO Linux?]
**[http://tomoyo.sourceforge.jp/wiki-e/?WhatIs#comparison Comparison with SELinux, Smack and AppArmor]
**[http://tomoyo.sourceforge.jp/wiki-e/?WhatIs#benchmark Benchmark]
*[http://tomoyo.sourceforge.jp/wiki-e/ TOMOYO Linux Wordwide Wiki Page.]
*[http://spreadsheets.google.com/pub?key=p5SYVEC0jPMPuctiaabyZAw&output=html Distribution Chart]

*[http://sourceforge.jp/projects/tomoyo TOMOYO Linux @SourceForge.jp]

== LiveCD (ISO image) ==

TOMOYO Linux enabled Ubuntu 7.10 LiveCD is available. It's 100% safe and fun (and free, of course) to try. Please take your time to explore the amazing world of policy auto learning.

[http://tomoyo.sourceforge.jp/wiki-e/?TomoyoLive TOMOYO Linux Live!]

== Quick and Easy Install Guide ==

*[http://tomoyo.sourceforge.jp/en/1.5.x/1st-step/centos5/ CentOS 5]
*[http://tomoyo.sourceforge.jp/en/1.5.x/1st-step/fc6/ Fedora Core 6]
*[http://tomoyo.sourceforge.jp/en/1.5.x/1st-step/etch/ Debian Etch]
*[http://tomoyo.sourceforge.jp/en/1.5.x/1st-step/ubuntu6.10/ Ubuntu 6.10]

Other distributions

*http://sourceforge.jp/projects/tomoyo/files/

== Install ==

=== LSM version (version 2.1.1) ===

*http://tomoyo.sourceforge.jp/en/2.1.x/

=== Original hook version (version 1.5.2) ===

*http://tomoyo.sourceforge.jp/en/1.5.x/install.html

=== TOMOYO Linux on [http://www.linuxfromscratch.org/lfs/ LFS] ===

For those "tough guys", TOMOYO Linux just runs fine on [http://www.linuxfromscratch.org/lfs/ LFS]. Find yourself and make your own version.

*[http://cblfs.cross-lfs.org/index.php/TOMOYO TOMOYO CBLFS]
*[http://tomoyo.sourceforge.jp/wiki-e/?TomoyoOnLFS TOMOYO Linux on "Linux From Scratch"]

== Browse the code ==

*[http://tomoyo.sourceforge.jp/cgi-bin/lxr/source/ TOMOYO Linux LXR]

== Presentations ==

=== 2008-2-24 [http://www.fosdem.org/2008/ FOSDEM2008] ([http://www.fosdem.org/2008/schedule/devrooms/embedded Embedded Developer Room]) ===

*[http://www.fosdem.org/2008/schedule/events/embedded_tomoyo_secure TOMOYO Linux for Secure Embedded]

=== 2007-11-29 [http://www.pacsec.jp/speakers.html PacSec 2007] ===

*[http://www.thinkit.co.jp/free/article/0712/9/1/ PacSec2007 Report]
*[http://sourceforge.jp/projects/tomoyo/document/PacSec2007-en-no-demo.pdf TOMOYO Linux: A Practical Method to Understand and Protect Your Own Linux Box]
*[http://sourceforge.jp/projects/tomoyo/document/PacSec2007-en-demo.pdf TOMOYO Linux: A Practical Method to Understand and Protect Your Own Linux Box (with demo)]
*[http://sourceforge.jp/projects/tomoyo/document/PacSec2007-handout.pdf Handouts (bilingual)]

=== 2007-06-29 [http://www.linuxsymposium.org/2007/index_2007.php Ottawa Linux Symposium 2007] ===

*[http://sourceforge.jp/projects/tomoyo/document/ols2007-tomoyo-20070629.pdf/en/5/ols2007-tomoyo-20070629.pdf TOMOYO Linux BoF]

=== 2007-04-18 CELF Worldwide [http://www.celinux.org/elc2007/index.html Embedded Linux Conference 2007] ===

*[http://sourceforge.jp/projects/tomoyo/document/elc2007-presentation-20070418.pdf/en/5/elc2007-presentation-20070418.pdf "TOMOYO Linux - A Lightweight and Manageable Security System for PC and Embedded Linux"]
*[http://sourceforge.jp/projects/tomoyo/document/elc2007-tutorial-20070418.pdf/en/3/elc2007-tutorial-20070418.pdf TOMOYO Linux Tutorial]

== For the memory of OLS2007 ==

*[http://tomoyo.sourceforge.jp/wiki-e/?OLS2007-BOF Memorial of OLS2007 BOF]
*[http://picasaweb.google.co.jp/haradats/OLS2007 OLS2007 Photos]
*[http://www.thinkit.co.jp/free/article/0709/8/1/ 「熱い言葉に背中を押されて」] (in Japanese)
*[http://www.thinkit.co.jp/free/article/0709/8/1/ 「海外での講演、そして新たなチャレンジへ」] (in Japanese)

== Articles ==

[http://kerneltrap.org/Linux/TOMOYO_Linux TOMOYO Linux | KernelTrap]

== Readings ==

*[http://www.thinkit.co.jp/free/article/0706/21/1/ 「初体験 TOMOYO Linux」] (in Japanese)
*[http://tomoyo.sourceforge.jp/wiki/?WorldOfTomoyoLinux The World of TOMOYO Linux] (in Japanese)

== Mainline ==

=== Activities ===

*[http://lwn.net/Articles/238049/ 1st posting (13 Jun, 2007)]
*[http://lwn.net/Articles/246930/ 2nd posting (24 Aug, 2007)]
*[http://lwn.net/Articles/252652/ 3rd posting (2 Oct, 2007)]
*[http://lwn.net/Articles/254503/ 4th posting (11 Oct, 2007)]
*[http://lwn.net/Articles/258905/ 5th posting (17 Nov, 2007)]
*[http://lwn.net/Articles/263179/ TOMOYO Linux Security Goal (27 Dec, 2007)] ([http://thread.gmane.org/gmane.linux.kernel.lsm/5022 thread])
*[http://lwn.net/Articles/264187/ 6th posting (8 Jan, 2008)]

[http://tomoyo.sourceforge.jp/wiki-e/?WhatIs#mainlining More detail]

=== Forecast ===

*[http://www.linux-foundation.org/en/Linux_Weather_Forecast/security Linux Weather Forecast/security - The Linux Foundation]

== Contact ==

Toshiharu Harada (Project Manager), Senior Technical Manager, NTT DATA CORPORATION

haradats_at_nttdata.co.jp

[[Category:Linux]]

Linux Tiny-FAQ

2008-02-10T21:12:56Z

Glenn: +cat

== FAQ: The Linux-Tiny Tree ==

This FAQ was originally written as a contribution to the Linux-Tiny effort which aims to make it possible to create a lean-and-mean Linux kernel. This document is a copy of the original [http://itmaze.com.au/articles/linux-tiny/ linux-tiny FAQ]. It is intended to evolve into a more generic Embedded Linux FAQ over time.

A special note should be made here about Matt Mackall <mpm at selenic.com> who originally put linux-tiny together and made it into something that others could download.

== Credits ==

Contributors:

* Mahendra <Mahendra_M at infosys.com>
* Matt Mackall <mpm at selenic.com>
* Rob Landley <rob at landley.net>
* Tim Bird <tim.bird at am.sony.com>

Author: Onno Benschop <onno at itmaze.com.au>

== What is this all about? ==

This project is a collection of patches to the Linux kernel to remove different aspects of the kernel in order to save space. For example, removing all the error messages from the kernel removes about 300k from it.

A kernel uses space in all manner of different ways, the space it takes up on disk, the space it takes to load the code into memory and the space it takes to store information while it is running. Linux-Tiny is a set of patches to the mainline kernel (at kernel.org) that addresses the way the kernel uses space. The patches you'll find here are intended to be included into the main kernel-tree and some have already been submitted for such use.

This is useful in places where space is limited, or in the case of error messages, there is no-one to see them - like in embedded systems. Linux-Tiny is also for users of small or legacy machines such as 386's and handhelds.

A description on the how and why is available here:

http://www.selenic.com/tiny-about/

and here is a really detailed description:

http://www.selenic.com/tiny-about/Reprint-Mackall-OLS2004.pdf

== How do I use it? ==

This project will not supply you with an actual kernel, it gives you a set of patches that you need to apply to a kernel source tree, after which you can configure and compile your new trimmed kernel.

The Linux-Tiny patches default to the standard kernel configuration, so you'll need to turn on or off those settings you want to change.

The process to apply Linux-Tiny is pretty straightforward if you've patched a kernel before, but if you haven't there are a few steps you'll need to take: (and you should read the question "Will it work on any kernel?")

1. Download kernel sources
2. Download the appropriate patch set for that kernel
3. cd /path/to/my/linux/sources/
4. patch -p1 </path/to/patch/Linux-Tiny.patch
5. Check for reject files and fix where appropriate
6. Configure the kernel
7. Compile the kernel

== What Difference does it make? ==

Well, that depends entirely on what you're trying to achieve. Different patches and different ideas in size reduction are suited to different purposes. The various size reductions have varying impacts on different aspects of the kernel. Efforts are being made to document and compare sizes.

=== Config Size Test ===

Tim Bird has written an program to automatically test the kernel size under different configurations. As an example, this table was produced for a test run conducted on Oct 1st, 2007, on an i386 target (called 'nut'): http://testlab.celinuxforum.org/otlwiki/ConfigSizeTestResultsNutOct1

One of the main purposes of this program is to find problems with the Linux-tiny patches. Also, this data could be used to help identify the options having the largest effect on the kernel size.

=== Bloat Watch ===

Matt Maccall wrote bloatwatch at CELF 2006, and the data that was gathered then is still available, at: http://testlab.celinuxforum.org/bloatwatch/index.cgi

It provides a great amount of detail, and the ability to compare versions over time. As such, it really was intended to be a kernel size regression analysis tool.

== What is the difference between the broken-out and the patch files? ==

The broken out patches contain all the patches as individual patches, so you can choose to apply one or all of them, where the patch file contains all the patches as one big patch, applied mostly in the intended merge order from simple to complex.

For example, one set of patches turns off all error messages. Another set of patches changes the way memory is allocated. The broken out set contains the two different patches as separate entities, the complete patch contains both of the changes together.

The reason it is like that, is because not everyone will want to apply all patches, or will be able to apply all patches - see below.

== Will it work on any kernel? ==

Yes.

Seriously, the answer is in typical computing fashion: "That depends"

If you're new to patching, maybe a brief explanation is needed. Pardon me if this is too remedial, but I wanted to write low level since I don't know your experience. Sorry in advance for the length.

Patches are created by diff'ing two source bases against each other. A patch is like a set of instructions for converting one source base into the other. Inside the patch are both the changes themselves, and context information which helps patch to identify the correct place to make each change.

If you try to apply a patch to a different version of software than it was originally created from, you may encounter problems. Patch may not find the correct place to make a change. When this happens, it leaves a reject file, showing the change that it could not make.

The Linux-Tiny patches were created based on a kernel.org version of Linux. Almost all kernels shipped by the major Linux distribution vendors are significantly different from the kernel.org version of Linux. Distribution vendors add many (often hundreds) of patches to their kernels, in order to provide extra features to their products. This includes drivers, protocols, file systems, and other enhancements.

Since Linux-Tiny is a fairly big patch-set, it is very likely that there will be overlap between some of the changes it makes and some of the changes made by your Linux distribution.

If you get rejects while patching, it means that not all of the patch could apply. It is possible that the rejects don't matter, but you can't know that without looking at them. Basically, any time you get rejects you need to examine them and either 1) fix them or 2) decide they can be ignored, before proceeding.

Many times, rejects can be fixed pretty easily. A common cause of rejects is multiple additions to the beginning or end of something. Often, in these cases, the changes themselves don't really interfere with each other. But the change in text from one patch causes the patch program to be unable to match the context for a change from another patch.

For example, take the following text and patches. Starting with a simple text file describing a fish, there are two patches, one of which adds stuff about a dog and one adds stuff about a cat. Semantically, these patches don't interfere with each other, and there is no harm in applying both changes to the file. However, patch has problems with them.

file 'A':

I have a fish, named Charlie.
He swims in a fishbowl.
He doesn't eat much.

file '1.patch':

--- A 2004-10-21 10:49:53.547578239 -0700
+++ B 2004-10-21 10:50:40.395525796 -0700
@@ -1,3 +1,5 @@
I have a fish, named Charlie.
He swims in a fishbowl.
He doesn't eat much.
+My dog is named Spot.
+Spot is friendly and wags his tail.

file '2.patch':

--- A 2004-10-21 10:49:53.547578239 -0700
+++ C 2004-10-21 10:51:05.435687319 -0700
@@ -1,3 +1,5 @@
I have a fish, named Charlie.
He swims in a fishbowl.
He doesn't eat much.
+My cat is named cleo.
+Cleo scratches the couch.

Each of these patches could be applied successfully to file A, individually. However, if you try to apply these patches in sequence, like so:

cp A D
patch D <1.patch
patch D <2.patch

You'll get a reject on the second patch. The place where the description of the cat is supposed to be is now different, and patch gives up.

To fix this, you need to add the rejected changes manually, taking into account the differences caused by the other changes. (Think of 1.patch as your Linux distribution changes and 2.patch as the Linux-Tiny patch set.)

Doing this is usually pretty easy. You look at the rejected hunks, and compare the lines they intended to patch from the original file with the lines in your source base. Compare a kernel.org version of Linux, with your distribution version of Linux, for the file and lines mentioned in the reject file. In the example above, I would have to decide if I wanted the description of the dog to come before or after the description of the cat. I would make the change manually. (If this were source, I would then recompile and test extensively... :-)

For extra credit, once you have made your changes, you can create a new Linux-Tiny patch set by diff'ing your distribution-tiny with your original distribution (which you should have saved off earlier). Your distribution users may appreciate having such a "personal-tiny" patch set to work with.

== How do I get access to this information? ==

Access has been created to a repository of the Linux-tiny patches using the Mercurial SCM:

* http://selenic.com/repo/tiny

This has all of the old -tiny releases nicely tagged and new changes will show up here first.

Assuming you have Mercurial and quilt installed, grabbing the latest -tiny work is as simple as:

cd linux
hg clone http://selenic.com/repo/tiny patches
quilt push -a

This will hopefully also make it easier for people to keep in sync with Matt Mackall's changes and send him fixes, and make it easier for him to track changes in the upstream kernel.

More information about Mercurial here:

* http://selenic.com/mercurial/wiki/

== Is there more that I can read? ==

Tim Bird has done some work with Linux-Tiny on an ARM OSK board. He has written some raw notes on the subject.

Thomas Lundquist has done some work using lzma rather than bzip to do further compression and a diff for a 2.4 Kernel is available.

Ming-Ching Tiew has completed some 2.4 and 2.6 patches for lzma. Both vmlinuz and init are available. Patches for busybox are included as well as a utility called lzmacat. (Note, this requires the LZMA SDK.)

Some work is being done with LinuxBIOS and Linux-Tiny.

[[Category:Linux]]

Timing APISpecification

2008-02-10T21:12:25Z

Glenn: +Category:Community

<pre>#noprint
VERSION 0.2 - for other versions, click the "info" button.

'''Table Of Contents:'''
[[TableOfContents]]
#noprint</pre>

== Introduction ==

This specification is for a timing API for the Linux kernel which is available early during the bootup process.

== Rationale ==
The purpose of this specification is to provide a platform-neutral API for timing
instrumentation of the Linux kernel. Currently, there are a few different timing
[[APIs]] in the kernel used for such instrumentation. For example, on the i386 architecture
the Time Stamp Counter (TSC) register of the processor is used for getting fast timestamps
for low overhead and high resolution time measurements. (This has historically been
made available with the get_cycles() function call.) Many timing measurement systems
for the Linux kernel have been based on either get_cycles() or directly on a call
to read the TSC register, such as rdtscll().

There are a few problems with this setup. First, these functions are not supported
on all architectures and platforms of interest. Thus, measurement systems that
utilize them are not portable to these other platforms. Also, other functions
which ARE available on all platforms are either high overhead, or are not
usable from the very beginning of kernel bootup, or both.

This specification, then, is intended to address the need for a common API
available on all embedded platforms, for a low-overhead, high resolution clock
source accessible very early (even before setup_arch()) in the boot sequence
of the kernel.

== Specifications ==
# The Linux kernel SHALL support a configuration to provide the following described timing API.
1.1 The configuration option to enable this feature MUST be called CONFIG_FAST_TIMESTAMPS
2. When CONFIG_FAST_TIMESTAMPS is enabled, the kernel SHALL provide the following 2 routines:
2.1 void store_timestamp(timestamp_t *t)

2.2 void timestamp_to_timeval(timestamp_t *t, struct timeval *tv)
3. The store_timestamp() API MUST be available and useful no later than upon the completion of
the setup_arch() routine in the kernel.
3.1 The store_timestamp() API SHOULD be available and useful as soon as possible after start_kernel()
4. The size and resolution of timestamp_t *t SHOULD be such that timestamps can record at least 30
seconds of timing information.
4.1 The size of timestamp_t SHOULD be no greater than 64 bits.
5. store_timestamp() MUST produce values for timestamp that are monotonically increasing with each
call ''on the same processor'' (with the exception of overruns or explicit clock value changes).
6. The resolution of timestamp, after conversion to timeval, MUST NOT be less than 1 jiffy.
6.1 The resolution of timestamp, after conversion to timeval, SHOULD be at least 100 usec.

6.2 A resolution of at least 1 usec for values of timestamp is preferred.

== Notes (informational and non-normative) ==
- The clock needs to be accessible at least during the boot up period of the kernel.
- A free-running clock could be set up by firmware (before kernel start) and polled afterwards
by the kernel in order to measure total system bootup time. Note that this implies that the
timestamp value does not need to start at 0 with the beginning of execution of the kernel
or the setup of the timer device.
- It is preferred that the timestamp value returned should not fluctuate with changes in
CPU frequency, but this is not mandatory.
- Alternative, it should be possible to avoid changes in the CPU frequency during the timing period.
- The values returned need not be linearly related. That is, it is acceptable for the values
to be non-linear, as long as the conversion to timeval (sec, nsec) is correct.
Thus, as one example of value management, it is possible to store the hardware clock value
in the low 32 bits, and the number of rollovers in the high 32 bits. This works even if the
clock source itself is less than 32 bits wide (eg 12 bits, or 16 bits).
- It is not necessary for there to be an accurate correlation of timevals produced by
timestamp_to_timeval() to wall clock times. That is, there is no requirement to implement
a strict gettimeofday call.
- the purpose of separating the "store" function from the "to_timeval" function, is to make
the overhead of actually acquiring the timestamp as little as possible. Some callers may
immediately convert the timestamp to a timeval, but other time-sensitive code may defer the
conversion to timeval units until some later time (or indefinitely).
- The size of timestamp_t should be as small as possible. It is expected that timestamp values will be
stored in trace logs which may accumulate data points quickly. It is expected that on many
embedded architectures, timestamp_t will be defined as an unsigned long (32 bits), and that
it will NOT be defined to be any greater in size than an unsigned long long (64 bits).
- Embedded processors have CPU clock frequencies ranging from a few MHZ to a few GHZ.
A 1 GHZ clock will overrun 32 bits in about 4 seconds. This means that for
fast-running hardware timestamp_t should probably be an unsigned long long.

== References ==
There is general information about the requirements, other timing mechanisms, and proposals at [http://tree.celinuxforum.org/pubwiki/moin.cgi/InstrumentationAPI InstrumentationAPI]

== Remaining Issues ==
- /\ ''It would be good to try this out somewhere''

[[Category:Community]]

Timer Hash Array Project

2008-02-10T21:12:04Z

Glenn: +Category:Community

== timer-hash-array patch ==

The patch: [http://tree.celinuxforum.org/patches/timer-hash-array-04.07.16/timer-hash-array-04.07.16.patch timer-hash-array.patch]

The test data: [http://tree.celinuxforum.org/patches/timer-hash-array-04.07.16/timer-hash-tests-04.07.16.txt timer-hash-tests.txt]

The kernel instrumentation: [http://tree.celinuxforum.org/patches/timer-hash-array-04.07.16/timer-hash-array-test-04.07.16.diff timer-hash-array-test.diff]

The user space test program: [http://tree.celinuxforum.org/patches/timer-utils-04.07.02.tar.gz timer-utils.tar.gz]

Handy patch with instrumentation: [http://tree.celinuxforum.org/patches/timer-hash-array-04.07.16/timer-hash-array-with-test-04.07.16.patch timer-hash-array-with-test.patch]

The original with instrumentation: [http://tree.celinuxforum.org/patches/timer-hash-array-04.07.16/linux-2.6.7.orig-with-test-04.07.16.patch linux-2.6.7.orig-with-test.patch]

Everything in one: [http://tree.celinuxforum.org/patches/timer-hash-array-04.07.16.tar.bz2 timer-hash-array.tar.bz2]

== Other Links ==

CELF's HRT project page: [[Hrt Project]]

CELF's patch archive page: [[Patch Archive]]

[[Category:Community]]

Test Systems

2008-02-10T21:10:42Z

Glenn: +Category:Community

Here is a quick list of different test systems (and test projects) for Linux:

This information has also been posted to the OSDL test-tools wiki.

See: http://developer.osdl.org/dev/test_tools/index.php/Test_Projects

There was a testing summit held at [[Linux World]] in San Francisco in August of 2006.
The minutes from the meeting are at:
http://lists.osdl.org/pipermail/test-tools/2006-August/000000.html

As a result of the meeting, a mailing list and wiki were created.
These resources are intended to be used to coordinate the efforts
of the disparate testing projects in Linux. The list info is at:
https://lists.osdl.org/mailman/listinfo/test-tools

The wiki is at:
http://developer.osdl.org/dev/test_tools/index.php/Main_Page

== old page content ==
This content was moved to the OSDL test-tools wiki at:
http://developer.osdl.org/dev/test_tools/index.php/Test_Projects

It is preserved here only temporarily. (TRB - Aug, 2006)

ABAT - https://sourceforge.net/projects/abat/
- does a download/build/boot regression test for several mainline kernel trees, as soon as new versions are available
- test results matrix is available here:
- http://ftp.kernel.org/pub/linux/kernel/people/mbligh/abat/regression_matrix.html

Linux Test Project
- http://ltp.sourceforge.net/

CELF's Proposed Test Lab
- [[Open Test Lab]]

[[K Auto Build]] - http://armlinux.simtec.co.uk/index.html
- [[KAuto Build]] project for ARM architectures
- Article about the site is at: http://www.linuxdevices.com/news/NS4646877354.html

There was a testing summit held

I wanted to report that there was a testing summit held at
[[Linux World]] in San Francisco a few weeks ago. I represented
the CELF test lab effort at the meeting.

[[Category:Community]]

Test Lab Server Spec

2008-02-10T21:10:11Z

Glenn: +Category:Community

This is a specification the CELF Test Lab Server

VERSION 0.1 - for other versions, click the "info" button.

'''Table Of Contents:'''

== Introduction ==
This specification describes the configuration and conventions used for the CELF Open Test Lab
server. This includes conventions and interfaces between the server and lab clients, and
between the server and lab hosts.

This document is related to the [[Test Lab Architecture]] document, where the overall
architecture of the lab is described.

=== Rationale ===
This specification is needed in order to:
* define the detailed interactions and interfaces between the server and other test lab elements

== Definitions ==
; host : : A machine used to control or access one or more target boards.

; client : : A machine used to request lab services

; server : : The machine that is the central server for the CELF test lab.

=== Server ===
The lab server has many distinct roles in the operation of the lab:
* it holds reservations (scheduling) of lab machines for interactive use [''I'm still deciding on this one - reservation info could be stored on the host'']
* it is a web server with lab information and results publication
* it is a repository for tests and test suites
* it holds information about requested automated tests (test jobs)
* it collates nightly regression test results

Note that the server is almost entirely passive. As can be seen below, the host
machines have the major role in initiating test activities as part of
automated testing. This more easily allows a single test to be performed
at disjoint times. That is, a single test that is targetted at multiple
boards run on each target independently. Instead of requiring "gang" scheduling
of the targets, this allows much more flexibility in scheduling the job.

Also, the nodes for which a job is request do not need to be "online" (actively
connected to the lab or the server) when a test is scheduled, or even when the
job is run on other nodes. This makes it possible to request and run jobs
on nodes that are only intermittently available to the lab.

Another reason for this is to make it easier to set up automated private testing.
If the server initiated testing, it would need an ssh account on the private
machine. In a "host-pull" model, only web access to the server is required from the host,
which is much easier to configure for private machines (which are often behind
strict corporate firewalls).

== Use Cases ==
See [[Test Lab Architecture]] for detailed steps for each of the major lab use cases.

== Required Interfaces and Capabilities ==

/\ [ This section is still under construction ]

=== Desired operations on the host machine ===
This section describes in simple sentences, the requirements on the server machine.
After each statement, in parenthesis, is a possible command that
could be used to implement the required operation.

[not written yet.]

=== Required commands ===
== Notes (informational and non-normative) ==
[additional information or clarifying comments]

== References ==
[ pointer(s) to implementation, open source project(s), POSIX specification, etc.]

=== Related work ===
[none listed so far]

== Remaining Issues ==
[this is a placeholder section for listing issues while the spec is under development.
It should be empty when the spec is completed (or the issues should be deferable to
a subsequent version of the spec).]

[[Category:Community]]

Test Lab Programs

2008-02-10T21:09:54Z

Glenn: +Category:Community

Here is information about programs used in the CELF Open Test Lab:

Table Of Contents:

== Programs ==
* 'target' is a program to build software and control a target board from a host machine
** 'target' is described at [[Target Program Usage Guide]], and is downloable from that page.
* a sample test which uses 'target' is available, downloadable here: [[Media:preset-test.py]]
* some lab software may eventually find its way to the [http://sourceforge.net/projects/otl-progs/ otl-progs project] on SourceForge.net.

== Ideas ==
=== Using git bisect to find the offending patch ===
At the 2005 Linux Kernel Summit, when Tim presented the idea of providing size
regression data to the kernel developers, Linus recommended that the individual
that caused the regression be identified by the automated test.

Here's how Linus described how to use "git bisect" to find a broken patch.
(from LKML, 9/20/2005)

<pre>
It is. Just get the "id" file that is associated with a snapshot, and it

gives the git commit ID for that state.

So for example, the 2.6.14-rc1-git3 snapshot is associated with the ID
file patch-2.6.14-rc1-git3.id, which contains

v2.6/snapshots(0)$ cat patch-2.6.14-rc1-git3.id
065d9cac98a5406ecd5a1368f8fd38f55739dee9

so once you know that something broke between rc1-git3 and rc1-git4, you

can now do

git bisect start
git bisect good 065d9cac98a5406ecd5a1368f8fd38f55739dee9
git bisect bad bc5e8fdfc622b03acf5ac974a1b8b26da6511c99

and off you go..

Linus
</pre>

[[Category:Community]]

Test Lab Board Suppliers Guide

2008-02-10T21:09:40Z

Glenn: +Category:Community

Table Of Contents:

== Introduction ==
This document describes the features required and procedures to follow to put
a board in the CELF open test lab (see [[Open Test Lab]]). This document is primarily
targeted at semi-conductor vendors or product vendors who wish to add a board
to the lab.

The requirements section describes:
* what is required of the firmware
* what is required on the host
* what is required on the target
* hardware required to control reset and power on the target

The procedures section describes:
[ more stuff, see below]

== Overview ==
Each target board in the lab will be connected to a single Linux host,
which will be an x86 Linux desktop machine. A single host may connect to
more than one target. On the host will be installed
a complete Linux development environment, as well as cross-compilers, tools,
and any special programs required to access and control the target board.

Because the lab uses the boards in an automated fashion
it is required that the host be able to perform a number of
control operations via (unattended) Linux command line programs.
This includes:
* compiling and installing a kernel for use on the board,
* installing a new root filesystem for use on the board,
* compiling and installing user-space programs for use on the board

== Desirable configurations ==
=== Preferred configuration (1) ===
The preferred configuration for testing most features of the board
will be as follows:
* target connected to host via ethernet
* target connected to host via serial line
* kernel loaded from host via tftp
* root file system mounted from host via NFS
* root account with no password
* telnetd running on target, with root account accessible
* ability to reboot target from target command line
* ability to reset target from host command line
* ability to power cycle target from host command line
* Linux console available over serial line connected to host

=== Desired alternate configuration (2) ===
Some tests results will be better if the kernel
and root filesystem more closely match those of a shipping
product. For these tests, the following configuration is
highly desirable:
[ same as above, except:]
* kernel loaded from local storage (either flash or rotating media)
* root file system accessed/loaded from local storage
* mechanism for file transfer to/from target

This will require programs on the host to install the kernel
and root filesystem to target-local storage. This can
be accomplished via any one of: interaction with a known-good Linux
system operating on target, serial output to target firmware,
or interaction with board via secondary hardware (such as a jtag module, if one
is provided to the lab). However, in any case, the
control of these operations needs to be automatable,
and needs to be operable in spite of previous Linux kernel
boot failures. (That is, the control program on the host
needs to be able to reboot the machine and re-install
the kernel via good-kernel, firmware or hardware (even
if the last Linux kernel on the hardware was dead), WITHOUT
manual user intervention (such as changing board pins).)

== Requirements ==
The following items are needed for each board:
* cross-compile toolchains
** gcc is required
* kernel source code (and a location to get updates)
** default kernel configuration for the target board
* libraries and headers for user-space applications
** glibc is required
** uclibc as an alternate option is desirable but not required
* a binary distribution (user-space libraries and programs) for the board
** glibc is required
** busybox is required
** uclibc is desirable but optional
* mechanism to install a kernel via host command line
* mechanism to install a root filesystem via host command line
* mechanism to copy a program to the target file system
* mechanism to copy a program from the target filesystem to the host
* mechanism to reset and/or reboot the machine from
the host Linux command line.
** Sony has used a very simple control module connected to the parallel port of the host machine and the reset switch and power switch pins on the target board, with information available at:
http://tree.celinuxforum.org/CelfPubWiki/TargetSwitchControlFromParallelPort
** anything similar would be fine.
** if absolutely required, we can install a console switcher (which allows for power cycling via a telnet command), but I'd prefer something less drastic than cutting and restoring wall power to the board.

== Procedures ==
This section documents the following procedures related to board management in the CELF Open Test Lab:
* pre-testing a board in a lab configuration
* adding a board to the lab
* maintaining Linux software for the board in the lab
* providing hardware support for the board in the lab
* removing a board from the lab

=== Board pre-test ===
The board supplier should pre-test their configuration of the board
by running a sample test at their own facility, prior to sending
the board to the CELF open test lab facility.

This can be done by installing the necessary software on the host
machine and target machines, connecting the machines together,
and running the sample test. If the sample test is successful,
then the board can be sent to the lab with greater assurance
that it can be integrated without problems.

Here is an outline of the steps involved:
# download the 'target' program (with sample test: 'preset-test.py')
# install the cross toolchains
# install the kernel source
(this can be as simple as putting the kernel source in a tar file)
*** it is preferable that the kernel source be in the format: baseline (kernel.org) source + patches
# install the root filesystem
# install (or create) auxiliary programs for resetting and rebooting the target
# install the 'target' program
# configure the 'target' program by creating an appropriate '/etc/target.conf' file
# run preset-test.py

=== Adding a board to the lab ===
Here are the steps for adding a new board to the CELF open test lab:
* perform the board pre-test
* package the software for lab
** create tar files for toolchains
** create patch sets for kernel
** create tar file with auxiliary programs
* send the software to the lab:
** testlabmanager@tree.celinuxforum.org
** [or upload it to the lab web site - when web site is active - not yet]
* ship the board
** San Jose lab address:
please send it to:

<pre>
attn: Tim Bird
Sony Electronics
3300 Zanker Road, SJ3B6
San Jose, CA, 95134
</pre>

or directly to the lab at:

<pre>
CE Linux Forum Lab Manager
Office #171
50 Airport Parkway
San Jose, CA 95110-1011
</pre>

Since the lab is unmanned most days, please notify the lab manager
when you sending hardware directly to the lab, so we can know to
go in to the office to receive it.

** Tokyo lab address:

<pre>
[ need Tokyo lab address here ]
</pre>

* provide a board support contact

=== Updating board-specific software the lab ===
[describe updating kernel source or patches here.]

=== Providing hardware support to the lab ===
Each board supplier should provide contact information for a person
from their organization that can provide support assistance for their
target board in the lab. This person will be contacted by the lab
manager of the lab where their board is located, in the event that
there is an unrecoverable hardware failure of the board.

=== Removing a board from the lab ===
* contact the lab manager
* provide shipping information

[[Category:Community]]

Test Lab Architecture

2008-02-10T21:09:22Z

Glenn: +Category:Community

'''Table Of Contents:'''

== Introduction ==
This document describes the architecture of the CE Linux Forum Open Test Lab.
The architecture of the lab is intended to support multiple activities envisioned
for the lab. See [[Open Test Lab]] for an introduction and description of the
purpose of the lab.

=== Access use cases ===
There are four broad types of activities that the test lab is intended to support:
- interactive software development and testing (single-user testing on a remote board)
- custom automated testing against multiple boards (a single user running an ad-hoc test remotely)
- automated regression testing against lab boards (lab-driven tests against multiple boards)
- private testing (accessing tests from the lab to run against a local (possibly private) board)

On the [[Open Test Lab]] page, these are described as:
* Interactive board usage via remote access
* Automated multi-platform testing
* Nightly regression tests
* Private test usage

== Definitions ==
; server : : A single machine in the lab is used as the main server for lab operations. This machine

processes board reservations, coordinates nightly regression tests, is a repository for lab test software,
and provides information and results for the lab to the public.
; host : : The machine used to control or access one or more target boards.

; node : : The collection of machines consisting of a single host and one or more targets.

; client : : A machine that accesses the lab for testing services.

; target : : A development board which can be accessed by the lab software.

; target software : : The collection of software that is run on the target. The target software is

compiled on the host machine, and transferred to the target machine for execution. This includes
the kernel, and user-space libraries, daemons, utilities and other programs.
; toolchain : : The collection of programs used to create software for a target machine. This

consists of the compiler (including preprocessor), assembler, linker and other programs used
for manipulating source and machine code into a form usable on the target. This usually consists
of software from the Linux packages for gcc, binutils, and glibc. The toolchains are run
on the host machine of a node.

A host and target combination (a node) can be located anywhere in the world - either physically in
the main lab, in a satellite lab, or at some point on the Internet. Operationaly, it is useful to
distinguish between those that are made available for public use (and are managed by CELF or an affiliated
entity), and those that are used used privately. The following terminology is used for these
different nodes:

; lab node : : An host/target combination managed by CELF

; private node : : An host/target combination NOT managed by CELF.

== Architecture Diagram ==
Here is a diagram of the architecture of the CELF Open Test Lab.
{|
|-
| [[Media:CELF-test-lab-diagram2.png]]
|}

=== Server ===
More details about the server for the lab are at [[Test Lab Server Spec]].

One important attribute of the server that is reflected strongly
in the architecture is that the server plays a very passive
role in the lab. As can be seen below, the host
machines have the major role in initiating test activities as part of
automated testing. See the [[Test Lab Server]]

== Use Cases ==
=== Interactive use of a board remotely ===
A developer wants to use an individual board for interactive testing. In this mode,
the developer follows certain high-level steps to work with the board:

Steps:
* the developer identifies a board using information on lab web site
** each board should have a page on the lab wiki
* the developer reserves a time slot to use the board
** reservation information is maintained for each board on its associated host
** the server presents a web form to the user for reserving time on a board
**** the client accesses the server the using http
**** the server runs a command on the host make or change a reservation
* the developer logs in to the host for that board (using ssh)
** this requires an account that the client can use, on the host
** lab hosts have static IP addresses that can be accessed from outside the lab
**** the server has host-target mappings, and host IP addresses
* the developer compiles the software for the board
** user can compile on the lab host, or send binaries to the host from their own client
** user can supply their own configuration and patches for software on the host
** lab host has toolchains, kernel source and linux distribution for each target it controls
* the developer installs the software to target (or use directly from host - e.g. nfs-mounted root filesystem)
* the developer reboots target, from the host
* the developer uses a serial console or network login from host to interactively work on the target
* the developer (on the target) runs tests, examine results, etc.
* the developer can collect results onto host, and transfer them back to client, if desired

In the lab diagram, an example of this would be client A accessing Target 1, through Lab host A.
The reservation would be sent to the server. The host would grant the reservation after checking
with the server.

=== Automated multi-platform testing ===
In this scenario, a developer uses the lab to perform the same test on multiple boards.
This could be a small subset of boards, or conceivably all the boards registered with the
lab.

* the developer identifies a set of boards using information on lab web site
* the developer uploads the test software and test paramaters to the server
* the host checks with the server periodically, and initiates a test that is requested for one of its boards
** the host uses http to access the server
* the host downloads the software for the test from the server
** test programs for the target must be packaged somehow /\ [''specifications for host and client test package formats are needed'']
* the host compiles the software for the board (if appropriate)
** the host has toolchains, kernel source and linux distribution for each target it controls
* the host installs the software to target
* the host reboots the target
* the host runs the tests on the target
* the host collects the results
* the host uploads the results to the server
* the server collates the results and publishes them on the web site

In the lab diagram, an example of this would be client A requesting a test to be run on
Target 1 through Lab host A and target 3 through lab host B.

=== Automated Periodic Regression Test ===
In this case, a lab administrator (CELF) sets up a test to be run on a periodic
basis (e.g. nightly), on a certain set of boards.

* the administrator identifies a set of boards using information on lab web site
* the administrator reserves a periodically-repeating time slot for those boards
* the administrator uploads the test software and test paramters to the server
* the host initiates the test, at the start of the reserved time slot
* the host checks for updates to the linux kernel or other software involved in the test
** when a new version is detected, the host downloads the code automatically
* the host compiles the software for the board (if appropriate)
** the host has toolchains, kernel source and linux distribution for each target it controls
* the host installs the software to target
* the host reboots the target
* the host runs the tests on the target
* the host collects the results from the target
* the host uploads the results to the server
* the server collates the results and publishes them on the web site
* the server may do additional processing of the results, to provide enhanced feedback to the community
** for example, the server may do an automated binary search for the patch that caused a a particular regression, and report that to the community

=== Private testing ===
This mode of usage allows an individual to use all the tests on the server, in
an interactive or automated fashion, on their own machines.
* PRE-REQUISITE - the user verifies that their host/target combination interoperates with the lab system
** the user downloads an interoperability test and runs it
* the developer selects a test on the test server
** the available tests are listed on the lab server (wiki)
* the developer downloads the test
* the developer runs the test on their local host/target combination

An example of this would be Client B downloading tests from the lab server, and running them
locally (as a host) to run a test on private target 1.

== Required Interfaces and Capabilities ==
There are multiple elements in the lab system, with interfaces between each one.

This section describes the interfaces between different elements, in terms of
protocols, formats and conventions. The model of interaction (push vs. pull,
synchronous vs. asynchronous, interactive vs. automated) is also described.

/\ ''Note - This section is still under construction''

=== Interface between client and server ===
A client access the server via http. The server runs a web server, with a
wiki to provide data about each board in the system.

Jobs for each host to run are located on the server.
A reservation for interactive use of a board is a special type of job.
Many jobs will not have a specific time slot.

* protocols:
** http
* data:
** web page of information about each target board
** web page per job
*(the schema is not yet fully defined, but it includes:
***** lab node for job (host:target)
***** requesting account
***** requested start time (specific time or ANY)
***** requested duration (specific time or NONE)
***** duration estimate (average of previous durations of this test on this target)
***** priority (1=high, 5=low)
***** status (one of: pending, in-progress, complete)
***** current client (in the case of an active reservation)
***** result summary (one-line description of result)
***** result detail (multi-line description of result)
***** test script
** web page showing list of jobs and reservations per target
** user account

When a job is submitted, the server creates a page for
each lab node for which the job is requested. The server rejects
job requests or reservations which conflict with those already
scheduled for a node. (This can only happen with job requests
with a specific time or duration.)

=== Interface between host and server ===
A server does not directly initiate work on a host. Rather, the host
initiates contact with the server, to obtain reservation requests.
At regular intervals, the host downloads the list of its jobs
and determines which one it will "execute". At the end of a job,
the host uploads the results and completion information to the server.

The host also uploads its target information and status periodically
to the server.

* protocols:
*** http
* data:
*** web page of information about each target board (schema not defined yet)
*** web page per job
*** web page with list of jobs per target

During a reservation, the host does nothing automated. That is,
is doesn't run a script from the server, but it might
record secure-shell access during the reservation time slot, and report
the activity log as the result of the "job". This could be used to
detect wasted time slots (if no one logs in).

=== Interface between host and target ===
The interface between the host and the target is specified in [[Remote Board Access Spec]]

Note that there is no specified interface between a client and a target,
or between a server and a target. Targets are only accessible via
a host machine.

The host machine is the initiator of all activity in the system.
The target machine is a slave to the host.

=== Interface between host and Internet ===
A host may be required to download software directly from the Internet
as part of building the software for a target. A host needs http access
to the Internet.

* protocols:
** http

=== Interface between client and host ===
The only direct interface between a client and a host is
interactive command line access, via ssh.

* protocols:
** ssh (from client to host)
* conventions:
** the default command shell for an incoming client is bash
** programs and utilities that are available on the host are specified in [[Remote Board Access Spec]]

At the time of an ssh login, the host machine may validate the
client's reservation by contacting the server. (Or, it may "know" that this
client is allowed to access the machine, by having recently checked with the server).
The host may update the status of the job with the list of the current user for
the target.

=== Desired operations on the host machine ===
This section describes in simple sentences, the requirements on the host machine for
this architecture. After each statement, in parenthesis, is a possible command that
could be used to implement the required operation.

- A user MUST be able to log in to the host machine remotely (ssh)
- A user SHOULD be able to verify that the target machine is available for use ("target status")
- A user MUST be able to reserve a target for use, for a period of time ("target acquire [timeout]")
- A user SHOULD be able to unreserve a target, or relinquish a held reservation ("target release")
- A user or program MUST be able to build the Linux kernel for the target, on the host
- obtain kernel source for target ("target get_kernel")
- set up build environment for kernel ("target setenv")
- get default kernel configuration ("target get_config")
- set specific configuration paramenters ("target set_config [options]")
- A user or program SHOULD be able to build a root filesystem for target, from source
- A user or program MUST be able to build an arbitrary C program for target
- ability to put source for arbitrary program on the host???
- set up build environment for programs (target setup_build_environment???)
- A user or program MUST be able to install kernel ("target kinstall")
- A user or program MUST be able to install a program ("target cp")
- A user or program SHOULD be able to install a new rootfs ("target fsinstall")
- A user or program MUST be able to transfer files from the target to the host ("target cp")
- A user or program MUST be able to restart a target machine, by at least one of the following methods:
- resetting the target hardware ("target reset")
- power cycling the target machine ("target reboot")
- A user MUST be able to interactively access the Linux console of the target("target console")
- A user SHOULD be able to interactively access a target command shell ("target login")
- A user or program MUST be able to execute an arbitrary command on the target ("target run")

- A host machine MUST periodically poll the server for it's list of tasks (wget)
- A host machine MUST be able to download materials from the lab server (wget)
- A LAB host machine MUST respond to a request for reservation information from the server ???
- The server MUST be able to query a LAB host machine for it's reservation status ???

[[Category:Community]]

Test Farm Project

2008-02-10T21:08:56Z

Glenn: +Category:Community

Table Of Contents:

== Description ==
This page describes the CELF embedded development board test farm project. The purpose of this
project is to provide benefits to open source developers, CE developers and embedded board
vendors.

=== Rationale ===
This project is important to CELF because it solves a few different problems in the open source community:
* developer access to hard-to-obtain boards
* developer access to a variety of boards and processors
* target board testing by a variety of developers

== Resources ==
=== Projects ===
* http://l4x.org/k/ - Linux Kernel Cross Compile Test Bench, by Jan Dittmer
* http://openembedded.org/ - Open Embedded project (with Tim Ansell and Philip Blundell)
** Mailing list archives at: http://news.gmane.org/gmane.comp.handhelds.openembedded
** Presentation on using bitbake and OE - http://www.vanille.de/tools/FOSDEM2005.pdf
* http://armlinux.simtec.co.uk/index.html - [[KAuto Build]] project for ARM architectures
** Article about the site is at: http://www.linuxdevices.com/news/NS4646877354.html

=== Specifications ===
See [[Remote Board Access Spec]]

Also, we need to document rtest.

== Downloads ==
Nothing right now.

== How To Use ==
The following high-level usage scenarios are envisioned:
* an open source developer logs in to a host machine and runs tests manually on a specific, individual target board
* an open source developer submits a patch and compiles it (and tests it) on many different boards
* CELF runs a large test suite on a periodic basis, and delivers the results to the open source community
(possible audiences would be kernel arch maintainers, toolchain developers, and target board vendors)

Either of the first two scenarios would be available to a CE vendor engineer (CELF member) acting as
an open source developer.

== Future Work/Action Items ==
Here is a list of things that could be worked on for this feature:
- publish the spec.
- publish the hardware specs for the remote reset switch
- release the existing programs for comment
- finish the programs enough to perform first compile-time testing
- run a pilot program
- get a board server set up somewhere
- get a set of pilot boards
- Samsung, Renesas, TI (OSK)??
- ask for volunteers to test the program
- Erik Andersen, Matt Mackall, Paul Mundt, SZWG
- get the first test suite
- [[Linux Tiny Test Project]]

[[Category:Community]]

Technology Watch List

2008-02-10T21:08:25Z

Glenn: +Category:Community

This page lists technologies and projects that CELF
members are interested in the status of. This includes kernel patches,
new technology research, and middleware and user-space projects of key interest
for consumer electronics products. The projects may be
the topics of discussion at CELF meetings, and we plan to watch and
report the status of these technologies.

Please add any information you have about the technology items listed below!!

== Latest Watchlist ==
The '''Status''' field in the table below indicates whether this feature is on track for being
mainlined. The '''When was last activity''' field indicates the kernel version number or date when
the last activity was noted for this feature. This could be the last kernel version where
bits from this patch were mainlined, or the last date of visible feature development activity outside
the main tree.

== Kernel Stuff ==
{| border="1" cellspacing="0" cellpadding="5"
|-bgcolor="#80c0d0"
!Technology, Feature or Patch
!Status
!When was last activity
!Notes
|-
|[http://elinux.org/Linux_Tiny Linux-tiny] || Now in active development. Patches published for 2.6.23 || Latest patches were published Oct. 13, 2007 || Maintainer is Michael Opdenacker. See http://elinux.org/Linux_Tiny for details about the patch status. Michael presented some good results at ELC Europe.
|-
|squashfs ||Latest release is 3.3, released Feb 5, 2008 (See [http://www.nabble.com/-ANN--Squashfs-3.3-released-tt13541973.html Squashfs 3.3 released]. Not mainlined. Phillip Lougher said he was aiming for Oct 2007 mainline attempt. Unfortunately, an accident involving his hand has slowed things down. ||Last mainline attempt was over a year ago. || -
|-
|AXFS ||Not mainlined yet. ||Last mainline attempt was summer, 2007 ||Jared will try to mainline again soon??
|-
|LogFS ||not mainlined. ||? ||CELF hired Jörn Engel in Dec, 2008 to complete and mainline a first release of logfs
|-
|LTTng ||core not mainlined. Markers were mainlined in 2.6.24 ||? || LTTng instrumentation bits were changed to use markers, in early 2007
|-
|[http://elinux.org/System_Tap SystemTap] (and Kprobes) for non-i386 arches || ARM support merged for 2.6.25 ||? || KProbes ports for ARM, MIPS and PPC32 were reported on at ELC 2007, SystemTap for SH was demo'ed at ELC 2007
|-
|kpagemap - memory instrumentation ||mainlined in Feb, 2008 (for 2.6.25) ||Feb 2008 || Linus asked Matt to do some other work related to kpagemap.
|-
|KFT ([http://elinux.org/Kernel_Function_Trace Kernel Function Trace])||not mainlined - broken on ARM (with gcc > 4.x), PPC64 has problems (reports parent funcs of inlines). Nicholas McGuire is taking over maintainership from Tim Bird, with funding from CELF||last published external patches for 2.6.21|| -
|-
|kernel trace system (RT-preempt latency-trace, refactored for general use)||not mainlined||Feb 2008 ||version 8 of the patches was submitted by to LKML by Steven Rostedt in Jan, 2008
|-
|printk-times (arch support) ||fully mainlined? ||April, 2005 ||Some arches had problems with accessing the clock too early in the kernel bootup sequence, but a new setup routine defers turning on the timestamping until after timekeeping is initialized
|-
|RT-preempt ||some parts mainlined (last part was high res. timers in 2.6.21) ||2.6.21? || Next target is to integrate threaded interrupts in 2.6.23?? Still not integrated threaded interrupt in 2.6.23
|-
|App Armour ||not mainlined ||May, 2007 || Some kernel developers still have objections to path-based security[[BR]][http://lwn.net/SubscriberLink/254740/f71fe8e26c906233/ LWN.net] mention App Armour
|-
|[http://elinux.org/TomoyoLinux TOMOYO Linux] || not mainlined || [http://lwn.net/Articles/258905/ Nov 17, 2007] (4th post) [http://elinux.org/TomoyoLinux#Mainline (trying now)] || "TOMOYO Linux has only recently surfaced on the wider mailing lists; its reception has not been entirely friendly. This project's developers have some work to do if they are (1) to get past the same obstacles which have slowed AppArmor, and (2) show that their project is sufficiently different from AppArmor to merit inclusion as yet another security framework." (from [http://www.linux-foundation.org/en/Linux_Weather_Forecast/security Linux Weather Forecast])
|-
|powertop ||? ||? || ACPI version available; reported to be portable to non-Intel architectures, but no attempt known
|-
|PM QoS ||in 2.6.23-mm1 || Oct '07 || (see http://lesswatts.org) need Embedded folks to take a look and help define the interface, expand the features and raise issues from the embedded perspective.
|-
| Userspace I/O || Seems to be merged into mainline (see: http://lwn.net/Articles/242483/ ) ||July, 2007 || References: http://www.kroah.com/log/linux/uio.html
|-
| Kernel How To (Japanese and Chinese translation) || Seems to be merged into mainline ||July, 2007 || -
|}

== Middleware ==
{|border="1" cellpadding="5" cellspacing="0"
|-bgcolor="#80c0d0"
!Project
!Status
!When was last activity
!Notes
|-
|libdlna || Developer has added support for all profiles except MPEG-4 and WMV ( http://hg.geexbox.org/libdlna/ ) ||29 Aug, 07 || Short term goal is to provide DLNA support to Ushare media server, long term goal is to provide generic DLNA reference library [[BR]] References: http://libdlna.geexbox.org/
|}

[[Category:Community]]

Project Status Table

2008-02-10T21:05:50Z

Glenn: +Category:Community

This is the main project list for the forum:
{|
|- bgcolor="80C0c0"
| align="center" | '''Project'''
| align="center" | '''Page'''
| align="center" | '''WG'''
| align="center" | '''Status'''
| align="center" | '''Implementation'''
| align="center" | '''Next Action'''
|- bgcolor="80C0c0"
| align="center" | .
| .
| .
| .
| x86
| ARM
| PPC
| MIPS
| SH
| .
|- bgcolor="80C0c0"
| align="center" | '''Bootup Time projects'''
|-
| KFI
| [[Kernel Function Instrumentation]]
| BT
| bgcolor="#00c080" | documented
| bgcolor="#80c0ff" | 5
| 0
| bgcolor="#80c0ff" | 5
| 0
| 0
| send to LKML
|-
| printk-times
| [[Printk Times]]
| BT
| bgcolor="#00c080" | documented
| bgcolor="#80c0ff" | 5
| 0
| bgcolor="#80c0ff" | 5
| 0
| 0
| send to LKML
|-
| LTT
| [[Linux Trace Toolkit]]
| BT
| bgcolor="#ffc060" | researched
| 0
| 0
| bgcolor="#80ff80" | 3
| 0
| 0
| Need to test PPC fix
|-
| Preset lpj
| [[PresetLPJ]]
| BT
| bgcolor="#80c0ff" | accepted
| bgcolor="#80c0ff" | 5
| bgcolor="#80c0ff" | 5
| bgcolor="#80c0ff" | 5
| bgcolor="#80c0ff" | 5
| bgcolor="#80c0ff" | 5
| Done
|-
| bgcolor="#80C0c0" align="center" | '''System Size projects'''
|-
| Linux-tiny
| [[Linux Tiny]]
| SZ
| bgcolor="#ffc060" | researched
| 0
| 0
| 0
| 0
| 0
| fill in this status
|-
| bgcolor="#80C0c0" align="center" | '''Other projects'''
|}

Key for status codes:
{|
|- bgcolor="80c0c0"
| align="center" | '''Status codes'''
|-
| bgcolor="#ff8080" | not started/no progress
|-
| bgcolor="#ffc060" | researched
|-
| bgcolor="#ffff80" | implemented
|-
| bgcolor="#80ff80" | measured
|-
| bgcolor="#00c080" | documented
|-
| bgcolor="#80c0ff" | accepted
|}

Key for Implementation codes:
{|
|- bgcolor="80c0c0"
| align="center" | '''Implementation color key'''
|-
| 0(no work done)
|-
| bgcolor="#ffc060" | 1 (patches apply cleanly)
|-
| bgcolor="#ffff80" | 2 (compiles)
|-
| bgcolor="#80ff80" | 3 (runs)
|-
| bgcolor="#00c080" | 4 (works)
|-
| bgcolor="#80c0ff" | 5 (accepted in kernel.org)
|}

[[Category:Community]]

Project List

2008-02-10T21:05:30Z

Glenn: +Category:Community

Here is a list of possible projects that member companies might have worked
on or might work on in the future, in the CE Linux forum:

/\ NOTE: This is just a list of potential areas of interest. A listing here
is NOT a commitment by a company or the forum to work on, endorse or promote
a particular technology area or project.

[[Category:Community]]