Difference between revisions of "Android Kernel Features"

From eLinux.org
Jump to: navigation, search
(other kernel changes: add ram console)
(Android mainlining project)
(30 intermediate revisions by 4 users not shown)
Line 1: Line 1:
== List of Kernel features unique to Android ==
+
== Kernel features unique to Android ==
 +
In the course of development, Google developers made some changes to the Linux kernel.  The amount
 +
of changes is not extremely large, and is on the order of changes that are customarily made to
 +
the Linux kernel by embedded developers (approximately 250 patches, with about 3 meg. of
 +
differences in 25,000 lines).  The changes include a variety of large and small additions, ranging
 +
from the wholesale addition of a flash filesystem (YAFFS2), to very small patches to augment Linux
 +
security (paranoid networking patches).
 +
 
 +
Various efforts have been made over the past few years to submit these to changes to mainline
 +
(mostly by Google engineers, but also by others), with not much success so far.
 +
 
 +
=== Resources ===
 +
A very good overview of the changes is available in a talk by John Stultz at ELC 2011.
 +
(The talk has a somewhat misleading name.)
 +
* [[Media:Elc2011_stultz.pdf|Android OS for Servers]]- John Stultz, ELC 2011
 +
** This talks breaks down the differences between an Android Linux kernel and a stock Linux kernel, and provides information about the features of each.
 +
* http://www.lindusembedded.com/blog/2010/12/07/android-linux-kernel-additions/
 +
** Lindus Embedded (Alex Gonzalez) has a listing of kernel changes based on an Android kernel for the Freescale MX51 SOC, with some good information about each change.
 +
* http://www.linuxfordevices.com/c/a/Linux-For-Devices-Articles/Porting-Android-to-a-new-device/
 +
** Peter McDermott's excellent description of his work to port Android to the Nokia N810.
 +
** Also, see his annotated list of modified and added kernel files, at: http://www.linuxfordevices.com/files/misc/porting-android-to-a-new-device-p3.html
 +
* http://www.slideshare.net/jollen/android-os-porting-introduction
 +
** Jollen Chen's excellent presentation on system-level Android features, including an overview of kernel features unique to Android: ''Note: Parts of the presentation are in Chinese''
 +
 
 +
 
 +
 
 +
=== Temporary inclusing in mainline 'staging' ===
 +
Some changes were temporarily adding the "staging" driver area in the stock kernel, but were removed due to lack of support. See [http://www.kroah.com/log/linux/staging-status-12-2009.html Greg KH blog post on -staging for 2.6.33], where he announces to '''remove''' various Android drivers from -staging.
 +
 
 +
=== Android mainlining project ===
 +
Several groups and individuals are working to get kernel changes from Android mainlined into the Linux kernel.
 +
 
 +
Please see [[Android Mainlining Project]] for more information.
 +
 
 +
== List of kernel features unique to Android ==
 +
Here is a list of changes/addons that the Android Project made to the linux kernel.
 +
As of September, 2011, these kernel changes are not part of the standard kernel and are only available in the Android kernel trees in the Android Open Source project.
 +
 
 +
This list does not include board- or platform-specific support or drivers (commonly called "board support").
 +
 
 
=== Binder ===
 
=== Binder ===
* Binder - corba-like IPC
+
Binder is an Android-specific interprocess communication mechanism, and remote method invocation system.
** used instead of SysV IPC for interprocess communication
+
 
** The Linux version of Binder was originally derived from a project by PalmSource to implement a CORBA-like message-passing or method invocation system. Documentation on that system is at: http://www.angryredplanet.com/~hackbod/openbinder/docs/html/index.html
+
See [[Android Binder]]
** implementation is at: <tt>drivers/android/binder.c</tt>, with include file: <tt>include/linux/binder.h</tt>
 
  
 
=== ashmem ===
 
=== ashmem ===
Line 29: Line 67:
 
* ASHMEM_GET_PIN_STATUS
 
* ASHMEM_GET_PIN_STATUS
 
* ASHMEM_PURGE_ALL_CACHES
 
* ASHMEM_PURGE_ALL_CACHES
 +
 +
From a thread on android-platform [http://groups.google.com/group/android-platform/browse_thread/thread/2fa6ba5ef9f81d22/0b91a39d108d02fc source]
 +
 +
You can create a shared memory segment using:
 +
<pre>
 +
    fd = ashmem_create_region("my_shm_region", size);
 +
    if(fd < 0)
 +
        return -1;
 +
    data = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
 +
    if(data == MAP_FAILED)
 +
        goto out;
 +
</pre>
 +
 +
In the second process, instead of opening the region using the same name, for
 +
security reasons the file descriptor is passed to the other process via binder IPC.
 +
 +
The libcutils interface for ashmem consists of the following calls: (found in system/core/include/cutils/ashmem.h)
 +
 +
* int ashmem_create_region(const char *name, size_t size);
 +
* int ashmem_set_prot_region(int fd, int prot);
 +
* int ashmem_pin_region(int fd, size_t offset, size_t len);
 +
* int ashmem_unpin_region(int fd, size_t offset, size_t len);
 +
* int ashmem_get_size_region(int fd);
  
 
=== pmem ===
 
=== pmem ===
Line 40: Line 101:
 
the MSM7201A, but could be used for other chipsets as well.  For now,
 
the MSM7201A, but could be used for other chipsets as well.  For now,
 
you're safe to turn it off on x86.
 
you're safe to turn it off on x86.
 +
</pre>
 +
 +
David Sparks wrote the following:
 +
[http://groups.google.com/group/android-framework/msg/06bb9d8c294ce6d7 source]
 +
<pre>
 +
2. ashmem and pmem are very similar. Both are used for sharing memory
 +
between processes. ashmem uses virtual memory, whereas pmem uses
 +
physically contiguous memory. One big difference is that with ashmem,
 +
you have a ref-counted object that can be shared equally between
 +
processes. For example, if two processes are sharing an ashmem memory
 +
buffer, the buffer reference goes away when both process have removed
 +
all their references by closing all their file descriptors. pmem
 +
doesn't work that way because it needs to maintain a physical to
 +
virtual mapping. This requires the process that allocates a pmem heap
 +
to hold the file descriptor until all the other references are closed.
 +
 +
3. You have the right idea for using shared memory. The choice between
 +
ashmem and pmem depends on whether you need physically contiguous
 +
buffers. In the case of the G1, we use the hardware 2D engine to do
 +
scaling, rotation, and color conversion, so we use pmem heaps. The
 +
emulator doesn't have a pmem driver and doesn't really need one, so we
 +
use ashmem in the emulator. If you use ashmem on the G1, you lose the
 +
hardware 2D engine capability, so SurfaceFlinger falls back to its
 +
software renderer which does not do color conversion, which is why you
 +
see the monochrome image.
 
</pre>
 
</pre>
  
Line 45: Line 131:
 
* logger - system logging facility
 
* logger - system logging facility
 
** This is the kernel support for the 'logcat' command
 
** This is the kernel support for the 'logcat' command
** The kernel driver for the serial devices for logging are in the source code drivers/android/logging.c
+
** The kernel driver for the serial devices for logging are in the source code <tt>drivers/misc/logger.c</tt>
** See [[Android logger]] for more information
+
** See [[Android logger]] for more information about the kernel code
 +
** See [[Android Logging System]] for an overview of the system it supports
  
 
=== wakelocks ===
 
=== wakelocks ===
* wakelock - used for power management
+
* wakelock - used for power management files <tt>kernel/power/wakelock.c</tt>
 
** Holds machine awake on a per-event basis until wakelock is released
 
** Holds machine awake on a per-event basis until wakelock is released
 
** See [[Android Power Management]] for detailed information
 
** See [[Android Power Management]] for detailed information
Line 58: Line 145:
 
** implementation at: <tt>drivers/misc/lowmemorykiller.c</tt>
 
** implementation at: <tt>drivers/misc/lowmemorykiller.c</tt>
 
** also at: <tt>security/lowmem.c</tt>
 
** also at: <tt>security/lowmem.c</tt>
** [Need a description of this here]
 
  
=== alarm ===
+
Informally known as the Viking Killer, the OOM handler simply kills processes as available memory becomes low. The kernel module follows rules for this that are supplied from user space in two ways:
* alarm
+
 
 +
1. init writes information about memory levels and associated classes:
 +
 
 +
* The write value must be consistent with the above properties.
 +
* Note that the driver only supports 6 slots, so we have combined some of the classes into the same memory level; the associated processes of higher classes will still be killed first.
 +
** From /init.rc:
 +
    write /sys/module/lowmemorykiller/parameters/adj 0,1,2,4,7,15
 +
    write /sys/module/lowmemorykiller/parameters/minfree 2048,3072,4096,6144,7168,8192
 +
 
 +
2. User space sets the oom_adj of processes to put them in the correct class for their current operation. This redefines the meaning of oom_adj from that used by the standard OOM killer to something that is more aggressive and controlled.
 +
 
 +
These oom_adj levels end up being based on the process lifecycle defined here:
 +
http://developer.android.com/guide/topics/fundamentals.html#proclife
 +
 
 +
=== Alarm timers ===
 +
This is the kernel implementation to support Android's AlarmManager. It lets user space tell the kernel when it would like to wake up, allowing the kernel to schedule that appropriately and come back (holding a wake lock) when the time has expired regardless of the sleep state of the CPU.
 +
 
 +
==== POSIX Alarm Timers ====
 +
Note that POSIX Alarm timers, which implement this functionality (but not identically), was
 +
accepted into mainline Linux in kernel version 3.0.
 +
 
 +
See [https://lwn.net/Articles/429925/ Waking Systems from Suspend] and http://lwn.net/Articles/439364/
  
 
=== paranoid network security===
 
=== paranoid network security===
Line 67: Line 174:
 
** See [[Android_Security#Paranoid_network-ing]]
 
** See [[Android_Security#Paranoid_network-ing]]
  
=== timed gpio ===
+
=== timed output / timed gpio ===
 
Generic gpio is a mechanism to allow programs to access and manipulate gpio registers from user space.
 
Generic gpio is a mechanism to allow programs to access and manipulate gpio registers from user space.
  
"Timed gpio" appears to be a system to allow for similar gpio manipulation, but with an added capability
+
Timed output/gpio is a system to allow chaning a gpio pin and restore it automatically after a specified timeout.
to automatically set a value in a gpio after a specified timeout.
+
See <tt>drives/misc/timed_output.c</tt> and <tt>drives/misc/timed_gpio.c</tt>
 
+
This expose a user space interface used by the vibrator code.
On my ADP1, there is a driver at:
 
  
I'm not sure who uses this, or how it is used, but this is a directory with the following items:
+
On ADP1, there is a driver at:
 
<pre>
 
<pre>
 
# cd /sys/bus/platform/drivers/timed-gpio
 
# cd /sys/bus/platform/drivers/timed-gpio
Line 100: Line 206:
 
</pre>
 
</pre>
  
Possibly this means that one of the GPIO pins on the ADP1 is tied to a power wakeup event???
+
=== RAM_CONSOLE ===
 +
This allows saving the kernel printk messages to a buffer in RAM, so that after a kernel panic
 +
they can be viewed in the next kernel invocation, by accessing /proc/last_kmsg.
 +
 
 +
[Would be good to get more details on how to set this up and use it here!]
 +
[I guess this is something like pramfs?]
  
 
=== other kernel changes ===
 
=== other kernel changes ===
 
Here is a miscellaneous list of other kernel changes in the mistral Android kernel:
 
Here is a miscellaneous list of other kernel changes in the mistral Android kernel:
* switch events - drivers/switch/*
+
* switch events - drivers/switch/* userspace support for monitoring GPIO via sysfs/uevent used by vold to detect USB
 
* USB gadget driver for ADB - drivers/usb/gadget/android.c
 
* USB gadget driver for ADB - drivers/usb/gadget/android.c
 
* yaffs2 flash filesystem
 
* yaffs2 flash filesystem
 
* support in FAT filesystem for FVAT_IOCTL_GET_VOLUME_ID
 
* support in FAT filesystem for FVAT_IOCTL_GET_VOLUME_ID
* RAM console
 
 
* and more...
 
* and more...
  
Line 114: Line 224:
 
The file [http://android.git.kernel.org/?p=kernel/common.git;a=blob_plain;f=Documentation/android.txt;hb=HEAD Documentation/android.txt] has a list of required configuration options for a kernel to support
 
The file [http://android.git.kernel.org/?p=kernel/common.git;a=blob_plain;f=Documentation/android.txt;hb=HEAD Documentation/android.txt] has a list of required configuration options for a kernel to support
 
an Android system.
 
an Android system.
 
== Resources ==
 
* Peter McDermott's excellent description of his work to port Android to the Nokia N810.
 
** See http://www.linuxfordevices.com/c/a/Linux-For-Devices-Articles/Porting-Android-to-a-new-device/
 
** Also, see his annotated list of modified and added kernel files, at: http://www.linuxfordevices.com/files/misc/porting-android-to-a-new-device-p3.html
 
 
* Jollen Chen's excellent presentation on system-level Android features, including an overview of kernel features unique to Android: ''Note: Parts of the presentation are in Chinese''
 
** http://www.slideshare.net/jollen/android-os-porting-introduction
 
  
 
[[Category:Android]]
 
[[Category:Android]]

Revision as of 11:27, 12 December 2011

Kernel features unique to Android

In the course of development, Google developers made some changes to the Linux kernel. The amount of changes is not extremely large, and is on the order of changes that are customarily made to the Linux kernel by embedded developers (approximately 250 patches, with about 3 meg. of differences in 25,000 lines). The changes include a variety of large and small additions, ranging from the wholesale addition of a flash filesystem (YAFFS2), to very small patches to augment Linux security (paranoid networking patches).

Various efforts have been made over the past few years to submit these to changes to mainline (mostly by Google engineers, but also by others), with not much success so far.

Resources

A very good overview of the changes is available in a talk by John Stultz at ELC 2011. (The talk has a somewhat misleading name.)


Temporary inclusing in mainline 'staging'

Some changes were temporarily adding the "staging" driver area in the stock kernel, but were removed due to lack of support. See Greg KH blog post on -staging for 2.6.33, where he announces to remove various Android drivers from -staging.

Android mainlining project

Several groups and individuals are working to get kernel changes from Android mainlined into the Linux kernel.

Please see Android Mainlining Project for more information.

List of kernel features unique to Android

Here is a list of changes/addons that the Android Project made to the linux kernel. As of September, 2011, these kernel changes are not part of the standard kernel and are only available in the Android kernel trees in the Android Open Source project.

This list does not include board- or platform-specific support or drivers (commonly called "board support").

Binder

Binder is an Android-specific interprocess communication mechanism, and remote method invocation system.

See Android Binder

ashmem

  • ashmem - Android shared memory
    • implementation is in mm/ashmem.c

According to the Kconfig help "The ashmem subsystem is a new shared memory allocator, similar to POSIX SHM but with different behavior and sporting a simpler file-based API."

Apparently it better-supports low memory devices, because it can discard shared memory units under memory pressure.

To use this, programs open /dev/ashmem, use mmap() on it, and can perform one or more of the following ioctls:

  • ASHMEM_SET_NAME
  • ASHMEM_GET_NAME
  • ASHMEM_SET_SIZE
  • ASHMEM_GET_SIZE
  • ASHMEM_SET_PROT_MASK
  • ASHMEM_GET_PROT_MASK
  • ASHMEM_PIN
  • ASHMEM_UNPIN
  • ASHMEM_GET_PIN_STATUS
  • ASHMEM_PURGE_ALL_CACHES

From a thread on android-platform source

You can create a shared memory segment using:

    fd = ashmem_create_region("my_shm_region", size); 
    if(fd < 0) 
        return -1; 
    data = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); 
    if(data == MAP_FAILED) 
        goto out; 

In the second process, instead of opening the region using the same name, for security reasons the file descriptor is passed to the other process via binder IPC.

The libcutils interface for ashmem consists of the following calls: (found in system/core/include/cutils/ashmem.h)

  • int ashmem_create_region(const char *name, size_t size);
  • int ashmem_set_prot_region(int fd, int prot);
  • int ashmem_pin_region(int fd, size_t offset, size_t len);
  • int ashmem_unpin_region(int fd, size_t offset, size_t len);
  • int ashmem_get_size_region(int fd);

pmem

  • PMEM - Process memory allocator
    • implementation at: drivers/misc/pmem.c with include file at: include/linux/android_pmem.h
    • Brian Swetland says:
The pmem driver is used to manage large (1-16+MB) physically contiguous
regions of memory shared between userspace and kernel drivers (dsp, gpu,
etc).  It was written specifically to deal with hardware limitations of
the MSM7201A, but could be used for other chipsets as well.  For now,
you're safe to turn it off on x86.

David Sparks wrote the following: source

2. ashmem and pmem are very similar. Both are used for sharing memory 
between processes. ashmem uses virtual memory, whereas pmem uses 
physically contiguous memory. One big difference is that with ashmem, 
you have a ref-counted object that can be shared equally between 
processes. For example, if two processes are sharing an ashmem memory 
buffer, the buffer reference goes away when both process have removed 
all their references by closing all their file descriptors. pmem 
doesn't work that way because it needs to maintain a physical to 
virtual mapping. This requires the process that allocates a pmem heap 
to hold the file descriptor until all the other references are closed. 

3. You have the right idea for using shared memory. The choice between 
ashmem and pmem depends on whether you need physically contiguous 
buffers. In the case of the G1, we use the hardware 2D engine to do 
scaling, rotation, and color conversion, so we use pmem heaps. The 
emulator doesn't have a pmem driver and doesn't really need one, so we 
use ashmem in the emulator. If you use ashmem on the G1, you lose the 
hardware 2D engine capability, so SurfaceFlinger falls back to its 
software renderer which does not do color conversion, which is why you 
see the monochrome image. 

logger

  • logger - system logging facility
    • This is the kernel support for the 'logcat' command
    • The kernel driver for the serial devices for logging are in the source code drivers/misc/logger.c
    • See Android logger for more information about the kernel code
    • See Android Logging System for an overview of the system it supports

wakelocks

  • wakelock - used for power management files kernel/power/wakelock.c
    • Holds machine awake on a per-event basis until wakelock is released
    • See Android Power Management for detailed information

oom handling

  • oom handling modifications
    • lowmem notifications
    • implementation at: drivers/misc/lowmemorykiller.c
    • also at: security/lowmem.c

Informally known as the Viking Killer, the OOM handler simply kills processes as available memory becomes low. The kernel module follows rules for this that are supplied from user space in two ways:

1. init writes information about memory levels and associated classes:

  • The write value must be consistent with the above properties.
  • Note that the driver only supports 6 slots, so we have combined some of the classes into the same memory level; the associated processes of higher classes will still be killed first.
    • From /init.rc:
   write /sys/module/lowmemorykiller/parameters/adj 0,1,2,4,7,15
   write /sys/module/lowmemorykiller/parameters/minfree 2048,3072,4096,6144,7168,8192

2. User space sets the oom_adj of processes to put them in the correct class for their current operation. This redefines the meaning of oom_adj from that used by the standard OOM killer to something that is more aggressive and controlled.

These oom_adj levels end up being based on the process lifecycle defined here: http://developer.android.com/guide/topics/fundamentals.html#proclife

Alarm timers

This is the kernel implementation to support Android's AlarmManager. It lets user space tell the kernel when it would like to wake up, allowing the kernel to schedule that appropriately and come back (holding a wake lock) when the time has expired regardless of the sleep state of the CPU.

POSIX Alarm Timers

Note that POSIX Alarm timers, which implement this functionality (but not identically), was accepted into mainline Linux in kernel version 3.0.

See Waking Systems from Suspend and http://lwn.net/Articles/439364/

paranoid network security

timed output / timed gpio

Generic gpio is a mechanism to allow programs to access and manipulate gpio registers from user space.

Timed output/gpio is a system to allow chaning a gpio pin and restore it automatically after a specified timeout. See drives/misc/timed_output.c and drives/misc/timed_gpio.c This expose a user space interface used by the vibrator code.

On ADP1, there is a driver at:

# cd /sys/bus/platform/drivers/timed-gpio
# ls -l
--w-------    1 0        0            4096 Nov 13 02:11 bind
lrwxrwxrwx    1 0        0               0 Nov 13 02:11 timed-gpio -> ../../../../devices/platform/timed-gpio
--w-------    1 0        0            4096 Nov 13 02:11 uevent
--w-------    1 0        0            4096 Nov 13 02:11 unbind

Also, there is a device at:

# cd /sys/devices/platform/timed-gpio
# ls -lR
.:
lrwxrwxrwx    1 0        0               0 Nov 13 01:34 driver -> ../../../bus/platform/drivers/timed-gpio
-r--r--r--    1 0        0            4096 Nov 13 01:34 modalias
drwxr-xr-x    2 0        0               0 Nov 13 01:34 power
lrwxrwxrwx    1 0        0               0 Nov 13 01:34 subsystem -> ../../../bus/platform
-rw-r--r--    1 0        0            4096 Nov 13 01:34 uevent

./power:
-rw-r--r--    1 0        0            4096 Nov 13 01:34 wakeup

RAM_CONSOLE

This allows saving the kernel printk messages to a buffer in RAM, so that after a kernel panic they can be viewed in the next kernel invocation, by accessing /proc/last_kmsg.

[Would be good to get more details on how to set this up and use it here!] [I guess this is something like pramfs?]

other kernel changes

Here is a miscellaneous list of other kernel changes in the mistral Android kernel:

  • switch events - drivers/switch/* userspace support for monitoring GPIO via sysfs/uevent used by vold to detect USB
  • USB gadget driver for ADB - drivers/usb/gadget/android.c
  • yaffs2 flash filesystem
  • support in FAT filesystem for FVAT_IOCTL_GET_VOLUME_ID
  • and more...

Kernel configuration options

The file Documentation/android.txt has a list of required configuration options for a kernel to support an Android system.