Subset Libc Specification

From eLinux.org
Revision as of 20:47, 17 November 2008 by FransMeulenbroeks (Talk | contribs)

Jump to: navigation, search

Table Of Contents:


Description

CELF has been researching the possibilty of creating a "subset" Libc specification. This would be a specification for a reduced set of C-library APIs (or possibly ABIs) that would be appropriate for CE products.

Several small-footprint libraries are already available for Linux (see the Projects section below for a partial listing.)

Rationale

The C library is one of the largest pieces of software in a CE product, that CE products all have in common. Anecdotal evidence suggests that the current de-facto standard C library for Linux, glibc, is larger than needed for many CE application areas. It is hoped that by producing a specification for reducing API functionality required for CE products, it will allow easier and more reliable use of small-footprint C libraries in the CE space.

Resources

Projects

There are numerous small C library projects, including the following:

A special small C library is klibc. See the following resources:

There doesn't appear to be a real webpage for klibc, but the code is available from every Linux kernel mirror:

   ftp://ftp.<country>.kernel.org/pub/linux/libs/klibc/

(old) announce on LWN:

   http://lwn.net/Articles/7117/

Discussions related to klibc on Kernel Traffic:

   http://www.kerneltraffic.org/kernel-traffic/topics/Klibc.html


Specifications

There are a number of existing

  • POSIX has a set of "application profile environments", which are sub-sets of POSIX APIS for real-time operation systems.
    • POSIX standard which defines these is: IEEE Std 1003.13-1998 There are four different profiles, labeled PSE 51, 52, 53, and 54.
      • PSE 51, 52 do not require multiple process support
      • PSE 51, 53 do not require file systems
      • All require threads, other real-time and classic Unix features
      • PSE 52 API is a subset of PSE 54, PSE 51 is a subset of PSE 52
    • There's a presentation on the inadequacy of POSIX PSE 52 at: http://diicoe.disa.mil/coe/aog_twg/twg/rttwg/pse_52_discussion.ppt
  • Red Hat created a specification for reduced functionality, called EL/IX
  • The Embedded Linux Consortium produced a specification, with subsetted tiers:
  • The Linux Standard Base (LSB) is a specification for a FULL Linux C API. The subset libc should therefore be a strict subset of the LSB API.

Downloads

  • none so far

Utility programs

not applicable

How To Use

How to validate

CELF has had informal talks with the Free Standards Group (the current owners of the Linux Standard Base specification) about leveraging existing LSB test infrastructure to test subset libc implementations. Our main point of contact for this has been LSB lead developer Stuart Anderson.

Sample Results

[Examples of use with measurement of the effects.]

Case Study 1

Case Study 2

Status

  • Status: not started
 (one of: not started, researched, implemented, measured, documented, accepted)
  • Architecture Support:
  (for each arch, one of: unknown, patches apply, compiles, runs, works, accepted)
    • i386: unknown
    • ARM: unknown
    • PPC: unknown
    • MIPS: unknown
    • SH: unknown

Ideas on small-library compatibility with glibc, from an expert

Erik Andersen, the maintainer of uClibc, was asked about the possibility of ABI compatibility between uClibc and glibc, and the following was his response: (This was from a private message October 10, 2004 - Erik gave his permission to publish this.)

>> Eric,
>> 
>> SZWG(System Size Working Group) has just start the disucussion
>> of uClibc. The attached presentation material was provided by
>> Motorola how they approach to the size reduction. They tried
>> several things. P.6 shows their trial about uClibc. 
>> They saw huge size reduction by using uClibc but encountered 
>> some issue at the same time.
>> Basically, they had compatibility issue between glibc(originally
>> used) and uClibc. They listed the following three candidate as
>> possible solution.

By "compatibility issue" you mean that existing binaries
that had previously been compiled vs glibc will not run
with uClibc without being recompiled.

Or more succinctly:
uClibc is API compatible with glibc, but not ABI compatible.

>> Possible Solution:
>> 1) Rebuild each component.Request 3rdpart vendors to rebuild.
>> 2) Modify uClibcto be API compatible with glibc, 
>>    including adding a versioning system and structure 
>>    modification.
>> 3) Write a lightweight "translation" or "pass-through" version 
>>    of glibc that satisfies the requirements of each executable 
>>    are met, but that calls the uClibclibrary to perform the 
>>    necessary work.
>> 
>> First, they tried 1) approach but they gave it up, because of
>> difficulty to ask several 3rd parties to modify and rebuild
>> their software.

I strongly recommend #1.  Recompiling applications with uClibc
is almost always very easy to do for applications that already
compile with glibc.

If the vendor is not technically capable of doing the needed
work, I have a consulting company that would be happy to provide
assistance to 3rd vendors and to Motorola.  :-)

>> Finally, Motorola selected 3) and developed
>> "translation" or "pass-through" layer having uClibc underneath.

Yuck.  That sounds absolutely horrible.

>> As I understood, uClibc - from API point of view - is very 
>> close to glibc. Which part can be incompatible ?

uClibc and glibc have nearly identical APIs.  With a very few
exceptions, almost any program that will compile with glibc
will also compile with uClibc.

http://www.uclibc.org/cgi-bin/cvsweb/uClibc/docs/Glibc_vs_uClibc_Differences.txt?view=auto

>> Is there a way to make uClibc fully compatible with glibc ?

In my opinion, uClibc _is_ compatible with glibc.  But it
is compatible at the source code (API) level.  Most code
can be easily recompiled vs the latest uClibc.  What you
are really asking about is binary, or ABI compatibility.

The largest issues preventing uClibc from having an ABI
that is 100% binary compatible with glibc are the following
things.

1) Naming.  uClibc's shared library loader, C library, and
    even start up functions are named differently from their
    glibc counterparts.

2) uClibc sometimes uses different opaque data types than
    glibc.

3) uClibc directly uses the linux kernel's arch specific
    data structures, such as 'stuct stat', while glibc almost
    always translates kernel data structures into separate
    user space data structures.  This causes uClibc to be
    somewhat more tightly coupled with a particular kernel
    major version (2.2.x, 2.4.x, 2.6.x) than glibc.  When
    changing from 2.4.x to 2.6.x, it is advisable to recompile
    uClibc.

4) uClibc's stdio code is completely different from glibc's.
    This causes significant ABI differences for functions that
    are possible pthread cancellation points, for functions that
    are allowed to be macros by SuSv3.  Additionally, uClibc
    allows BUFSIZ to be set to values different from that used by
    glibc.  Some stdio functions, such as fcloseall() and
    __fpending() can behave differently than their glibc
    counterparts.  Other stdio functions, such as fscanf() behave
    differently in cases where glibc does not comply with SuSv3.

5) /etc/timezone and the whole zoneinfo directory tree are not
    supported by uClibc.  uClibc uses /etc/TZ, set per the value
    of the TZ env variable, per SuSv3.

6) Symbol versioning.  All glibc symbols have specific symbol
    versioning applied, so glibc does not have an 'fopen' symbol,
    but rather has a 'fopen@GLIBC_2.0' symbol.  In some cases,
    such as with 'sys_siglist', glibc has a number of
    implementations of the same symbol (sys_siglist@GLIBC_2.0,
    sys_siglist@GLIBC_2.1, and sys_siglist@@GLIBC_2.3.3) in order
    to maintain ABI compatibility with earlier versions of glibc.

doubtless there are other reasons why uClibc's ABI does not and
will not easily match the glibc ABI.

 -Erik


Future Work/Action Items

Here is a list of things that could be worked on for this feature:

-