Linux Tiny Notes

From eLinux.org
Revision as of 12:44, 14 July 2007 by Chris (Talk | contribs)

Jump to: navigation, search

Here are some miscellaneous notes on using Linux tiny. I wrote this up really quickly while I was integrating Linux-tiny into a 2.6.11 kernel and doing some size testing.

I'm providing these raw notes so that others might benefit from my experience.

Patch Notes

Here are some notes on individual linux-tiny sub-patches:

devlist.patch

  • replaces device allocation code, don't know size impact, is NOT conditional
    • maybe should add, but don't know - looks dangerous

core-small.patch - CONFIG_CORE_SMALL

Add CONFIG_CORE_SMALL Kconfig option

CONFIG_CORE_SMALL patches

These are hash table and data structure static size adjustments

  • pid-max.patch
  • user-hash.patch
  • futex-queues.patch
  • tvec_bases.patch
  • con_buf.patch

config-net-small.patch

Add CONFIG_NET_SMALL Kconfig option

CONFIG_NET_SMALL patches

These tune some net-related data sizes

  • cache_defer_hash.patch
  • unix_socket_table.patch
  • inet_protos.patch
  • flow-cache-small.patch

Miscellanoues patches

  • small-8023.patch - what does this do?
  • tg3-oops.patch - what does this do?

*inline patches

  • just make big routines not inline (unconditionally)
    • Seems pretty safe to me.

06-crypto-sleep.patch

  • looks scary - not immediately obvious from patch how it saves size.

# measurement bits

Looks like very harmless stuff to instrument kernel for mem debugging, and give tools for parsing stuff.

  • kmalloc-accounting.patch - keep track of kmallocs
  • audit-bootmem - print out all bootmem allocations
  • bloat-o-meter.patch - provides bloat-o-meter program to compare two kernel binary images
    • This script shows the size difference on a symbol-by-symbol basis between two binaries (sorted by the amount of difference)
  • deprecate-inline.patch - mark inlines as deprecated so compiler emits warnings
    • Note that it provides count-inlines program to parse compiler output and report size of inlines.
  • func-size.patch - use information about inlines to calculate function sizes (script is very similar to count-inlines program)

kill-printk.patch

  • is single biggest reducer of kernel size, doesn't seem to work with serial console!!
    • CONFIG_PRINTK

nobug.patch

Allow elimination of BUG code, which expands with file and line strings. There are 1776 BUG() references in the 2.6.11 kernel. There are 1930 BUG_ON(xxx) references in the kernel.

  • uses CONFIG_BUG

nopanic.patch

too dangerous to turn off panic, not used.

no-elf-core.patch

Allow disabling of ELF core dumps

  • CONFIG_ELF_CORE

no-kcore.patch

Allow disabling of /proc/kcore, is x86-specific

  • uses CONFIG_PROC_KCORE

no-aio.patch

Allow disabling of Posix async IO

  • uses CONFIG_AIO

no-xattr.patch

Allow disabling of xattr syscalls

  • uses CONFIG_XATTR

fslock.patch

Allow disabling of file locking syscalls

  • uses CONFIG_FILE_LOCKING

direct-io-core.patch

Allow disabling of direct IO support

  • uses CONFIG_DIRECTIO

Changes in number of in-kernel items

max-swapfiles.patch

Make number of swapfiles configurable (default=32) (shift=5 => 32 files, shift=0 => 1 file)

  • uses CONFIG_MAX_SWAPFILES_SHIFT

ldiscs.patch

Make number of tty line disciplines configurable (default=16)

  • uses CONFIG_NR_LDISCS

max_user_rt_prio.patch

Make number of schedule slots configurable (default=100)

  • uses CONFIG_MAX_USER_RT_PRIO

change-hz.patch

It's not obvious how this changes the size of the kernel, and it appears to be x86 only

Notes on building and size reductions from various configs

Notes on OSK builds:

  • first, did 'quilt pop pm/deferred-resume-test.patch', then built to get vmlinux.baseline
	   text    data     bss     dec  filename
	2107356  333344  113680 2554380  vmlinux.baseline
  • did 'quilt push -a', then built to get vmlinux.patched
	bloat-o-meter vmlinux.baseline vmlinux.patched = +2920/-8884
	   text    data     bss     dec  filename
	2101372  333888  113648 2548908  vmlinux.patched
  • configured all parameters at minimums, then built
    • couldn't finish linking, and got the following errors:
      • undefined reference to add_preferred_console
      • undefined reference to generic_setxattr and generic_getxattr
      • undefined reference to printk
  • configured printk and xattrs back on, then built to get vmlinux.try1
	bloat-o-meter vmlinux.patches vmlinux.try1 = +4520/-350490
	   text    data     bss     dec  filename
	1734504   90616   89036 1914156  vmlinux.try1

Panic trying to mount rootfs via nfs

  • tried to isolate nfs problem:
    • configured with AIO, file locks and direct IO on - same problem
    • configured with 5 swap files, 8 tty line disciplines, and 100 mat rt levels - same problem
    • went back to working config - diffed and noticed NFS_FS was off
      • NFS_FS requires FILE_LOCKING

Table of Config options

Here is a table showing default value and value recommended for size savings.

Note that I couldn't use CONFIG_PRINTK, CONFIG_XATTR or CONFIG_FILE_LOCKING in my setup.

CONFIG option description default small
CONFIG_CORE_SMALL tune some kernel data sizes N Y
CONFIG_NET_SMALL tune some net-related data sizes N Y
CONFIG_KMALLOC_ACCOUNTING turn on kmalloc accounting N Y - for measurement
CONFIG_AUDIT_BOOTMEM print out all bootmem allocations N Y - for measurement
CONFIG_DEPRECATE_INLINES cause compiler to emit info about inlines N Y - for measurement
CONFIG_PRINTK allow disable of printk code and message data Y N
CONFIG_BUG allow elimination of BUG (and BUG_ON??) code Y N
CONFIG_ELF_CORE allow disabling of ELF core dumps Y N
CONFIG_PROC_KCORE allow disabling of /proc/kcore Y N
CONFIG_AIO allow disabling of async IO syscalls Y N
CONFIG_XATTR allow disabling of xattr syscalls Y N
CONFIG_FILE_LOCKING allow disabling of file locking syscalls Y N
CONFIG_DIRECTIO allow disabling of direct IO support Y N
CONFIG_MAX_SWAPFILES_SHIFT number of swapfiles 5 0
CONFIG_NR_LDISCS number of tty line disciplines 16 2?
CONFIG_MAX_USER_RT_PRIO number of RT priority levels (schedule slots) 100 5?
Other config options These are not in Linux-tiny, but help with size default small
CONFIG_KALLSYMS load all symbols for debugging/kksymoops Y N
CONFIG_SHMEM allow use of shmem filesystem Y N


Some results

This was for a quick check of Linux-tiny on an OSK board.

section vmlinux.baseline vmlinux.try3 delta percent
text 2107356 2057276 -50080 -2%
data 333344 100988 -232356 -69%
bss 113680 92784 -20896 -18%
total 2554380 2251048 -303332 -11%


Program for producing report (Download)

Miscellaneous test notes

  • kernel would not link with CONFIG_PRINTK=n
    • undefined reference to add_preferred_console
    • undefined reference to printk
  • kernel would not link with CONFIG_XATTR=n
    • undefined reference to generic_setxattr and generic_getxattr
  • kernel would not mount root fs (NFS) with CONFIG_FILE_LOCKING=n

Config changes

[tbird@crest osk]$ diff config.baseline config.try3
--- config.baseline     2005-05-23 17:31:57.000000000 -0700
+++ config.try3 2005-05-23 17:31:40.000000000 -0700
@@ -1,7 +1,7 @@
 #
 # Automatically generated make config: don't edit
 # Linux kernel version: 2.6.11.10-alp
-# Mon May 23 16:49:43 2005
+# Mon May 23 17:27:14 2005
 #
 CONFIG_ARM=y
 CONFIG_MMU=y
@@ -34,34 +34,33 @@
 CONFIG_FASTBOOT=y
 # CONFIG_RTC_NO_SYNC is not set
 CONFIG_PRESET_LPJ=0
-# CONFIG_EMBEDDED is not set
-CONFIG_KALLSYMS=y
-CONFIG_KALLSYMS_ALL=y
-CONFIG_KALLSYMS_EXTRA_PASS=y
+CONFIG_EMBEDDED=y
+# CONFIG_KALLSYMS is not set
 # CONFIG_KMALLOC_ACCOUNTING is not set
 # CONFIG_AUDIT_BOOTMEM is not set
 CONFIG_PRINTK=y
-CONFIG_ELF_CORE=y
-# CONFIG_CORE_SMALL is not set
-# CONFIG_NET_SMALL is not set
-CONFIG_FUTEX=y
-CONFIG_EPOLL=y
-CONFIG_AIO=y
+# CONFIG_ELF_CORE is not set
+CONFIG_CORE_SMALL=y
+CONFIG_NET_SMALL=y
+# CONFIG_FUTEX is not set
+# CONFIG_EPOLL is not set
+# CONFIG_AIO is not set
 CONFIG_XATTR=y
 CONFIG_FILE_LOCKING=y
-CONFIG_DIRECTIO=y
+# CONFIG_DIRECTIO is not set
+# CONFIG_UID16 is not set
 # CONFIG_MEASURE_INLINES is not set
 CONFIG_CC_OPTIMIZE_FOR_SIZE=y
-CONFIG_SHMEM=y
+# CONFIG_SHMEM is not set
 CONFIG_CC_ALIGN_FUNCTIONS=0
 CONFIG_CC_ALIGN_LABELS=0
 CONFIG_CC_ALIGN_LOOPS=0
 CONFIG_CC_ALIGN_JUMPS=0
 # CONFIG_TINY_CFLAGS is not set
-CONFIG_MAX_SWAPFILES_SHIFT=5
-CONFIG_NR_LDISCS=16
-CONFIG_MAX_USER_RT_PRIO=100
-# CONFIG_TINY_SHMEM is not set
+CONFIG_MAX_SWAPFILES_SHIFT=0
+CONFIG_NR_LDISCS=2
+CONFIG_MAX_USER_RT_PRIO=5
+CONFIG_TINY_SHMEM=y

 #
 # Loadable module support
@@ -435,6 +434,7 @@
 CONFIG_SERIO=y
 CONFIG_SERIO_SERPORT=y
 # CONFIG_SERIO_CT82C710 is not set
+# CONFIG_SERIO_LIBPS2 is not set
 # CONFIG_SERIO_RAW is not set

 #
@@ -912,7 +912,7 @@
 # CONFIG_DEBUG_SPINLOCK is not set
 # CONFIG_DEBUG_KOBJECT is not set
 CONFIG_BUG=y
-CONFIG_DEBUG_BUGVERBOSE=y
+# CONFIG_DEBUG_BUGVERBOSE is not set
 CONFIG_DEBUG_INFO=y
 # CONFIG_DEBUG_FS is not set
 CONFIG_FRAME_POINTER=y

Series file

Please note that I didn't try all the patches. I only integrated and used 37 of the broken-out patches. You can see this from the series file I used, which follows:

####################################
# start of linux tiny patches
# key:
# ## = patches Tim commented out
# # = patches commented out in original 2.6.11-tiny broken out series file

##tiny/tiny-extraversion.patch
#
# stuff already in -mm
#
##tiny/kgdb-ga.patch
##tiny/kgdboe-netpoll.patch
tiny/use-funit-at-a-time.patch
##tiny/devlist.patch
#
# tiny stuff
#
# core
tiny/core-small.patch
tiny/pid-max.patch
tiny/user-hash.patch
tiny/futex-queues.patch
tiny/tvec_bases.patch
tiny/con_buf.patch
# net
tiny/config-net-small.patch
tiny/cache_defer_hash.patch
tiny/unix_socket_table.patch
tiny/inet_protos.patch
tiny/flow-cache-small.patch
#
# Unconditional bits
#
##tiny/small-8023.patch
##tiny/tg3-oops.patch
tiny/namei-inlines.patch
tiny/buffer-inlines.patch
tiny/fs_open-inlines.patch
tiny/inode-inlines.patch
tiny/ext2namei-inlines.patch
tiny/fslocks-inline.patch
##tiny/06-crypto-sleep.patch
#
# build options
#
tiny/tiny-cflags.patch
#
# measurement bits
#
tiny/kmalloc-accounting.patch
tiny/audit-bootmem.patch
tiny/bloat-o-meter.patch
tiny/deprecate-inline.patch
tiny/func-size.patch
#
# trimmed debugging support
#
##tiny/no-doublefault.patch
tiny/kill-printk.patch
##tiny/tiny-panic.patch
tiny/nobug.patch
##tiny/nopanic.patch
tiny/no-elf-core.patch
tiny/no-kcore.patch
#
# trimmed code
#
# TRB - 5/19 - haven't looked at these yet
##tiny/tiny-crc.patch
##tiny/threadinfo-ool.patch
##tiny/semaphore-inline.patch
##tiny/slob.patch
##tiny/slob-accounting.patch
##tiny/mempool-shrink.patch
#tiny/sysfs-backingstore.patch
##tiny/no-translations.patch
#tiny/tinyvt.patch
#
# trimmed apis
#
# TRB - 5/19 - haven't looked at ## ones yet
##tiny/ptrace.patch
##tiny/remove-vm86.patch
##tiny/sysenter.patch
tiny/no-aio.patch
tiny/no-xattr.patch
tiny/fslock.patch
tiny/direct-io-core.patch
tiny/uid16.patch
#tiny/posix-timers.patch
##tiny/ethtool.patch
##tiny/inetpeer.patch
##tiny/net-filter.patch
##tiny/dev_mcast.patch
#tiny/rtnetlink.patch
##tiny/igmp.patch
##tiny/binfmt-script.patch
##tiny/elf-no-aout.patch
#
# trimmed constants
#
tiny/max-swapfiles.patch
tiny/ldiscs.patch
tiny/max_user_rt_prio.patch
##tiny/change-hz.patch
#
# trimmed hw support
#
# TRB - 5/19 - haven't looked at ## ones yet
#tiny/small-nomodedb.patch
##tiny/ide-hwif.patch
##tiny/sbf.patch
##tiny/serial-pci.patch
##tiny/dmi_blacklist.patch
##tiny/pci-quirks.patch
##tiny/tsc.patch
##tiny/cpu-support.patch
##tiny/mtrr.patch
#
# inflate cleanups
#
# TRB - 5/19 - haven't looked at ## ones yet
##tiny/inflate-noinline.patch
##tiny/inflate-formatting.patch
##tiny/inflate-legacy.patch
##tiny/inflate-input.patch
##tiny/inflate-iostate.patch
##tiny/inflate-huft.patch
##tiny/inflate-crc.patch
##tiny/inflate-killglobals.patch
##tiny/inflate-initramfs.patch
##tiny/inflate-initrd.patch
##tiny/inflate-i386.patch
##tiny/inflate-arm.patch
##tiny/inflate-x86_64.patch
#
# unsorted
#
# TRB - 5/19 - haven't looked at ## ones yet
##tiny/inflate-roll-crc.patch
##tiny/netpoll-timeout.patch
##tiny/movsl-mask.patch
#tiny/rolled-md4.patch
#tiny/kill-ext3-md4.patch

# end of linux tiny patches
####################################