Kselftest Notes

Here are some notes about kselftest.

These were made as part of an attempt to improve the documentation for kselftest.

= documentation = The primary source for kselftest documentation is in the kernel git repository, at: Documentation/dev-tools/kselftest.rst

Observations:
 * the doc is not super-well-organized
 * early on, there's an overemphasis on the special requirements for hotplug testing
 * IMHO, this should be moved to a test-area-specific section of the document
 * the doc is a little disorganized
 * the multiple use cases should be documented
 * use case 1: developer testing something in their own area, during development, on a self-hosted system
 * use case 2: developer testing something in their own area, during development, in a host/target configuration
 * use case 3: user testing the kernel for regressions, on a self-hosted system
 * use case 4: user testing the kernel for regressions, on a host/target configuration
 * use case 5: CI system testing for regressions, on a self-hosted system
 * use case 6: CI system testing for regressions, in a host/target configuration
 * in the main document, there's no mention of TAP output being preferred

= code =
 * there's a variety of systems and code here

features

 * ability to run tests directly from the kernel source repository
 * ability to build tests separately from running them
 * ability to compile an individual test
 * ability to specify a list of tests to build/package/run

variables

 * KBUILD_OUTPUT=
 * TARGETS=
 * quicktest=1
 * O=

at top-level Makefile
kselftest kselftest-merge kselftest-clean

in kselftest top Makefile
run_tests hotplug run_hotplug

= execution =
 * run_kselftest.sh
 * runner.sh
 * kselftest_install.sh

= stats (as of Sep. 2019) =
 * number of TARGET areas: xx

= API usage =
 * kselftest.h is included in 68 source files
 * kselftest_harness.h is included in 7 source files

= problems I had running kselftest =

problem with relative KBUILD_OUTPUT directory
The TARGET directory was not built when using a relative KBUILD_OUTPUT directory.

I did the following:

$ export KBUILD_OUTPUT=../kbuild $ mkdir $KBUILD_OUTPUT $ make localmodconfig bzImage $ make TARGETS=size kselftest make[1]: Entering directory '/home/tbird/work/torvalds/kbuild' make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ ARCH=x86 -C ../../.. headers_install INSTALL ../kbuild/kselftest/usr/include gcc -static -ffreestanding -nostartfiles -s   get_size.c  -o ../kbuild/kselftest/size/get_size /usr/bin/ld: cannot open output file ../kbuild/kselftest/size/get_size: No such file or directory collect2: error: ld returned 1 exit status ../lib.mk:138: recipe for target '../kbuild/kselftest/size/get_size' failed make[3]: *** [../kbuild/kselftest/size/get_size] Error 1 Makefile:136: recipe for target 'all' failed make[2]: *** [all] Error 2 /home/tbird/work/torvalds/linux/Makefile:1240: recipe for target 'kselftest' failed make[1]: *** [kselftest] Error 2 make[1]: Leaving directory '/home/tbird/work/torvalds/kbuild' Makefile:179: recipe for target 'sub-make' failed make: *** [sub-make] Error 2

I checked, and there is no directory ../kbuild/kselftest/size

It appears to an handle absolute path for KBUILD_OUTPUT just fine. This worked: $ export KBUILD_OUTPUT=/home/tbird/work/torvalds/kbuild $ mkdir $KBUILD_OUTPUT $ make localmodconfig $ make TARGETS=size kselftest

The BUILD_TARGET directory is created by the tools/testing/selftests/Makefile, in the 'all' target.

Since this is executed in the tools/testing/selftests directory, relative paths are processed from this directory. In the failing example above, I found the directory: tools/testing/kbuild/kselftest/size (which is wrong)