Difference between revisions of "Kselftest Notes"

From eLinux.org
Jump to: navigation, search
(problems I had running kselftest)
 
Line 27: Line 27:
 
* ability to build tests separately from running them
 
* ability to build tests separately from running them
 
* ability to compile an individual test
 
* ability to compile an individual test
 +
** support for 'make -C tools/testing/selftest/<testdir>'
 +
** support for 'make TARGETS=<testdir>'
 
* ability to specify a list of tests to build/package/run
 
* ability to specify a list of tests to build/package/run
  
Line 53: Line 55:
 
= stats (as of Sep. 2019) =
 
= stats (as of Sep. 2019) =
 
* number of TARGET areas: xx
 
* number of TARGET areas: xx
 
  
 
= API usage =
 
= API usage =
Line 101: Line 102:
  
 
== notes ==
 
== notes ==
 +
=== BPF build issues ===
 +
the BPF tests require that Clang be installed, and are very sensitive to the version of
 +
clang that is installed.  See

Latest revision as of 12:32, 12 October 2019

Here are some notes about kselftest.

These were made as part of an attempt to improve the documentation for kselftest.

documentation

The primary source for kselftest documentation is in the kernel git repository, at: Documentation/dev-tools/kselftest.rst

Observations:

  • the doc is not super-well-organized
    • early on, there's an overemphasis on the special requirements for hotplug testing
      • IMHO, this should be moved to a test-area-specific section of the document
  • the doc is a little disorganized
    • the multiple use cases should be documented
      • use case 1: developer testing something in their own area, during development, on a self-hosted system
      • use case 2: developer testing something in their own area, during development, in a host/target configuration
      • use case 3: user testing the kernel for regressions, on a self-hosted system
      • use case 4: user testing the kernel for regressions, on a host/target configuration
      • use case 5: CI system testing for regressions, on a self-hosted system
      • use case 6: CI system testing for regressions, in a host/target configuration
  • in the main document, there's no mention of TAP output being preferred

code

  • there's a variety of systems and code here

features

  • ability to run tests directly from the kernel source repository
  • ability to build tests separately from running them
  • ability to compile an individual test
    • support for 'make -C tools/testing/selftest/<testdir>'
    • support for 'make TARGETS=<testdir>'
  • ability to specify a list of tests to build/package/run

variables

  • KBUILD_OUTPUT=<dir>
  • TARGETS=<space-separated list>
  • quicktest=1
  • O=<dir>

targets

at top-level Makefile

kselftest kselftest-merge kselftest-clean

in kselftest top Makefile

run_tests hotplug run_hotplug

execution

  • run_kselftest.sh
  • runner.sh
  • kselftest_install.sh

stats (as of Sep. 2019)

  • number of TARGET areas: xx

API usage

  • kselftest.h is included in 68 source files
  • kselftest_harness.h is included in 7 source files

problems I had running kselftest

problem with relative KBUILD_OUTPUT directory

The TARGET directory was not built when using a relative KBUILD_OUTPUT directory.

I did the following:

 $ export KBUILD_OUTPUT=../kbuild
 $ mkdir $KBUILD_OUTPUT
 $ make localmodconfig bzImage
 $ make TARGETS=size kselftest
 make[1]: Entering directory '/home/tbird/work/torvalds/kbuild'
 make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \
 	ARCH=x86 -C ../../.. headers_install
   INSTALL ../kbuild/kselftest/usr/include
 gcc -static -ffreestanding -nostartfiles -s    get_size.c  -o ../kbuild/kselftest/size/get_size
   /usr/bin/ld: cannot open output file ../kbuild/kselftest/size/get_size: No such file or directory
 collect2: error: ld returned 1 exit status
   ../lib.mk:138: recipe for target '../kbuild/kselftest/size/get_size' failed
 make[3]: *** [../kbuild/kselftest/size/get_size] Error 1
 Makefile:136: recipe for target 'all' failed
 make[2]: *** [all] Error 2
 /home/tbird/work/torvalds/linux/Makefile:1240: recipe for target 'kselftest' failed
 make[1]: *** [kselftest] Error 2
 make[1]: Leaving directory '/home/tbird/work/torvalds/kbuild'
 Makefile:179: recipe for target 'sub-make' failed
 make: *** [sub-make] Error 2

I checked, and there is no directory ../kbuild/kselftest/size

It appears to an handle absolute path for KBUILD_OUTPUT just fine. This worked:

 $ export KBUILD_OUTPUT=/home/tbird/work/torvalds/kbuild
 $ mkdir $KBUILD_OUTPUT
 $ make localmodconfig
 $ make TARGETS=size kselftest

The BUILD_TARGET directory is created by the tools/testing/selftests/Makefile, in the 'all' target.

Since this is executed in the tools/testing/selftests directory, relative paths are processed from this directory. In the failing example above, I found the directory: tools/testing/kbuild/kselftest/size (which is wrong)

notes

BPF build issues

the BPF tests require that Clang be installed, and are very sensitive to the version of clang that is installed. See