LTP device discovery protocol

Here is some information from Cyril Hrubis on the rationale for this system:

From https://patchwork.ozlabs.org/project/ltp/patch/20200623112827.10744-2-chrubis@suse.cz/#2465790

= Device discovery =

The problem
Each lab has a different hardware capabilities and configuration. A test that heavily depends on a hardware needs to get this information in order to be able to run correctly.

The solution
The test declares which devices it needs for a testing. In the uart test these are UART_RX and UART_TX which are two UART endpoints which are connected together.

This information is then passed as a parameter to a device-discovery.sh script that prints, possibly several lines, of device lists, which is then parsed by the test library and the test is executed accordingly.

The data are passed to the test as a environment variables, if these are set prior to the test start, we do not even attempt to do a device discovery. If these are unset, we run the device discovery and loop the test around the lists we got.

Why this solution?
The device discovery is lab specific and does not belong to the test itself. This is an attempt to abstract the interface between the test and the hardware so that we can finally add device drivers tests into LTP.

Missing pieces
There are stil a few missing pieces, but these are probably easy to fix as well.

Device reconfiguration
I suppose that we may need to run a command so that the devices are reconfigured as we need them. I.e. the device-discovery.sh will have to output a command that needs to be executed in order to prepare the physical environment e.g. relays in case of the UART.

Device parameters
We may as well need some extra info about the devices, e.g. is hardware flow connected in case of UART. So the device-discover.sh will add one more environment variable e.g. UART_PARS="hwflow" that could be parsed in the test as well.

-

= test meta-data = A LTP test can express it's requirement for devices in it's tst_test structure, as a list of strings indicating the environment variable names for items that it needs in order to execute:

Here is an example for a uart (serial port) test: static struct tst_test test = { .setup = setup, .test_all = run, .options = (struct tst_option[]) { {"b:", &baud_rate, "-b      Baud rate (9600, ...)"}, {"w", &hwflow  , "-w       Enable hwflow (RTS/CTS)"}, {"f:", &fname,    "-f       Binary file for transfers"}, {"s:", &buf_size, "-s       Binary buffer size"}, {} },  .needs_devices = (const char *const[]) {"UART_RX", "UART_TX", NULL}, .forks_child = 1, };


 * source: https://patchwork.ozlabs.org/project/ltp/patch/20200623112827.10744-3-chrubis@suse.cz/

This shows that this test requires UART_RX and UART_TX to be defined. These environment variables are expected to be filled with strings indicating the Linux device which represents the serial ports on the receiving and transmitting side of the test, respectively. For example, you might have: UART_RX=/dev/ttyUSB0 and UART_TX=/dev/ttyUSB0

= device-discovery.sh = This is a shell script which is called by a program when the required environment variables are not defined.

= control flow = In Cyril's example:
 * test program is executed
 * the test program checks to see if required environment variables are defined
 * if not, then the test program executes 'device-discovery.sh', passing the names of the environment variables it needs in order to execute
 * device-discovery.sh returns a space-separated list of variable assignments
 * device-discovery.sh returns one configuration per line
 * the test program uses the devices as indicated by the environment variables
 * the test loops over the list of returned configurations to perform the test as many times as indicated

= dependency information transfer = A test's required devices (names of required environment variables that define devices) can be obtained by running the test with the -h option.

$ uart01 -h ... Needed devices -- UART_RX UART_TX

= NOTES = With the ability to query a test for its required devices, the test framework can determine (and set up) the environment for the test. But the variable names and all semantics associated with them have to be well-known between the test framework and the test.

device-discovery.sh is located in the LTP root directory.

Would this work with an /etc/test-config file, where the variable names were either statically defined, or where the variable referenced a program which could return the value for the name? I think so. But when does allocation/deallocation happen?

Here is the outline for when a test framework is involved:
 * call test program to find out required devices
 * check if the lab has the required devices
 * allocate the devices for the duration of the test
 * possibly connect the devices to the DUT using relays or multiplexors
 * set the environment variables for the test
 * execute the test
 * deallocate the devices for the test