Fuego

Fuego is the official automated test framework for the LTSI project.

It was formerly called "JTA" (for Jenkins-based Test Automation), but the name was changed in March 2016 for branding purposes.

Please see the official page for this LTSI test project at:
 * http://ltsi.linuxfoundation.org/ltsi-test-project

Code for the test framework is available in 2 git repositories:
 * https://bitbucket.org/cogentembedded/fuego/
 * Downloadable with 'git clone https://bitbucket.org/cogentembedded/fuego.git'
 * https://bitbucket.org/cogentembedded/fuego-core/
 * Downloadable with 'git clone https://bitbucket.org/cogentembedded/fuego-core.git'

The quick introduction to Fuego is that it consists of a host/target script engine, with a Jenkins front-end, and over 50 pre-packaged tests, deployed in a docker container.

Fujitsu used the completed test framework, and reported their findings here:
 * http://jp.fujitsu.com/group/fct/downloads/v4/events/exhibit/practice-ltsi-test-framework-introduction-of-ethtool-test-set.pdf

Call to action
Please download and use this framework, and send feedback on it to the LTSI mailing list at:
 * https://lists.linuxfoundation.org/mailman/listinfo/ltsi-dev

Vision
The purpose of Fuego is to provide a test framework for testing embedded Linux, that is distributed and allows individuals and organizations to easily run their own tests, and at the same time allows people to share their tests with each other.

Historically, test frameworks for embedded Linux have been difficult to set up, and difficult to extend. In cases where a test program was reasonably self-contained, or test system was not very difficult to extend, the system was not easily applied in cross or embedded environments. Some very full frameworks are either not viewed as processor-neutral (LAVA), and are difficult to set up (LAVA), or are targeted at running tests on a dedicated group of boards or devices (http://kernelci.org/).

The vision of open source in general is one of sharing source code and capabilities, to expand the benefits to all participants in the ecosystem. The best way to achieve this is to have mechanisms to easily use the system, and easily share enhancements to the system, so that all participants can use and build on each others efforts.

The goal of Fuego is to provide a framework that any group can install and use themselves, while supporting important features like cross-compilation, host/target test execution, easy test administration. Test administration consists of starting tests (both manually and automatically), viewing test results, and detecting regressions. Ease of use is critical, to allow testers to use tests that are otherwise difficult to individually set up, configure, and interpret the results from. It is also important to make it very easy to share tests (scripts, configuration, results parsing, and regression detection methods).

Some secondary goals of this project are the ability for 3rd parties to initiate or schedule tests on our hardware, and the ability to share our test results with others.

The use of Jenkins as the core of the test framework already supports many of the primary and secondary goals. The purpose of this project is to augment the Jenkins system to support embedded configurations of Linux, and to provide a place for centralized sharing of test configurations and collateral.

Actions Items/Proposed extensions/To-Do list
Here is a list of things that need to be fixed or improved about the current system:

Fuego To Do List

Other Resources
Another wiki with Fuego stuff is at http://bird.org/fuego/FrontPage