Difference between revisions of "Testing"

From eLinux.org
Jump to: navigation, search
(add link to rfc2119)
(add section on test standards)
Line 33: Line 33:
 
Start collecting info for testing standards here:
 
Start collecting info for testing standards here:
 
  * https://tools.ietf.org/html/rfc2119 - ietf MUST, SHALL, MAY, etc. wording standards
 
  * https://tools.ietf.org/html/rfc2119 - ietf MUST, SHALL, MAY, etc. wording standards
 +
 +
Things we'd like to standardize in open source testing:
 +
* test nomenclature (test glossary)
 +
* how to specify test pre-requisites
 +
  * ex: assert_define ENV_VAR_NAME
 +
  * ex: kernel_config
 +
* test API
 +
  * host/target abstraction
 +
    * kernel installation
 +
    * file operations
 +
    * console access
 +
    * command execution
 +
  * test retrieval, build, deployment
 +
    * test execution:
 +
      * ex: 'make test'
 +
* test log output format
 +
  * counts
 +
  * subtest results
 +
* test package format
 +
  * meta-data for each test
 +
* test results
 +
  * baseline expected results for particular tests on particular platforms
 +
  * what tests can be skipped etc.

Revision as of 10:37, 8 November 2016

This page has lists of resources for Linux testing.

Presentations from events

LPC 2016 testing mini-summit (Nov 2016)

The 2016 Linux Plumbers Conference had a mini-summit on Testing and Fuzzing, as well as other sessions dedicated to open source testing issues. Here are some links related to that event:

collect presentation links above

Japan Jamboree 58-1 (Oct 2016)

Previous ELC and ELCE talks on testing

Testing Standards

Start collecting info for testing standards here:

* https://tools.ietf.org/html/rfc2119 - ietf MUST, SHALL, MAY, etc. wording standards

Things we'd like to standardize in open source testing:

* test nomenclature (test glossary)
* how to specify test pre-requisites
  * ex: assert_define ENV_VAR_NAME
  * ex: kernel_config
* test API
  * host/target abstraction
    * kernel installation
    * file operations
    * console access
    * command execution
  * test retrieval, build, deployment
    * test execution:
      * ex: 'make test'
* test log output format
  * counts
  * subtest results
* test package format
  * meta-data for each test
* test results
  * baseline expected results for particular tests on particular platforms
  * what tests can be skipped etc.