Difference between revisions of "Testing"

From eLinux.org
Jump to: navigation, search
(add section on test standards)
(Additional Resources)
(12 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
This page has lists of resources for Linux testing.
 
This page has lists of resources for Linux testing.
 +
 +
= Automated Testing page =
 +
The [[Automated Testing]] page has information about issues and initiatives related to automated testing of embedded Linux.
 +
 +
== Board Farm Page ==
 +
See the [[Board Farm]] page
 +
 +
= Testing Standards =
 +
See [[Test Standards]]
  
 
= Presentations from events =
 
= Presentations from events =
Line 12: Line 21:
 
** Applying mutation testing for testing RCU: Progress report - by Iftekhar Ahmed
 
** Applying mutation testing for testing RCU: Progress report - by Iftekhar Ahmed
 
** Integrating and running all the kernel tests - by Yannick Brosseau
 
** Integrating and running all the kernel tests - by Yannick Brosseau
** Linux Kernel Testing: Where are we ? - by Guenter Roeck
+
** [[Media:Linux-Kernel-Testing-Where-are-we.pdf|Linux Kernel Testing: Where are we? (PDF)]] - by Guenter Roeck
 
** Syzkaller: Future development - by Dmitry Vyukov
 
** Syzkaller: Future development - by Dmitry Vyukov
 
** Trinity: Next directions - by Dave Jones
 
** Trinity: Next directions - by Dave Jones
Line 26: Line 35:
 
** [[Media:ELCE-2016-Continuous-Integration-and-Autotest-Environment-using-Fuego_Master.pdf|Continuous Integration and Autotest Environment using Fuego (PDF)]](from ELCE 2016) - by Kengo Ibe, Kotaro Hashimoto, Mitsubishi Electric
 
** [[Media:ELCE-2016-Continuous-Integration-and-Autotest-Environment-using-Fuego_Master.pdf|Continuous Integration and Autotest Environment using Fuego (PDF)]](from ELCE 2016) - by Kengo Ibe, Kotaro Hashimoto, Mitsubishi Electric
 
** [[Media:The_Test_Activities_In_Renesas_Using_Fuego_Current.pdf|Current State of the Test Activities in Renesas Using Fuego (PDF)]] - by Masao Nomura, Renesas Electronics Corporation
 
** [[Media:The_Test_Activities_In_Renesas_Using_Fuego_Current.pdf|Current State of the Test Activities in Renesas Using Fuego (PDF)]] - by Masao Nomura, Renesas Electronics Corporation
 +
 +
== ELC Europe 2016 (Oct 2016) ==
 +
* test frameworks:
 +
** [[Media:Brown.pdf|KernelCI needs you! (PDF)]] - by Mark Brown, Linaro, and Kevin Hilman, BayLibre
 +
** [[Media:Continuous_Integration_and_Autotest_Environment_Using_Fuego.pdf|Continuous Integration and Autotest Environment Using Fuego (PDF)]] - by Kenji Tadano & Kengo Ibe, Mitsubishi Electric
 +
 +
* board farms:
 +
** [[Media:Herd_Your_Boards.pdf|Herd Your Baords, Become a Farmer]] by Geert Uytterhoeven, Glider bvba
 +
** [[Media:Automated_Testing_Laboratory_for_Embedded_Linux_Distributions.pdf|Automated Testing Laboratory for Embedded Linux Distributions (PDF)]] - by Pawel Wieczorek, Samsung R&D Institute Poland
 +
** [[Media:Building_a_Bards_Farm.pdf|Building a Board Farm: Continuous Integration and Remote Control (PDF)]] - by Antoine Tenart & Quentin Schulz, Free Electrons
  
 
== Previous ELC and ELCE talks on testing ==
 
== Previous ELC and ELCE talks on testing ==
 
* ELCE 2010 - [[Media:Embedded_software_testing.pdf|Embedded Software Testing (PDF)]] - by Arnout Vandecappelle
 
* ELCE 2010 - [[Media:Embedded_software_testing.pdf|Embedded Software Testing (PDF)]] - by Arnout Vandecappelle
  
= Testing Standards =
 
Start collecting info for testing standards here:
 
* https://tools.ietf.org/html/rfc2119 - ietf MUST, SHALL, MAY, etc. wording standards
 
  
Things we'd like to standardize in open source testing:
+
= Additional Resources =
* test nomenclature (test glossary)
+
== web sites ==
* how to specify test pre-requisites
+
* [http://www.opensourcetesting.org/ Open Source Testing] web site
  * ex: assert_define ENV_VAR_NAME
+
** ''It seems to have ceased operations in 2017''
  * ex: kernel_config
+
 
* test API
+
== Papers on testing ==
  * host/target abstraction
+
* Paper on finding bugs in Unix programs using random input: http://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz.pdf
    * kernel installation
 
    * file operations
 
    * console access
 
    * command execution
 
  * test retrieval, build, deployment
 
    * test execution:
 
      * ex: 'make test'
 
* test log output format
 
  * counts
 
  * subtest results
 
* test package format
 
  * meta-data for each test
 
* test results
 
  * baseline expected results for particular tests on particular platforms
 
  * what tests can be skipped etc.
 

Revision as of 15:58, 30 October 2018

This page has lists of resources for Linux testing.

Automated Testing page

The Automated Testing page has information about issues and initiatives related to automated testing of embedded Linux.

Board Farm Page

See the Board Farm page

Testing Standards

See Test Standards

Presentations from events

LPC 2016 testing mini-summit (Nov 2016)

The 2016 Linux Plumbers Conference had a mini-summit on Testing and Fuzzing, as well as other sessions dedicated to open source testing issues. Here are some links related to that event:

collect presentation links above

Japan Jamboree 58-1 (Oct 2016)

ELC Europe 2016 (Oct 2016)

Previous ELC and ELCE talks on testing


Additional Resources

web sites

Papers on testing