Automated Testing Stack

From eLinux.org
Jump to: navigation, search

The Automated Testing Stack consist of layers of software which can intercommunicate. The interface between layers can be specified, so that different implementations of the layers can interoperate with each other. This is similar to other "stacks" in the software world.

Some terminology is borrowed from the OSI model.

See Test Stack Layers

Brainstorming

Stack elements:

  • Board control
  • Device under test
  • connections
  • report generator
  • job dispatcher
  • device database (has information about devices)
  • Test
  • Result
  • change detector

objects:

  • test
  • test results
  • results database
  • board (node)
  • host
  • job
  • run (build)
  • request
  • connection/channel
  • facility
  • user
  • interface (results presentation interface)
  • scheduler

QA operations

  • detect soemthing has changed
  • build software
  • install it
  • test it
    • determine if a result matters
      • (previously) establish a baseline behavior that is acceptable
      • match current behavior with baseline behavior
  • report problems

layers

reference

SLAV

The SLAV testing stack defines 4 layers, known as:

  • Perun - detection, scheduler, presenter?
    • crawls URLs
    • reports changes
    • submit Weles jobs
    • collect artifacts
    • interpret results
  • Weles - job/board scheduler
    • parses YAML
      • YAML job definitions - actions executed on DUT
        • deploy, boot, test, collect
    • collects assets
    • requests DUT
    • performs tests
  • Boruta - farm management system
    • schedules requests
      • priority
      • device groups
      • delayed access
    • provides convenient access to selected Dryad
  • Dryad - has an interface for board control (network)
    • manages a single DUT
    • knows DUT capabilities
    • requires 2 interfaces (power, network)