Automated Testing Summit 2018

From eLinux.org
Revision as of 21:25, 10 April 2018 by Koan (talk | contribs)
Jump to: navigation, search

This is a public planning page for the Automated Testing Summit

2018 meeting

Coordinated by: Tim Bird

Action items:

  • Determine attendees
    • create "wanted attendees" list
    • ask for interested parties
  • Determine venue
    • ask candidate attendees about which event, length, format works best for them
  • see who's interested in coming
  • send e-mail to find out which event, format, etc.

Venue (candidates)

  • Plumbers 2018 - Vancouver, Canada - Nov 13-15
  • ELC Europe 2018 - Edinburgh, Scotland - Oct 22-24

Note: For Plumbers, ATS would be a micro-conference during the event. For ELC Europe, ATS could be before, during or after the event.

Attendees

wanted attendees

  • Pawel Wieczorek (Samsung) - SLAV
  • Kevin Hillman (BayLibre) - KernelCI, lab-in-a-box
  • Tom Gall (Linaro) - LKFT
  • Niel Williams (Linaro) - LAVA
  • Jan Luebbe (Pengutronix) - Labgrid
  • Tim Bird (Sony) - Fuego, ttc
  • Andrew Murray (Witekio) - started this initiative
  • Daniel Sangorrin (Toshiba) - Fuego/CIP
  • Khiem Nguyen (Renesas) - LTSI testing
  • more people from industry
    • (comcast? - ask Khem for contact (done))

possible invitees

  • Yannick Brosseau (Facebook) -
  • Dmitry Vyokov (Google) - KASAN, syskaller testing
  • Kenji Tadano (Mitsubishi)
  • Kengo Ibe (Mitsubishi)
  • someone from yocto? (who?)
  • someone from buildroot? (Thomas Petazzoni?) (is the buildroot community interested in this?)

interested in attending

  • Geert Uytterhoeven - general board farm
  • Kieran Bingham - general board farm
  • Jan-Simon Möller - general board farm, especially LAVA & KernelCI
  • Marco Cavallini (Koan) - general board farm

Agenda (brainstorming)

Here is some brainstorming on an agenda...

  • board farm survey (lightning talks)
    • what are people using?
    • what works?
    • what's missing?
      • board discovery/lab introspection?
  • layers and interfaces
    • what layers are supported, needed?
    • any way to leverage/separate/isolate existing software?
  • what tests need to be supported?
    • boot-time
    • run-time
    • package-based (package unit tests)
    • driver (hardware specific?)
      • requiring specialized hardware external to board (e.g. canbus simulator, hdmi frame-grabber)
    • multinode
      • how to allocate/schedule multiple pieces of equipment for a test (e.g. 2 or more nodes for a network test)
  • results reporting
    • centralized server and API to it (kernelCI json?)
  • sharing tests
  • how to define standards
    • de-facto only? (dominant project? (cough, LAVA))
    • documents?
  • survey of existing projects, and what pieces they focus on (or don't)