Automated Testing Summit 2018

From eLinux.org
Revision as of 11:31, 30 April 2018 by Tim Bird (talk | contribs) (add Augustin Bethencourt)
Jump to: navigation, search

This is a public planning page for the Automated Testing Summit

2018 meeting

Coordinated by: Tim Bird

Action items:

  • Determine attendees
    • create "wanted attendees" list
    • ask for interested parties
  • Determine venue
    • ask candidate attendees about which event, length, format works best for them
  • see who's interested in coming
  • send e-mail to find out which event, format, etc.

Venue (candidates)

  • Plumbers 2018 - Vancouver, Canada - Nov 13-15
  • ELC Europe 2018 - Edinburgh, Scotland - Oct 22-24

Note: For Plumbers, ATS would be a micro-conference during the event. For ELC Europe, ATS could be before, during or after the event.

Attendees

wanted attendees

  • Pawel Wieczorek (Samsung) - SLAV
  • Kevin Hillman (BayLibre) - KernelCI, lab-in-a-box
  • Tom Gall (Linaro) - LKFT
  • Neil Willliams (Linaro) - LAVA
  • Jan Lübbe (Pengutronix) - Labgrid
  • Tim Bird (Sony) - Fuego, ttc
  • Andrew Murray (Witekio) - started this initiative
  • Daniel Sangorrin (Toshiba) - Fuego/CIP
  • Khiem Nguyen (Renesas) - LTSI testing
  • Guenter Roeck - (Google) - ChromeOS testing, kerneltests.org
  • Fengguang Wu - (Intel) - 0 day test service
  • Agustin Benito Bethencourt (CodeThink) - board@desk testing for CIP (presentation)
  • more people from industry
    • (comcast? - ask Khem for contact (done))

possible invitees

  • Yannick Brosseau (Facebook) -
  • Dmitry Vyokov (Google) - KASAN, syskaller testing
  • Kenji Tadano (Mitsubishi)
  • Kengo Ibe (Mitsubishi)
  • Shuah Khan (Samsung) - kselftest
  • Cyril Hrubis (?) - LTP
  • someone from yocto? (who?)
  • someone from buildroot? (Thomas Petazzoni?) (is the buildroot community interested in this?)
  • someone from U-Boot? (Tom Rini/Stephen Warren/Simon Glass)?
  • Sjoerd Simmons (Collabora) - kernelci contributor
  • Guillaume Tucker (Collabora) - kernelci contributor
  • Matt Hart (Linaro) - upstream-focused testing, kernelci

interested in attending

Please add your preferred co-located conference (ELCE, Plumbers, or something else), if you have one.

  • Geert Uytterhoeven - general board farm (ELCE or FOSDEM)
  • Kieran Bingham - general board farm
  • Jan-Simon Möller - general board farm, especially LAVA & KernelCI
  • Marco Cavallini (Koan) - general board farm
  • Chris Fiege (Pengutronix) - board farm hardware, special development tools (USB-SD-Mux, ...)
  • Michal Simek (Xilinx) - general board farm
  • Punnaiah Choudary Kalluri (Xilinx) - general board farm
  • Neil Willliams (Linaro) - LAVA
  • Tim Bird (Sony) - Fuego, ttc (either ELCE or Plumbers is OK)
  • Carlos Hernandez (Texas Instruments) - manages board farm at TI
  • Otavio Salvador (O.S. Systems) - (prefers ELCE)
  • Jan Lübbe (Pengutronix) - labgrid (prefers just after ELCE)
  • Milosz Wasilewski (Linaro) - LKFT (ELCE or FOSDEM are fine) - can cover LKFT if Tom can't make it
  • Muhammad Shafique (Mentor) - general board farm, LAVA & KernelCI
  • Mirza Krak (Endian Technologies AB) - (ELCE or FOSDEM)
  • Tim Orling (Yocto Project) - (ELCE, possibly Plumbers)
  • Matt Hart (Linaro) - board farm, PDUDaemon, LAVA and KernelCI
  • Rouven Czerwinski (Pengutronix) - labgrid (prefers just after ELCE)
  • Mark Brown (Linaro) - Board farm stuff, kernelci, consuming results
  • Alan Bennett (Open Source Foundries) - (prefers ELCE)
  • Tyler Baker (Open Source Foundries)
  • Manuel Traut (Linuxtronix) - r4d, jenkins, ci-rt
  • Anna-Maria Gleixner (Linuxtronix) - r4d
  • Zhenfeng Zhao (Wind River) - (either ELCE or Plumbers is fine)
  • Shuah Khan (Samsung) - kselftest (Linaro connect?)
  • Steve McIntyre (Linaro) - LAVA (any)
  • Matteo Vit (Starware Design) - general board farm, hardware (ELCE)

Agenda (brainstorming)

Here is some brainstorming on an agenda...

  • board farm survey (lightning talks)
    • what are people using?
    • what works?
    • what's missing?
      • board discovery/lab introspection?
  • layers and interfaces
    • what layers are supported, needed?
    • any way to leverage/separate/isolate existing software?
  • what tests need to be supported?
    • boot-time
    • run-time
    • package-based (package unit tests)
    • driver (hardware specific?)
      • requiring specialized hardware external to board (e.g. canbus simulator, hdmi frame-grabber)
    • multinode
      • how to allocate/schedule multiple pieces of equipment for a test (e.g. 2 or more nodes for a network test)
  • results reporting
    • centralized server and API to it (kernelCI json?)
  • sharing tests
  • how to define standards
    • de-facto only? (dominant project? (cough, LAVA))
    • documents?
  • survey of existing projects, and what pieces they focus on (or don't)