Automated Testing Summit 2018

Revision as of 13:39, 10 September 2018 by Tim Bird (talk | contribs) (Pre-meeting work)
Jump to: navigation, search

This is a public planning page for the Automated Testing Summit

2018 meeting

Coordinated by: Tim Bird and Kevin Hilman

Action items:

  • Finalize sessions, presentations, and discussion topics
  • Coordinate registration, signage and badging with ELCE/OSSE

Date and Venue

  • Location: Edinburgh, Scotland
  • Venue: in the Edinburgh International Conference Centre
  • Date: October 25, 2018
  • Room: to be announced
  • Time: 9:00 am to 5:00 pm

Lunch will be included. Attendance is by invitation and is free-of-charge.

Sponsorship is provided by the Core Embedded Linux Project of the Linux Foundation.


wanted attendees

  • Pawel Wieczorek (Samsung) - SLAV
  • Kevin Hilman (BayLibre) - KernelCI, lab-in-a-box
  • Milosz Wasilewski (Linaro) - LKFT
  • Matt Hart (Linaro) - LAVA, upstream-focused testing, kernelci
  • Neil Willliams (Linaro) - LAVA
  • Jan Lübbe (Pengutronix) - Labgrid
  • Tim Bird (Sony) - Fuego, ttc
  • Andrew Murray (Witekio) - started this initiative
  • Daniel Sangorrin (Toshiba) - Fuego/CIP
  • Khiem Nguyen (Renesas) - LTSI testing
  • Guenter Roeck - (Google) - ChromeOS testing,
  • Fengguang Wu - (Intel) - 0 day test service
  • Agustin Benito Bethencourt (CodeThink) - board@desk testing for CIP (presentation)
  • Shuah Khan - (Samsung) - kselftest
  • Greg KH (Linux Foundation) - LTS maintainer
  • Michael Larabel (Phoronix, phoronic-test-suites, phoromatic,, - Phoronix Test Suites
  • Cyril Hrubis (SUSE) - LTP
  • more people from industry
    • (comcast? - ask Comcast folks if they are interested)

possible invitees

  • Yannick Brosseau (Facebook) -
  • Dmitry Vyokov (Google) - KASAN, syskaller testing
  • Kenji Tadano (Mitsubishi)
  • Kengo Ibe (Mitsubishi)
  • someone from yocto? (who?)
  • someone from buildroot? (Thomas Petazzoni?) (is the buildroot community interested in this?)
  • someone from U-Boot? (Tom Rini/Stephen Warren/Simon Glass)?
  • Sjoerd Simons (Collabora) - kernelci contributor
  • Guillaume Tucker (Collabora) - kernelci contributor

interested in attending

Please add your preferred co-located conference (ELCE, Plumbers, or something else), if you have one.

  • Geert Uytterhoeven - general board farm (ELCE or FOSDEM)
  • Kieran Bingham - general board farm
  • Jan-Simon Möller - general board farm, especially LAVA & KernelCI
  • Marco Cavallini (Koan) - general board farm
  • Chris Fiege (Pengutronix) - board farm hardware, special development tools (USB-SD-Mux, ...)
  • Michal Simek (Xilinx) - general board farm
  • Punnaiah Choudary Kalluri (Xilinx) - general board farm
  • Neil Willliams (Linaro) - LAVA
  • Tim Bird (Sony) - Fuego, ttc (either ELCE or Plumbers is OK)
  • Carlos Hernandez (Texas Instruments) - manages board farm at TI
  • Otavio Salvador (O.S. Systems) - (prefers ELCE)
  • Jan Lübbe (Pengutronix) - labgrid (prefers just after ELCE)
  • Milosz Wasilewski (Linaro) - LKFT (ELCE or FOSDEM are fine) - can cover LKFT if Tom can't make it
  • Muhammad Shafique (Mentor) - general board farm, LAVA & KernelCI
  • Mirza Krak (Endian Technologies AB) - (ELCE or FOSDEM)
  • Tim Orling (Yocto Project) - (ELCE, possibly Plumbers)
  • Matt Hart (Linaro) - board farm, PDUDaemon, LAVA and KernelCI
  • Rouven Czerwinski (Pengutronix) - labgrid (prefers just after ELCE)
  • Mark Brown (Linaro) - Board farm stuff, kernelci, consuming results
  • Alan Bennett (Open Source Foundries) - (prefers ELCE)
  • Tyler Baker (Open Source Foundries)
  • Manuel Traut (Linuxtronix) - r4d, jenkins, ci-rt
  • Anna-Maria Gleixner (Linuxtronix) - r4d
  • Zhenfeng Zhao (Wind River) - (either ELCE or Plumbers is fine)
  • Shuah Khan (Samsung) - kselftest (Linaro connect?)
  • Steve McIntyre (Linaro) - LAVA (any)
  • Matteo Vit (Starware Design) - general board farm, hardware (ELCE)
  • Dan Rue (Linaro) - Linaro LKFT
  • Ryan S Arnold (Linaro) - LAVA & LKFT (either ELCE, LPC, or Linaro Connect)
  • Roland Hieber (Pengutronix)
  • Eystein Stenberg ( (either ELCE or Plumbers)
  • Nobuhiro Iwamatsu (Cybertrust Japan / CIP / Debian / U-Boot) - (prefers ELCE)
  • Alice Ferrazzi (Cybertrust Japan / Gentoo Kernel Project Leader) (either ELCE or Plumbers) kselftest contributor / Gentoo kernel CI
  • Marc Kleine-Budde (Pengutronix) - (prefers just after ELCE)
  • Tsugikazu Shibata (LTSI) - (prefers ELCE)
  • Hirotaka Motai (Mitsubishi Electric) - (prefers ELCE)

Pre-meeting work

See Test Stack Layers

Tim is working on a glossary and survey. Kevin has a high-level CI-Loop diagram for discussion.

See Test Stack Survey for the work in progress.

Agenda (brainstorming)

Here is some brainstorming on an agenda...

  • board farm survey (lightning talks)
    • what are people using?
    • what works?
    • what's missing?
      • board discovery/lab introspection?
  • layers and interfaces
    • what layers are supported, needed?
    • any way to leverage/separate/isolate existing software?
  • what tests need to be supported?
    • boot-time
    • run-time
    • package-based (package unit tests)
    • driver (hardware specific?)
      • requiring specialized hardware external to board (e.g. canbus simulator, hdmi frame-grabber)
    • multinode
      • how to allocate/schedule multiple pieces of equipment for a test (e.g. 2 or more nodes for a network test)
  • results reporting
    • centralized server and API to it (kernelCI json?)
  • sharing tests
  • how to define standards
    • de-facto only? (dominant project? (cough, LAVA))
    • documents?
  • survey of existing projects, and what pieces they focus on (or don't)