Automated Testing Summit 2019

This is a public planning page for the Automated Testing Summit

Please see: https://events.linuxfoundation.org/events/ats-2019/

= 2019 meeting = Coordinated by: Tim Bird This is now a full-blow Linux Foundation event, with sponsors, a CFP, program and everything.

See the Linux Foundation Event page for details.

Mailing list
Inter-project discussions on Automated Testing occur on the "Automated Testing" mailing list of the Yocto Project
 * See https://lists.yoctoproject.org/listinfo/automated-testing

= Date and Venue =
 * Location: Lyon, France
 * Venue: Lyon Convention Center, 50 Quai Charles de Gaulle
 * Date: October 31, 2019
 * Rooms: Pasteur Lounge and Rhone 3AB
 * Time: 9:00 am to 17:15 (5:15 pm)


 * co-located (following) Embedded Linux Conference Europe and Open Source Summit Europe
 * ELCE and OSSEU run Oct 28-30
 * see https://events.linuxfoundation.org/events/embedded-linux-conference-europe-2019/

Attendance is by open to the public, and we are projecting the attendee fee to be $250, with this fee waived for speakers.

= Summit Pre-planning = Here is a list of action items to do to prepare for the summit:
 * organize program committee (done)
 * get sponsors (done)
 * create sponsorship prospectus (done)
 * contact candidate sponsors (done)
 * hold "Call for Proposals" (done)
 * need to determine CFP dates (done)
 * determine agenda (done)
 * organize program committee (done)
 * advertise for attendees (done)
 * reserve venue, catering, AV rentals (done)
 * create signage, web site, registration system (done)
 * hold event (done)

For private planning information, see the private planning wiki page

Call for Presentations

 * CFP Open: Tuesday, May 28
 * CFP Close: Friday, August 13
 * Notifications Out: September 2
 * Schedule Out: September 4
 * Slides Due: October 24, 2019
 * Event Date: October 31, 2019

Suggested Topics
 * Board Farms and Test Labs - hardware, software, useful practices
 * Layers and Interfaces - components of your automated CI loop and protocols between them
 * Results Reporting - results aggregation, visualization, notifications, etc.
 * Defining standards - propose or discuss conventions for test definitions, results formats, layer interoperability (e.g. protocols), etc.
 * Existing Projects - describe your test system or framework and any challenges you overcame or that are still outstanding
 * Scalability - supporting big data, or handling distributed builds or tests
 * Ease of use - making testing and test development accessible to non-QA folk (developers, end-users)

= Pre-meeting work = Here's what Tim thinks we need to do:
 * Refine the test stack reference diagram
 * Organize committees for specific APIs
 * designate a head for each committee

= Sponsors = Sponsorship was provided by:
 * Sony - Platinum sponsor
 * Core Embedded Linux Project (of the Linux Foundation) - Gold sponsor
 * Linaro - Bronze sponsor
 * Civil Infrastructure Platform - Bronze sponsor

= Schedule = See https://ats19.sched.com/

= Summit Artifacts = documents and presentations
 * for presentations - see below

Key decisions

 * all systems to make a kcidb client (upload results)
 * extend the kcidb schema
 * https://github.com/kernelci/kcidb
 * priority (over test definition unification work)
 * use LTP metadata format as initial standard
 * add meta-data convert to kselftest (Tim)
 * build a prototype to move boards between LAVA and labgrid (Jan)
 * want to collect more information for "Hardware Design for Testing"
 * Contributions are Welcome: https://github.com/SmithChart/Designing-for-Automated-Testing/
 * get something working for board management layer for poweron/poweroff/serial-console
 * continue work on test definition common schema and central repository (Tim)
 * add information about hardware you find useful, or have designed please add to Board_Farm
 * focus on meeting at Plumbers next year
 * continue using automated-testing@lists.yoctoproject.org

Action items from meeting
Action items:
 * post slides to this page
 * send minutes