Difference between revisions of "JTA guide notes"

From eLinux.org
Jump to: navigation, search
Line 31: Line 31:
 
* The use of dark red boxes for inter-document links make the table of contents hard to read.  Can this be changed to something more subtle? (like a different color or just underline?)
 
* The use of dark red boxes for inter-document links make the table of contents hard to read.  Can this be changed to something more subtle? (like a different color or just underline?)
 
* Throughout the document, there are subtle misuses of English, and a number of typos. Sometimes, articles (like 'a', or 'the') are missing.  I will try to send patches to fix the ones I found.
 
* Throughout the document, there are subtle misuses of English, and a number of typos. Sometimes, articles (like 'a', or 'the') are missing.  I will try to send patches to fix the ones I found.
 +
 +
Some basic architecture and design information is missing from the document.
 +
 +
Here are some questions that should be answered:
 +
- what are the steps performed during a test?
 +
- for each step, is it performed on host or on target?
 +
  - there seem to be the steps pre_test, build, deploy, test_run, and test_processing (from test.sh)
 +
      - these seem to be defined in the <test>-script.sh file.
 +
- what does "assert_define" mean?
 +
  - how are the variable definitions transferred from the host to the target
 +
- what does "report" mean in the test_run function?
 +
- how are "cmd", "put" and "get" translated into the ov_transport_* functions?
  
 
=== intro ===
 
=== intro ===

Revision as of 14:21, 21 October 2015

Here is feedback on the JTA users guide.

The guide has an overall good structure, but it missing important details to explain the system and help people do basic tasks.

Tasks overview

  • install the system
    • connect to the system interface
  • architecture overview
  • goals overview
  • review of all screens and options
    • see list of targets
    • see list of tests
  • run a simple test (to test the system and see how it operates)
    • should support some qemu target (qemuarm?) for examples
    • see results from a test
    • see results from a series of tests
    • see test results visualization
  • add their own hardware target
  • add their own test
    • adding a new simple test
    • adding an existing test program
      • adding a program that requires compilation
        • adding a toolchain and cross-build environment
  • share their test configuration
  • share their new test

Issues/Feedback

Here are some issues with the guide:

  • I don't like the fonts. Using Acroread on Ubuntu 12.04, the font was very thin and I had a hard time reading it. It is possible to use a heavier-weight font? Or is this an Acroread problem? (Tim: it appears to be an Acroread problem, in Document Viewer, the fonts are a bit better - but it's still a bit hard to read.)
  • The use of dark red boxes for inter-document links make the table of contents hard to read. Can this be changed to something more subtle? (like a different color or just underline?)
  • Throughout the document, there are subtle misuses of English, and a number of typos. Sometimes, articles (like 'a', or 'the') are missing. I will try to send patches to fix the ones I found.

Some basic architecture and design information is missing from the document.

Here are some questions that should be answered:

- what are the steps performed during a test?
- for each step, is it performed on host or on target?
  - there seem to be the steps pre_test, build, deploy, test_run, and test_processing (from test.sh)
     - these seem to be defined in the <test>-script.sh file.
- what does "assert_define" mean?
  - how are the variable definitions transferred from the host to the target
- what does "report" mean in the test_run function?
- how are "cmd", "put" and "get" translated into the ov_transport_* functions?

intro

  • What does "it does not impose any demands on board or distributions" mean?

install (chapter 2)

  • should we document the default JTA_ENGINE_PATH?
    • can the end-user change this, or is this something that the Debian Jenkins package expects?
  • I see lots of references to OSV - what is this?

board_config (chapter 3)

  • I don't understand what a front-end Jenkins entity is? Is this a process on the host?
    • So, to add a new target, you define a .board file, and then create a Target in the Jenkins interface?
  • the board config overlay is confusing. I haven't been introduced to the notion of overlays yet.
    • this needs more introduction or explanation
  • template-dev should be called "template-board" or "board-template" or "board-skeleton", IMHO
  • I created a Jenkins target, and re-wrote the BOARD_OVERLAY environment variable to be boards/beaglebone-black.board - is this right?
    • Jenkins didn't create the .board file for me? Do I do this manually? If so, this should be described in the doc.
    • I manually copied template-dev.board to beaglebone-black.board in /home/jenkins/boards - is this right?

running_tests (chapter 4)

adding_test (chapter 5)

overlays (chapter 6) (Base classes and overlays)

testplans (chapter 7)

  • section 7.3 says that specs are read from the overlays/specs directory - but I think this should be overlays/test_specs

reports (chapter 8)

  • this chapter is empty

listings (chapter 9)