Difference between revisions of "JTA To Do List"
(add bug about empty SSH_PORT) |
(→Bugs?: add bug about syntax error in cron format for periodic entry) |
||
Line 76: | Line 76: | ||
user name and ip address) | user name and ip address) | ||
** Should not use -p in sshpass command line if SSH_PORT is empty | ** Should not use -p in sshpass command line if SSH_PORT is empty | ||
+ | * I tried to enter a periodic line of "/5 * * * *" (without the quotes), and the system gave me an error: | ||
+ | ** line 1:1: unexpected token: / | ||
=== Artemi's items === | === Artemi's items === |
Revision as of 16:17, 26 January 2016
This is a list of items to be done to improve JTA.
Note: Maybe make this a table if it gets big enough and we need more structure - but use a list for now
Tim's ideas
- improve the guide
- The guide is pretty sad - it needs a lot more detail
- what does it need? see JTA guide notes
- the guide needs some kind of "hello test" instructions
- the guide needs detailed instructions for how to integrate yocto and buildroot sdks
- what does it need? see JTA guide notes
- The guide is pretty sad - it needs a lot more detail
- get a new name - JTA is OK, but how about:
- JELTS - Jenkins-based Embedded Linux Test System
- FELT - Framework for Embedded Linux Testing ("It's smooth")
- Fuego - no meaning, just a cool name (reference to Tierra Del Fuego, a penguin habitat) (spanish for "fire")
- This is my favorite.
- It looks like there's already a "fuego" package for linux (C++ libraries for 'Go' programs??)
- testbook - combination of "test" and "facebook" - already used in usb-testbook - see http://testbook.net/usb-trial.html
- testbox - combination of "test" and "box" (which is popular for embedded linux tools - busybox, toybox, etc.)
- add support for serial interface to target boards
- can execute easily enough, but need get and put files over serial console - look at ymodem?
- add support for adb interface to target boards
- this should be trivial for the .board file - but possibly less so for the distro file?)
- convert documetation to asciidoc
- pre-install test binaries, for multiple architectures
- this should lower the barrier to entry
- there's no need to install toolchains (toolchains are needed for new tests only)
- there's no need to rebuild the tests constantly
- test using ttc
- add target_test as a test
- write hello test tutorial for qemuarm
- write hello test tutorial for qemux86 (this is even easier??)
- submit talk proposal for ELC
- submit BOF proposal for ELC
- get rid of extraneous jenkin-isms in the interface
- eg. "Build a Maven 2/3 project" on the New Job page. - what does this mean?, do I care? It's needless junk in our interface.
- add test dependencies, and run a sanity-checker before tests (to avoid false negatives)
- check for kernel config, target configuration features (ability to change kernel, ability to reboot), etc.
- respond on mailing list within 24 hours.
- response times from cogent are very bad (over a week, and sometimes not at all)
- move the project to github? (I don't like the bitbucket hosting)
Bugs?
- when creating a new board, the environment variable "DISTR" was set to distribs/nologger.dist, which caused a missing "DISTRIB" message
- it appears that this environment variable name should be DISTRIB
- in the release I'm testing in January 2016 (jta-core last commit Nov 30 2015, and jta-public last commit Nov 30 2015)
- I was making userdata/conf/boards/bbb.board (copied from template-dev.board) and neither has either DISTR or DISTRIB.
- adding DISTRIB="distribs/nologger.dist" to bbb.board still resulted in the error
- This doesn't go in the board file, it goes in the jenkins target configuration (/userdata/conf/config.xml)
- need to send a patch for template-dev in this file (does it come from jta-core or jta-public?)
- in the file /home/jenkins/tests/Benchmark.bc/bc-scripts/bc-device.sh, it sets BC_EXPR1=$1 and also!! BC_EXPR2=$1
- Shouldn't that be BC_EXPR2=$2 ??
- in the file /home/jenkins/tests/Benchmark.bc/bc-script.sh, the function test_run has the line "report "cd $JTA_HOME/jta.$TESTDIR; ./bc-device.sh $BENCHMARK_BC_EXPR1 $BENCHMARK_BC_EXPR1"
- Shouldn't that second parameter be $BENCHMARK_BC_EXPR2 (!!)
- in my log I was seeing both arguments written at "2+2", although I was using the default bc spec.
- the sample specs for the BC example seem to have poor names
- instead of the specs being named bc_exp1 and bc_exp2, they should be bc_mult and bc_add, to describe what they do.
- this is expecially true since exp1 and exp2 are used inside the script themselves to indicate the first and second expressions to evaluate
- if you click on "graph" to see the results of the test, then go "back" in the browser, it re-runs the test again.
- When working with beaglebone, I had a test failure with the error
{{{ + sshpass -e ssh -o ServerAliveInterval=30 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=15 -p root@10.0.1.33 'mkdir -p /tmp/jta.Benchmark.bc && cd /tmp/jta.Benchmark.bc && cat /var/log/messages > bbb.2016-01-26_18-07-01.6.after' Bad port 'root@10.0.1.33' + abort_job 'Error while ROOTFS_LOGREAD command execution on target' }}}
- The error comes from a mis-configured SSH_PORT (if empty, the -p parameter uses the value from the next field, which is the
user name and ip address)
- Should not use -p in sshpass command line if SSH_PORT is empty
- I tried to enter a periodic line of "/5 * * * *" (without the quotes), and the system gave me an error:
- line 1:1: unexpected token: /
Artemi's items
Here is a list that Artemi sent me in October, 2015
- Tim's hardware lab... integration/support of Sony debug board
- Consider synchronization/update to align with latest versions of Jenkins
- Update LTP test suite
- Consider update/synchronize with latest public versions of other test suites (e.g. iperf/netperf and other benchmarks)
- Split into separate repositories and make tests separate from the framework
- Consider Docker-based installation (to avoid - "failed to install" setup issues)
- Qemu target support
- Consider adding more elegant/straightforward way of running separate test on target, command-line interface