[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [LTP] New README
> This brings us to the topic of a runtime-driver; something responsible
> for retrieving command lines and running them. It might also be
> responsible configuration analysis and result reporting. Configuration
> analysis refers to the runtime-driver being able to report whether a
> test program or even a specific test case within a test program failed.
You meant "Result Reporting" in the last sentence above.
You also hit one of my pet peeves. A test succeeds when it finds a
failure. A test fails when it gives wrong results, whether reporting
a PASS or a FAIL (test case failure). Results Reporting (or Results
Analysis) will summarize which, if any, of the test cases or test
programs reported a failure of the software under test.
As long as I'm on pet peeves, here's another instance, taken from the
FAQ "How do I analyze the results":
dup02 1 PASS : dup(-1) Failed, errno=9 : Bad file descriptor
dup02 2 PASS : dup(1500) Failed, errno=9 : Bad file descriptor
The message that the dup call "Failed" can be misleading. The dup call
didn't fail -- it correctly reported an error for bad input. The test
would communicate much better if the word "Failed" were simply deleted
from the output. I use the term "negative test case" if I want to
indicate that a test item is "failing in a correct and expected manner".
Janet Clegg firstname.lastname@example.org
Silicon Graphics, Inc 651) 683-5207