Re: [LTP] RE: summary of tests included in ltp?

[Nathan Straz <nstraz@sgi.com>]

I've worked out a set of descriptions for the tests currently available.
It's mostly a cut and paste job, but it gives a description to each
command line that was in the runtest/quickhit file.  I would like to
know how much information people want in these descriptions.  Skim
through and summarize which descriptions you like and don't like.

You can take a look at the file here:
http://oss.sgi.com/cgi-bin/cvsweb.cgi/~checkout~/ltp/runtest/quickhit?rev=1.2&content-type=text/plain

I spoke with Rich and Aaron at length about this.  We have a few goals
about a description mechanism in LTP.
  1. The file should be easily vi-able. We think requiring and interface
     to read or add test descriptions is a Bad Idea(tm).
  2. The file should be easily parsable.  We want to have the option of
     adding other interfaces to it, like a web page to quickly jump
     between related tests or complex searching methods.
  3. The format should be extendible.  At some point we will want to add
     such things as dependencies and "see also" and other junk.  Make
     this as painless as possible.  

Here is what I came up with:
------(snip)------
testtag01 sometestprogram -a param1 -b param2
# This is a description of the test program
# I don't care what you type here but it should be helpful
------(snip)------

 a. Each test case it uniquely defined by a "tag."  This is just some
    identifier.  Everything after the tag, to the end of the line, is
    the command line to run the test.
 b. Test descriptions are placed behind comments, which are just like
    shell comments, they start with a '#' character.  Descriptions can
    be as many lines as you like.
 c. I have been putting a space between the '#' and the text of the
    description for readability.  I'm thinking that special fields could
    be added without the space.  So you could add a cross reference like:
------(snip)------
dup03 dup03
# Negative test for dup(2) (too many fds)
#SeeAlso dup(2) dup01 dup02 quickhit
------(snip)------
 d. This file can still be read by the current "harness" and a few
    others that we are looking at.

So that's my quick attempt at an easy to use and maintain description
mechanism.  It doesn't need an interface, it's easy to read and edit,
and you could extend it easily.  I know this isn't going to scale to
thousands of tests well.  Once we make some progress on the framework
and configuration analysis fronts, this will definitely get revisited.

-- 
Nate Straz                                              nstraz@sgi.com
sgi, inc                                           http://www.sgi.com/
Linux Test Project                    http://oss.sgi.com/projects/ltp/