Re: [LTP] Re: TET [Mea Culpa]

["Sean A. Channel" <schannel@cobalt.com>]

What you describe is pretty much how I've been using TET as a general
automation harness for other tests like LTP.

I usually add a non-trivial 'runtest' script in each test (e.g. LTP)
directory to drive those tests.  Since TET has a shell-api (as well as
numerous others), the runtest script itself can be api-compliant.
Ideally, a runtest script would auto-sense the TETness of it's environment,
and select api-compliant functions where possible.

In this way, TET can run the whole show in automation, but I can still 
run any individual tests by hand with 'runtest'.
-s

On Thu, 14 Sep 2000, John George/Beaverton/IBM wrote:

]
]By setting TET_API_COMPLIANT to False the test cases could remain
]completely
]separate from the test harness.  By doing this there would be no dependency
]on
]the TET harness to execute any of the tests.  The only requirement would be
]that the tests return zero on success.  I have successfully ran the
]quickhit tests under
]TET3.3 with exception of the ones that take command line arguments.  As far
]as
]I can tell there is no way to pass command line arguments to a test case
]while
]running TET without its API framework.
]
]Some of the advantages I see to using TET:
]
]1.      The scenario file can be setup to allow tests to be grouped.  This
]would
]        accommodate those who want to test a specific feature (e.g. io,
]threads,
]        stress).
]
]2.      The scenarios can be setup to run tests repeatedly, in parallel, at
]        random, and/or for a specified amount of time.
]
]3.      The results of all the tests are collected into one journal file in
]a
]        uniform way.  Output to standard out and err can also be recorded
]in
]        the journal file by setting the TET_OUTPUT_CAPTURE variable.  This
]file
]        is then easily parsed to summarize the results.
]
]4.      TET will build, execute and clean up the tests.
]
]5.      Easy to setup.  Once the test source has been installed within the
]        TET directory structure only three short config files need to be
]        created.  One for building, one for executing and one for the
]        cleanup.  Secondly a scenario file must be created.  This is a
]simple
]        list of tests. All tests can be lumped together or separate
]scenarios
]        may be set up as I mentioned in item #2 above.
]
]Disadvantages:
]
]1.      TET does not allow for command line arguments to be passed.  Tests
]that
]        require arguments would need a wrapper.
]
]If used in this way I believe TET is a good solution.  It allows the tests
]to
]remain separate from the harness and provides many test management
]features.
]The "TETware User Guide" Revision 1.3 explains the demo test suites that
]accompany the TET3.3 download.  These demos provide a good example of how
]TET
]works.
]