[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [LTP] Re: TET [Mea Culpa]




By setting TET_API_COMPLIANT to False the test cases could remain
completely
separate from the test harness.  By doing this there would be no dependency
on
the TET harness to execute any of the tests.  The only requirement would be
that the tests return zero on success.  I have successfully ran the
quickhit tests under
TET3.3 with exception of the ones that take command line arguments.  As far
as
I can tell there is no way to pass command line arguments to a test case
while
running TET without its API framework.

Some of the advantages I see to using TET:

1.      The scenario file can be setup to allow tests to be grouped.  This
would
        accommodate those who want to test a specific feature (e.g. io,
threads,
        stress).

2.      The scenarios can be setup to run tests repeatedly, in parallel, at
        random, and/or for a specified amount of time.

3.      The results of all the tests are collected into one journal file in
a
        uniform way.  Output to standard out and err can also be recorded
in
        the journal file by setting the TET_OUTPUT_CAPTURE variable.  This
file
        is then easily parsed to summarize the results.

4.      TET will build, execute and clean up the tests.

5.      Easy to setup.  Once the test source has been installed within the
        TET directory structure only three short config files need to be
        created.  One for building, one for executing and one for the
        cleanup.  Secondly a scenario file must be created.  This is a
simple
        list of tests. All tests can be lumped together or separate
scenarios
        may be set up as I mentioned in item #2 above.

Disadvantages:

1.      TET does not allow for command line arguments to be passed.  Tests
that
        require arguments would need a wrapper.

If used in this way I believe TET is a good solution.  It allows the tests
to
remain separate from the harness and provides many test management
features.
The "TETware User Guide" Revision 1.3 explains the demo test suites that
accompany the TET3.3 download.  These demos provide a good example of how
TET
works.