view release on metacpan or  search on metacpan

src/subversion/subversion/tests/README  view on Meta::CPAN

                ================================
                 A Subversion Testing Framework
                ================================


The three goals of Subversion's automated test-suite:

      1.  It must be easy to run.
      2.  It must be easy to understand the results.
      3.  It must be easy to add new tests.



Definition of an SVN "test program"
-----------------------------------

A Subversion test program is any executable that contains a number of
sub-tests it can run.  It has a standard interface:

1.  If run with a numeric argument N, the program runs sub-test N.

2.  If run with the argument `--list', it will list the names of all sub-tests.

3.  If run with no arguments, the program runs *all* sub-tests.

4.  The program returns either 0 (success) or 1 (if any sub-test failed).

5.  Upon finishing a test, the program reports the results in a format
    which is both machine-readable (for the benefit of automatic
    regression tracking scripts), and human-readable (for the sake of
    painstaking grovelling by hand in the dead of night):

      (PASS | FAIL): (argv[0]) (argv[1]): (description)

For example,

  [sussman@newton:~] ./frobtest 2
  PASS: frobtest 2: frobnicating fragile data
  [sussman@newton:~] 

Note that no particular programming language is required to write a
set of tests;  they just needs to export this user interface.



How to write new C tests
------------------------

The C test framework tests library APIs, both internal and external.

All test programs use a standard `main' function.  You write .c files
that contain only test functions --- you should not define your own
`main' function.

Instead, your code should define an externally visible array
`test_funcs', like this:

    /* The test table.  */
    struct svn_test_descriptor_t test_funcs[] =
    {
      SVN_TEST_NULL,
      SVN_TEST_PASS(test_a),
      SVN_TEST_PASS(test_b),
      SVN_TEST_PASS(test_c),
      SVN_TEST_NULL
    };

In this example, `test_a', `test_b', and `test_c' are the names of
test functions.  The first and last elements of the array must be
SVN_TEST_NULL.  The first SVN_TEST_NULL is there to leave room for
Buddha.  The standard `main' function searches for the final
SVN_TEST_NULL to determine the size of the array.

Instead of SVN_TEST_PASS, you can use SVN_TEST_XFAIL to declare that a
test is expected to fail. The status of such tests is then no longer
marked as PASS or FAIL, but rather as XFAIL (eXpected FAILure) or
XPASS (uneXpected PASS).

The purpose of XFAIL tests is to confirm that a known bug still
exists. When you see such a test uneXpectedly PASS, you've probably
fixed the bug it tests for, even if that wasn't your intention. :-)
XFAIL is not to be used as a way of testing a deliberately invalid
operation that is expected to fail when Subversion is working
correctly, nor as a place-holder for a test that is not yet written.

Each test function conforms to the svn_test_driver_t prototype:

        svn_error_t *f (const char **MSG,
                        svn_boolean_t MSG_ONLY
                        apr_pool_t *POOL);

When called, a test function should first set *MSG to a brief (as in,
half-line) description of the test.  Then, if MSG_ONLY is TRUE, the
test should immediately return SVN_NO_ERROR.  Else it should perform a
test.  If the test passes, the function should return SVN_NO_ERROR;
otherwise, it should return an error object, built using the functions
in svn_error.h.