view release on metacpan or  search on metacpan

lib/App/TestOnTap/Args.pod  view on Meta::CPAN

=head1 NAME

App::TestOnTap::Args

Commandline options and arguments understood for TestOnTap processing

=head1 VERSION

Version 1.001

=head1 SYNOPSIS

 testontap
 	[ @argfile ]      (see 'EXPANDING FILES IN ARGUMENT LIST' below for details) 
    [
        --usage | -?
            |
        --help | -h
            |
        --manual
            |
        --version
    ]
    [ -v | --verbose [ -v ... ] ]
    [ --workdirectory <path> ]
    [ { --skip || --include } <query> ]
    [ --jobs <n> ]
    [ --order <strategy> ]
    [ --timer || --no-timer ]
    [ --harness || --no-harness ]
    [ --dryrun || --no-dryrun ]
    [ --merge || --no-merge ]
    [ --configuration <cfgfile> || --cfg <cfgfile> ]
    [ --archive ]
    [ --savedirectory <path> ]
    [ --define | -D <key>=<value> [ --define ... ] ]
    <suitepath>
    [<suiteargs> ...]  

=head1 OPTIONS AND ARGUMENTS

All options can be abbreviated, as long as they are unambiguous. Option matching is case
sensitive. Forcibly end option parsing using '--'.

=head3 EXPANDING FILES IN ARGUMENT LIST

The command line may at any place contain arguments of the form F<@filename>, which means that the F<filename>
will be read for arguments to insert in place. This will help if the command line is very long or is just nicely
persisted for reuse.

Further, this will work recursively, i.e. any resulting argument of the same form, will be read. If the filename
is relative, it's assumed relative to the directory the previous file was in.

The example command line C<testontap --verbose @myargs --jobs 10 @moreargs> will be expanded by reading the F<myargs>
and F<moreargs> files (in the current directory, obviously). The read files may contain a line of
C<@/some/path/to/globalargs>, and assuming that file in turn contains C<@extraargs>,
that file is expected to exist as F</some/path/to/extraargs>.

Note that the file is line-oriented - each line in such a file will become a B<single> argument. No whitespace
splitting inside each value will occur, but some rules apply:

=over

=item * Environment variable expansion

Any construction of the form C<${name}> will be assumed to be a reference to the environment variable
'name', and the contents of it will be inserted in place. If the environment variable doesn't exist
it is an error.

=item * Line continuation

To avoid physical lines in the file to be extremely long, it can be continued on the next
physical (to any length) by ending the line with a backslash (\).

  Example:
    fee \
    fie \
    foo

Will become the argument 'fee fie foo'.

=item * Comments

Any line (after completed line continuation and not counting inital whitespace) having
a '#' as the first character, will be ignored as being a comment.

=item * Trimming

Any beginning and ending whitespace will be trimmed away.

lib/App/TestOnTap/Args.pod  view on Meta::CPAN

For more on selecting tests, also see the L<manual regarding the 'skip' config setting|App::TestOnTap/"THE SUITE CONFIG">.

=item B<--jobs E<lt>nE<gt>>

By default, tests will run in sequence, governed by any dependencies, and sorted alphabetically. Also by
default, they will always run serially.

By setting this option to a value higher than 1 allowing a maximum of I<n> jobs to execute concurrently.
However, for this to take effect, a C<parallelizable> rule must be in the suite configuration in order to
select which tests actually can run in parallel

The B<actual> amount of concurrently executing jobs are then subject to limiting factors, such as dependencies and
the parallelizability of a given test.

For more on parallelization, see the L<manual regarding the 'parallelizable' config setting|App::TestOnTap/"THE SUITE CONFIG">.

B<IMPORTANT NOTE:>
On Windows in particular, fork() is constrained in the number of parallel forks, so this value will
be capped to a maximum of 60 at this point, while defaulting to effectively 'infinite' on other platforms.
To experiment with other maximums, set the environment variable C<_TESTONTAP_MAX_JOBS> to the desired number.

=item B<--order E<lt>strategyE<gt>>

Setting a strategy controls the ordering of the tests.

However, note that the actual order of tests are primarily controlled by dependencies and possibly
the parallelizability of any given test. Thus, the order strategy is only relevant
when there are multiple tests to choose from during evaluation of 'next available test(s)'.

The default strategy is 'none', but the default for a given suite may also be set in its configuration.

=over

=item * C<none>

This is the default strategy and the resulting order depends on internal implementation details.
It may or may not look random, but for a guaranteed randomization, use the C<random> strategy.

=item * C<alphabetic> or C<ralphabetic>

This strategy simply uses an alphabetic sort on test names. Thus, this results in the order
'sm1', 'sm10', 'sm2' etc.

Adding the initial C<r> causes the order to be reversed.  

=item * C<natural> or C<rnatural>

This strategy makes a best effort to take numbering into account when sorting. It is basically alphabetic,
but takes embedded numbers into account if possible. Thus, this results in the order
'sm1', 'sm2', 'sm10' etc.

Adding the initial C<r> causes the order to be reversed.  

=item * random

This will explicitly shuffle the order randomly.

=back

=item B<--timer>, B<--no-timer>

Using C<--timer> turns on the display of timing information for tests when running.

Note that the information is always present in persisted results, this just controls the display when running a suite. 

=item B<--harness>, B<--no-harness>

By default test output is handled by a test harness.

Using C<--no-harness> turns on a special mode where tests are run with no capture and parsing of
output. Printouts go directly to stdout/err, and stdin is active.

This can be convenient to directly see the TAP (or other) output which can be handy
for debugging the tests themselves. Also, since stdin should work normally, using an
execmap that starts the test in debug mode enables them to run under a real debugger.

 # Example config file 'debug.cfg':
 
 # (copies of the real config settings)
 
 execmap = debug normal
 
 [EXECMAP normal]
 match = regexp(\.pl$)
 cmd = perl

 [EXECMAP debug]
 match = eq(some/test.pl)
 cmd = perl -d

and run with:

 testontap --no-harness --configuration debug.execmap some/path/to/suite
 
would start the some/test.pl file under the Perl debugger but run the rest normally.

Note that this renders several options and mechanisms useless:
for example C<--verbose>, C<--jobs>, C<--timer>. Also, the work directory
will miss most of the data normally seen, such as TAP output, aggregated
test results etc. C<--no-harness> will be automatically used when C<--dryrun> is used

=item B<--dryrun>, B<--no-dryrun>

By using C<--dryrun>, C<--no-harness> is implied.

No tests will be actually run, the tests will just be listed in the order they would have run in
the C<--no-harness> format plus lists of what any dependency queries actually select.

=item B<--merge>, B<--no-merge>

By default, writes to stderr in tests are passed directly through to the console (but see below).

However, as STDERR is unbuffered, this tends to make the console look very ugly and
the output are only marginally in sync with the ordinary harness output.

Using C<--merge> makes the harness merge stdout and stderr as one, which makes them
synchronized. However, the output is no longer visible unless C<--verbose> is used.

If any of the options C<--archive>, C<--savedirectory> or C<--workdirectory> are given, merging
is turned on, in order to force the output to the result 'tap' files. If this is not desired,
use C<--no-merge> explicitly.

=item B<--configuration E<lt>cfgfileE<gt>> (or B<--cfg ...>)

By default, the test suite is required to have a configuration file (even if it's empty).
With this option, another configuration file can be used instead. 

If this option is not used, and the configuration in the suite, some internal defaults will be used.  

For more on configuration, see the L<manual regarding the config setting|App::TestOnTap/"THE SUITE CONFIG">.

=item B<--archive>, B<--savedirectory E<lt>pathE<gt>> 

By default, the results of a test run are not saved.

By providing either or both of these options, results may be saved as either a ready-made archive
or a regular directory. The name of the saved file/directory will be named using the basename of the
suite plus a timestamp and the run id, which together should be entirely unique.

Giving only C<--archive>, the result will be saved as an archive in the current directory.

Giving C<--archive --savedirectory some/path> together, the result will be saved as an archive
in the directory some/path (the directory may or may not exist beforehand).
 
Giving only C<--savedirectory some/path>, the result archive will be a regular subdirectory in the
directory some/path (the directory may or may not exist beforehand).

For details of saved results, see the L<manual regarding 'persisted results'|App::TestOnTap/"PERSISTED RESULT">.

NOTE: by using either of these flags, the C<--merge> option is automatically turned on.

=item B<--define E<lt>keyE<gt>=E<lt>valueE<gt>>

The function of this option is to make it possible to pass on arbitrary key/value pairs to a saved result
in order to later enable various types of differentiation to be made when processing the saved result.
The defines are not made available to tests.

Can be given multiple times.