view release on metacpan or  search on metacpan

bin/spipe  view on Meta::CPAN

$p->run();

__END__

=pod

=head1 NAME

spipe - simple pipeline running interface

=head1 VERSION

version 0.9.1

=head1 SYNOPSIS

B<spipe> [B<--version> | [B<-?|-h|--help>] | [B<-g|--debug>] |
   B<[--graphviz> | B<[-c|--config> file | [B<[-d|--directory> value] |
   B<[-i|--input> string| B<[-it|--itype> string |
   [B<[--start> value] | [B<[--stop> value]

  spipe -config t/data/string_manipulation.yml -d /tmp/test

=head1 DESCRIPTION

Spipe is a control script for running simple pipelines read from
configuration files written in YAML language.

For internal details of the pipeline, check the documentation for the
perl module L<App::Pipeline::Simple>.

=head1 NAME

spipe - simple pipeline running interface

=head1 OPTIONS

=over 7

=item B<-v | --version>

Print out a line with the program name and version number.

=item B<-? | -h | --help>

Show this help.

=item B<-g | --debug>

Print out the UNIX command line equivalent of the pipeline and exit.

Reports parsing and logical errors.

=item B<--graphviz>

Print out a graphviz dot file.

Example one liner to display a graph of the pipeline:

  spipe -config t/data/string_manipulation.yml -graph > \
  /tmp/p.dot; dot -Tpng /tmp/p.dot| display

=item B<-c | --config> string

Path to the config file. Required unless there is a file called
config.yml in the current directory.

=item B<-d | --directory> string

Directory to keep all files.

If the directory does not exist, it will be created and a copy of the
config file will be copied into it under name C<config.yml>.

For subsequent runs of the that pipeline, you adjust the parameters in
the configuration file and rerun spipe without -config and -directory
options.

=item B<-i | --input> string

Optional input to pipeline.

=item B<-it | --itype> string

Type of the optional input. Values?

=item B<--start> string

ID of the step to start or restart the pipeline.

Fails if the prerequisites of the step are not met, i.e. the input
file(s) does not exist.

=item B<--stop> string

ID of the step to stop the pipeline. Defaults to the last step.

=item B<--verbose> int

Verbosity level. Defaults to zero. This will get translated to
Log::Log4perl levels:

  verbose   =  -1    0     1     2
  log level =  DEBUG INFO  WARN  ERROR

=back

=head1 RUNNING

Example run:

  spipe -config t/data/string_manipulation.xml -dir /tmp/test

reads instructions from the config file and writes all information to
the project directory.

The debug option will parse the config file, print out the command
line equivalents of all commands and print out warnings of problems
encountered in the file:

  spipe -config t/data/string_manipulation.xml -dir /tmp/test

An other tool integrated in the system is visualization of the
execution graph. It is done with the help of L<GraphViz> perl
interface module that will need to be installed from CPAN.

The following command line creates a Graphviz dot file, converts it
into an image file and opens it with the Imagemagic display program:

  spipe -config t/data/string_manipulation.xml -graph > \
    /tmp/p.dot; dot -Tpng /tmp/p.dot | display

=head1 CONFIGURATION

The default configuration is written in YAML, a simple and human
readable language that can be parsed in many languages cleanly into
data structures.

The YAML file contains four top level keys for the hash that the file
will be read into: 1) C<name> to give the pipeline a short name, 2)
C<version> to indicate the version number, 3) C<description> to give a
more verbose explanation what the pipeline does, and 4) C<steps>
listing pipeline steps.

  ---
  description: "Example of a pipeline"
  name: String Manipulation
  version: '0.4'
  steps:

Each C<step> is identified by an unique short ID and has a C<name>
that identifies an executable somewhere in the system
path. Alternatively, you can give the full path leading to the
executable file with key C<path>. The name will be added to the path
and padded with a suitable separator character when needed.

Arguments to the executable are given individually as key/value pairs
within the C<args> tag. A single hyphen is added in front of the
argument key when they are executed. If two hyphens are needed, just
add one the key. Arguments can exist without values, too.

  s3:
    name: cat
    args:
      in:
        type: redir
        value: s1.txt
      n:
      out:
        type: redir
        value: s3_mod.txt
    next:
      - s4

There are two special keys C<in> and C<out> that need to have a key
 C<type> defined. The IO C<type> can get several kinds of values:

=over

=item C<unnamed>

that indicates that the argument is an unnamed argument to the
executable

=item C<redir>

will be interpreted as UNIX redirection character '&lt' or '&gt'
depending on the context

=item C<file>