view release on metacpan or  search on metacpan

lib/BioX/Workflow/Plugin/Drake.pm  view on Meta::CPAN

                - thing: "other thing"
            process: |
                cp $INPUT $OUTPUT
        - grep_VARA:
            local:
                - OUTPUT: "{$self->outdir}/{$sample}.grep_VARA.csv"
            process: |
                echo "Working on {$self->{indir}}/{$sample.csv}"
                grep -i "VARA" {$self->indir}/{$sample}.csv >> {$self->outdir}/{$sample}.grep_VARA.csv \
                || touch {$self->OUTPUT}
        - grep_VARB:
            local:
                - OUTPUT: "{$self->outdir}/{$sample}.grep_VARA.grep_VARB.csv"
            process: |
                grep -i "VARB" {$self->indir}/{$sample}.grep_VARA.csv >> {$self->outdir}/{$sample}.grep_VARA.grep_VARB.csv || touch {$self->OUTPUT}

=head3 Notes on the drake.yml

Drake will stop everything if you're job returns with an exit code of anything
besides 0. For this reason we have the last command have a command1 || command2
syntax, so that even if we don't grep any "VARB" from the file the workflow
could continue.

=head3 Run it with default setup

    biox-workflow.pl --workflow workflow.yml > workflow.full.drake

=head3 Output with default setup

I don't want to inlcude the whole file, but you get the idea

    ;
    ; Generated at: 2015-06-21T11:01:24
    ; This file was generated with the following options
    ;	--workflow	drake.yml
    ;	--min	1
    ;

    ;
    ; Samples: test1, test2
    ;
    ;
    ; Starting Workflow
    ;

    ;
    ; Starting backup
    ;


    ;
    ; Variables
    ; Indir: /home/guests/jir2004/workflow
    ; Outdir: /home/guests/jir2004/workflow/output/backup
    ; Local Variables:
    ;	INPUT: {$self->indir}/{$sample}.csv
    ;	OUTPUT: {$self->outdir}/{$sample}.csv
    ;	thing: other thing
    ;

    /home/guests/jir2004/workflow/output/backup/$[SAMPLE].csv <- /home/guests/jir2004/workflow/$[SAMPLE].csv
        cp $INPUT $OUTPUT


    ;
    ; Ending backup
    ;


    ;
    ; Starting grep_VARA
    ;


Run drake

    drake --workflow workflow.full.drake

    The following steps will be run, in order:
      1: /home/user/workflow/output/backup/test1.csv <- /home/user/workflow/test1.csv [timestamped]
      2: /home/user/workflow/output/backup/test2.csv <- /home/user/workflow/test2.csv [timestamped]
      3: /home/user/workflow/output/grep_vara/test1.grep_VARA.csv <- /home/user/workflow/output/backup/test1.csv [projected timestamped]
      4: /home/user/workflow/output/grep_vara/test2.grep_VARA.csv <- /home/user/workflow/output/backup/test2.csv [projected timestamped]
      5: /home/user/workflow/output/grep_varb/test1.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test1.grep_VARA.csv [projected timestamped]
      6: /home/user/workflow/output/grep_varb/test2.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test2.grep_VARA.csv [projected timestamped]
    Confirm? [y/n] y
    Running 6 steps with concurrence of 1...

    --- 0. Running (timestamped): /home/user/workflow/output/backup/test1.csv <- /home/user/workflow/test1.csv
    --- 0: /home/user/workflow/output/backup/test1.csv <- /home/user/workflow/test1.csv -> done in 0.02s

    --- 1. Running (timestamped): /home/user/workflow/output/backup/test2.csv <- /home/user/workflow/test2.csv
    --- 1: /home/user/workflow/output/backup/test2.csv <- /home/user/workflow/test2.csv -> done in 0.01s

    --- 2. Running (timestamped): /home/user/workflow/output/grep_vara/test1.grep_VARA.csv <- /home/user/workflow/output/backup/test1.csv
    Working on /home/user/workflow/output/backup/test1csv
    --- 2: /home/user/workflow/output/grep_vara/test1.grep_VARA.csv <- /home/user/workflow/output/backup/test1.csv -> done in 0.01s

    --- 3. Running (timestamped): /home/user/workflow/output/grep_vara/test2.grep_VARA.csv <- /home/user/workflow/output/backup/test2.csv
    Working on /home/user/workflow/output/backup/test2csv
    --- 3: /home/user/workflow/output/grep_vara/test2.grep_VARA.csv <- /home/user/workflow/output/backup/test2.csv -> done in 0.01s

    --- 4. Running (timestamped): /home/user/workflow/output/grep_varb/test1.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test1.grep_VARA.csv
    --- 4: /home/user/workflow/output/grep_varb/test1.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test1.grep_VARA.csv -> done in 0.01s

    --- 5. Running (timestamped): /home/user/workflow/output/grep_varb/test2.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test2.grep_VARA.csv
    --- 5: /home/user/workflow/output/grep_varb/test2.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test2.grep_VARA.csv -> done in 0.08s
    Done (6 steps run).


=head3 Run in minified mode

As an alternative you can run this with the --min option, which instead of
printing out each workflow prints out only one, and creates a run-workflow.sh
which has all of your environmental variables.

This option is preferable if running on an HPC cluster with many nodes.

This WILL break with use of --resample, either local or global. You need to
split up your workflows as opposed to using the --resample option.

lib/BioX/Workflow/Plugin/Drake.pm  view on Meta::CPAN

    if($self->min){
        $self->full(0);
    }
    $self->wait(0);
    $self->comment_char(';');
};

=head3 after get_samples

Things to do if we decide to do a min version

=cut

after 'get_samples' => sub{
    my($self) = shift;

    return unless $self->min;

    open(my $fh, '>', 'run-workflow.sh') or die print "Could not open file $!\n";

    print $fh "#!/bin/bash\n\n";

    foreach my $sample (@{$self->samples}){
        print $fh <<EOF;
drake --vars "SAMPLE=$sample" --workflow workflow.drake
EOF
    }

    close $fh;

    chmod 0777, 'run-workflow.sh';

    $self->samples(["\$SAMPLE"]);
};

=head3 write_process

Fill in the template with the process

Ensure INPUT/OUTPUT exist

Prettyify the output a bit

=cut

before 'write_process' => sub{
    my($self) = shift;

    $DB::single=2;
    if((! $self->local_attr->exists('INPUT')) && ! $self->local_attr->exists('OUTPUT') ){
        print "$self->{comment_char} There is no INPUT or OUTPUT!\n";
    }

    #Make the formatting a big prettier
    my @tmp = split("\n", $self->process);
    $self->process(join("\n\t", @tmp));

    my $tmp = "{\$self->OUTPUT} <- {\$self->INPUT}\n\t";
    $DB::single=2;
    if($self->min){
        $tmp =~ s/\$SAMPLE/\$[SAMPLE]/g;
    }
    my $newprocess = $tmp.$self->process;
    $self->process($newprocess);
};

#before 'process_template' => sub {
    #my $self = shift;

    #my $tmp = "{\$self->OUTPUT} <- {\$self->INPUT}\n\t";
    #$DB::single=2;
    #if($self->min){
        #$tmp =~ s/\$SAMPLE/\$[SAMPLE]/g;
    #}
    #my $newprocess = $tmp.$self->process;
    #$self->process($newprocess);

#};

1;
__END__

=encoding utf-8

=head1 AUTHOR

Jillian Rowe E<lt>jillian.e.rowe@gmail.comE<gt>

=head1 COPYRIGHT

Copyright 2015- Jillian Rowe

=head1 LICENSE

This library is free software; you can redistribute it and/or modify
it under the same terms as Perl itself.

=head1 SEE ALSO

=cut