BioX-Workflow-Plugin-Drake
view release on metacpan or search on metacpan
workflow.yml
---
plugins:
- Drake
global:
- indir: /home/user/workflow
- outdir: /home/user/workflow/output
- file_rule: (.csv)$
rules:
- backup:
local:
- INPUT: "{$self->indir}/{$sample}.csv"
- OUTPUT: "{$self->outdir}/{$sample}.csv"
- thing: "other thing"
process: |
cp $INPUT $OUTPUT
- grep_VARA:
local:
- OUTPUT: "{$self->outdir}/{$sample}.grep_VARA.csv"
process: |
;
;
; Samples: test1, test2
;
;
; Starting Workflow
;
;
; Starting backup
;
;
; Variables
; Indir: /home/guests/jir2004/workflow
; Outdir: /home/guests/jir2004/workflow/output/backup
; Local Variables:
; INPUT: {$self->indir}/{$sample}.csv
; OUTPUT: {$self->outdir}/{$sample}.csv
; thing: other thing
;
/home/guests/jir2004/workflow/output/backup/$[SAMPLE].csv <- /home/guests/jir2004/workflow/$[SAMPLE].csv
cp $INPUT $OUTPUT
;
; Ending backup
;
;
; Starting grep_VARA
;
Run drake
drake --workflow workflow.full.drake
The following steps will be run, in order:
1: /home/user/workflow/output/backup/test1.csv <- /home/user/workflow/test1.csv [timestamped]
2: /home/user/workflow/output/backup/test2.csv <- /home/user/workflow/test2.csv [timestamped]
3: /home/user/workflow/output/grep_vara/test1.grep_VARA.csv <- /home/user/workflow/output/backup/test1.csv [projected timestamped]
4: /home/user/workflow/output/grep_vara/test2.grep_VARA.csv <- /home/user/workflow/output/backup/test2.csv [projected timestamped]
5: /home/user/workflow/output/grep_varb/test1.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test1.grep_VARA.csv [projected timestamped]
6: /home/user/workflow/output/grep_varb/test2.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test2.grep_VARA.csv [projected timestamped]
Confirm? [y/n] y
Running 6 steps with concurrence of 1...
--- 0. Running (timestamped): /home/user/workflow/output/backup/test1.csv <- /home/user/workflow/test1.csv
--- 0: /home/user/workflow/output/backup/test1.csv <- /home/user/workflow/test1.csv -> done in 0.02s
--- 1. Running (timestamped): /home/user/workflow/output/backup/test2.csv <- /home/user/workflow/test2.csv
--- 1: /home/user/workflow/output/backup/test2.csv <- /home/user/workflow/test2.csv -> done in 0.01s
--- 2. Running (timestamped): /home/user/workflow/output/grep_vara/test1.grep_VARA.csv <- /home/user/workflow/output/backup/test1.csv
Working on /home/user/workflow/output/backup/test1csv
--- 2: /home/user/workflow/output/grep_vara/test1.grep_VARA.csv <- /home/user/workflow/output/backup/test1.csv -> done in 0.01s
--- 3. Running (timestamped): /home/user/workflow/output/grep_vara/test2.grep_VARA.csv <- /home/user/workflow/output/backup/test2.csv
Working on /home/user/workflow/output/backup/test2csv
--- 3: /home/user/workflow/output/grep_vara/test2.grep_VARA.csv <- /home/user/workflow/output/backup/test2.csv -> done in 0.01s
--- 4. Running (timestamped): /home/user/workflow/output/grep_varb/test1.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test1.grep_VARA.csv
--- 4: /home/user/workflow/output/grep_varb/test1.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test1.grep_VARA.csv -> done in 0.01s
--- 5. Running (timestamped): /home/user/workflow/output/grep_varb/test2.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test2.grep_VARA.csv
--- 5: /home/user/workflow/output/grep_varb/test2.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test2.grep_VARA.csv -> done in 0.08s
Done (6 steps run).
Run in minified mode
need to split up your workflows as opposed to using the --resample
option.
biox-workflow.pl --workflow workflow.yml --min 1 > workflow.drake #This also creates the run-workflow.sh in the same directory
./run-workflow.sh
cat drake.log #Here is the log for the first run
2015-06-21 14:02:47,543 INFO Running 3 steps with concurrence of 1...
2015-06-21 14:02:47,568 INFO
2015-06-21 14:02:47,570 INFO --- 0. Running (timestamped): /home/user/workflow/output/backup/test1.csv <- /home/user/workflow/test1.csv
2015-06-21 14:02:47,592 INFO --- 0: /home/user/workflow/output/backup/test1.csv <- /home/user/workflow/test1.csv -> done in 0.02s
#So on and so forth
If you look in the example directory you will see a few png files,
these are outputs of the drake workflow.
=cut
Acknowledgements
example/drake.yml view on Meta::CPAN
---
plugins:
- Drake
global:
- indir: /home/jillian/projects/perl/BioX-Workflow/example
- outdir: /home/jillian/projects/perl/BioX-Workflow/example/OUT
- file_rule: (.csv)$
- min: 1
rules:
- backup:
local:
- INPUT: "{$self->indir}/{$sample}.csv"
- OUTPUT: "{$self->outdir}/{$sample}.csv"
- thing: "other thing"
process: |
cp $INPUT $OUTPUT
- grep_VARA:
local:
- OUTPUT: "{$self->outdir}/{$sample}.grep_VARA.csv"
process: |
lib/BioX/Workflow/Plugin/Drake.pm view on Meta::CPAN
=head3 workflow.yml
---
plugins:
- Drake
global:
- indir: /home/user/workflow
- outdir: /home/user/workflow/output
- file_rule: (.csv)$
rules:
- backup:
local:
- INPUT: "{$self->indir}/{$sample}.csv"
- OUTPUT: "{$self->outdir}/{$sample}.csv"
- thing: "other thing"
process: |
cp $INPUT $OUTPUT
- grep_VARA:
local:
- OUTPUT: "{$self->outdir}/{$sample}.grep_VARA.csv"
process: |
lib/BioX/Workflow/Plugin/Drake.pm view on Meta::CPAN
;
;
; Samples: test1, test2
;
;
; Starting Workflow
;
;
; Starting backup
;
;
; Variables
; Indir: /home/guests/jir2004/workflow
; Outdir: /home/guests/jir2004/workflow/output/backup
; Local Variables:
; INPUT: {$self->indir}/{$sample}.csv
; OUTPUT: {$self->outdir}/{$sample}.csv
; thing: other thing
;
/home/guests/jir2004/workflow/output/backup/$[SAMPLE].csv <- /home/guests/jir2004/workflow/$[SAMPLE].csv
cp $INPUT $OUTPUT
;
; Ending backup
;
;
; Starting grep_VARA
;
Run drake
drake --workflow workflow.full.drake
The following steps will be run, in order:
1: /home/user/workflow/output/backup/test1.csv <- /home/user/workflow/test1.csv [timestamped]
2: /home/user/workflow/output/backup/test2.csv <- /home/user/workflow/test2.csv [timestamped]
3: /home/user/workflow/output/grep_vara/test1.grep_VARA.csv <- /home/user/workflow/output/backup/test1.csv [projected timestamped]
4: /home/user/workflow/output/grep_vara/test2.grep_VARA.csv <- /home/user/workflow/output/backup/test2.csv [projected timestamped]
5: /home/user/workflow/output/grep_varb/test1.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test1.grep_VARA.csv [projected timestamped]
6: /home/user/workflow/output/grep_varb/test2.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test2.grep_VARA.csv [projected timestamped]
Confirm? [y/n] y
Running 6 steps with concurrence of 1...
--- 0. Running (timestamped): /home/user/workflow/output/backup/test1.csv <- /home/user/workflow/test1.csv
--- 0: /home/user/workflow/output/backup/test1.csv <- /home/user/workflow/test1.csv -> done in 0.02s
--- 1. Running (timestamped): /home/user/workflow/output/backup/test2.csv <- /home/user/workflow/test2.csv
--- 1: /home/user/workflow/output/backup/test2.csv <- /home/user/workflow/test2.csv -> done in 0.01s
--- 2. Running (timestamped): /home/user/workflow/output/grep_vara/test1.grep_VARA.csv <- /home/user/workflow/output/backup/test1.csv
Working on /home/user/workflow/output/backup/test1csv
--- 2: /home/user/workflow/output/grep_vara/test1.grep_VARA.csv <- /home/user/workflow/output/backup/test1.csv -> done in 0.01s
--- 3. Running (timestamped): /home/user/workflow/output/grep_vara/test2.grep_VARA.csv <- /home/user/workflow/output/backup/test2.csv
Working on /home/user/workflow/output/backup/test2csv
--- 3: /home/user/workflow/output/grep_vara/test2.grep_VARA.csv <- /home/user/workflow/output/backup/test2.csv -> done in 0.01s
--- 4. Running (timestamped): /home/user/workflow/output/grep_varb/test1.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test1.grep_VARA.csv
--- 4: /home/user/workflow/output/grep_varb/test1.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test1.grep_VARA.csv -> done in 0.01s
--- 5. Running (timestamped): /home/user/workflow/output/grep_varb/test2.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test2.grep_VARA.csv
--- 5: /home/user/workflow/output/grep_varb/test2.grep_VARA.grep_VARB.csv <- /home/user/workflow/output/grep_vara/test2.grep_VARA.csv -> done in 0.08s
Done (6 steps run).
=head3 Run in minified mode
lib/BioX/Workflow/Plugin/Drake.pm view on Meta::CPAN
This WILL break with use of --resample, either local or global. You need to
split up your workflows as opposed to using the --resample option.
biox-workflow.pl --workflow workflow.yml --min 1 > workflow.drake #This also creates the run-workflow.sh in the same directory
./run-workflow.sh
cat drake.log #Here is the log for the first run
2015-06-21 14:02:47,543 INFO Running 3 steps with concurrence of 1...
2015-06-21 14:02:47,568 INFO
2015-06-21 14:02:47,570 INFO --- 0. Running (timestamped): /home/user/workflow/output/backup/test1.csv <- /home/user/workflow/test1.csv
2015-06-21 14:02:47,592 INFO --- 0: /home/user/workflow/output/backup/test1.csv <- /home/user/workflow/test1.csv -> done in 0.02s
#So on and so forth
If you look in the example directory you will see a few png files, these are outputs of the drake workflow.
=cut
=head1 Acknowledgements
Before version 0.03
( run in 1.274 second using v1.01-cache-2.11-cpan-49f99fa48dc )