view release on metacpan or  search on metacpan

scripts/biosql/load_ontology.pl  view on Meta::CPAN

Constructing a file that evaluates to a hash reference is very
simple. The first non-space character needs to be an open curly brace,
and the last non-space character a closing curly brace. In between the
curly braces, write option name enclosed by single quotes, followed by
=> (equal to or greater than), followed by the value in single
quotes. Separate each such option/value pair by comma. Here is an
example:

{ 
    '-dbname' => 'mybiosql', '-host' => 'foo.bar.edu', '-user' => 'cleo' 
}

Line breaks and white space don't matter (except if in the value
itself). Also note that options only have a single dash as prefix, and
they need to be those accepted by Bio::DB::BioDB->new()
(L<Bio::DB::BioDB>) or Bio::DB::SimpleDBContext->new()
(L<Bio::DB::SimpleDBContext>). Those sometimes differ slightly from the
option names used by this script, e.g., --dbuser corresponds to -user.

Note also that using the above example, you can use it for --initrc
and still connect as user caesar by also supplying --dbuser caesar on
the command line. I.e., command line arguments override any parameters
also found in the initrc file.

Finally, note that if using this option with default file name and the
default file is not found at any of the default locations, the option
will be ignored; it is not considered an error.

=item --namespace $namesp 

The namespace (name of the ontology) under which the terms and
relationships in the input files are to be created in the database
[bioperl ontology]. Note that the namespace will be left untouched if the
object(s) to be submitted has it set already.

Note that the DAG-edit flat file parser from more recent (1.2.2 and
later) bioperl releases can auto-discover the ontology name.

=item --lookup

Flag to look-up by unique key first, converting the insert into an
update if the object is found. This pertains to terms only, as there
is nothing to update about relationships if they are found by unique
key (the unique key comprises of all columns).

=item --noupdate

Don't update if object is found (with --lookup). Again, this only
pertains to terms.

=item --remove

Flag to remove terms before actually adding them (this necessitates a
prior lookup). Note that this is not relevant for relationships (if
one is found by lookup, removing and re-adding has essentially the
same result as leaving it untouched).

=item --noobsolete

Flag to exclude from upload terms marked as obsolete. Note that with
this flag, any update, removal, or object merge that you specify using
other parameters will not apply to obsolete terms. I.e., if you have
terms existing in your database that are marked as obsolete in the
input file, using this flag will prevent the existing terms from being
updated to reflect the obsolete status. Therefore, this flag is best
used when first loading an ontology. You may want to consider using
--updobsolete instead.

Note that relationships found in the input file(s) that reference an
obsolete term will be omitted from loading with this flag in effect.

=item --updobsolete

Flag to exclude from upload terms marked as obsolete unless they are
already present in the database. If they are, they will be updated,
and the --mergeobjs procedure will apply. If they are not, they will
be treated as if --noobsolete had been specified. Note that
relationships will not be updated for obsolete terms.

In contrast to --noobsolete, using this flag will increase the
database operations mildly (because of the look-ups necessary to
determine whether obsolete terms are present, and the subsequent
update for those that are), but it will capture change of status for
existing terms. At the same time, you won't load obsolete terms from a
new ontology that you haven't loaded before.

=item --delobsolete

Delete terms marked as obsolete from the database. Note that --remove
together with --noobsolete will have the same effect. Note also that
specifying this flag will not affect those terms that are only in your
database but not in the input file, regardless of whether they are
marked as obsolete or not.

Be aware that even though deleting obsolete terms may sound like a
very sane thing to do, you may have annotated features or bioentries using
those terms. Deleting the obsolete terms will then remove those
annotations (qualifier/value pairs) as well.

=item --safe

flag to continue despite errors when loading (the entire object
transaction will still be rolled back)

=item --testonly 

don't commit anything, rollback at the end

=item --format

This may theoretically be any OntologyIO format understood by
bioperl. All input files must have the same format.

Examples: 
    # this is the default
    --format goflat
    # Simple ASCII hierarchy (e.g., eVoc)
    --format simplehierarchy

Note that some formats may come with event-type parsers, specifically
with XML SAX event parsers. While those aren't truly
OntologyIO-compliant parsers (they can't be because OntologyIO defines
a stream of ontologies as the API), this script supports them
nevertheless. For instance, at the time of this writing there is an
InterPro XML SAX event handler (aliased to --format interprosax) which
will persist terms to the database as they are encountered in the
event stream, which greatly reduces the amount of memory
needed. Credit for conceiving this idea and writing the SAX handler
goes to Juguang Xiao, juguang at tll.org.sg.

=item --fmtargs

Use this argument to specify initialization parameters for the parser
for the input format. The argument value is expected to be a string

scripts/biosql/load_ontology.pl  view on Meta::CPAN

                    '-testonly'    => $testonly_flag,
                    );


# The input parser may in fact be a SAX event handler, not a truly
# OntologyIO-compliant parser. A SAX handler needs to be treated
# fundamentally different from this point on than an OntologyIO
# compliant parser. While the former is to be handed off to a XML SAX
# parser, the latter needs to be looped over the ontologies it
# returns.

if ($ontin->isa("Bio::OntologyIO::Handlers::BaseSAXHandler")) {
    # this is a SAX event handler, not a true OntologyIO parser

    # pull in the XML SAX parser
    eval {
        require XML::Parser::PerlSAX;
    };
    croak "failed to load required XML SAX parser:\n$@" if $@;
    
    # complete setup of the SAX event handler: pass in our persistence handlers
    $ontin->persist_term_handler(\&persist_term, @persist_args);
    $ontin->persist_relationship_handler(\&persist_relationship,@persist_args);
    $ontin->db($db);

    # make sure the (default) ontology has a name
    my $ont = $ontin->_ontology();
    $ont->name($namespace) unless $ont->name;

    # instantiate the XML SAX parser and pass it the event handler
    my $parser = XML::Parser::PerlSAX->new(Handler => $ontin);

    # parsing the file will persist all terms and relationships, so we need
    # to delete the relationships first to avoid having stale ones around
    print STDERR "\t...deleting all relationships for ",$ont->name,"\n";
    remove_all_relationships('-ontology' => $ont, @persist_args);

    # now go ahead and parse the file
    print STDERR "\t...parsing and loading ",$ont->name,"\n";
    $parser->parse(Source => {SystemId => $files[0]});

    # Generate the transitive closure if requested
    if($compute_tc) {
        print STDERR "\t... transitive closure\n";
        compute_tc($db, $ont, $ontin->term_factory(), $compute_tc);
    }
    
    print STDERR "\tDone with ",$ont->name,"\n";

} else {
    # this is a truly OntologyIO compliant parser, or so I hope

    # loop over the input stream(s)
    while( my $ont = $ontin->$nextobj ) {
        # don't forget to add namespace if the parser doesn't supply one
        $ont->name($namespace) unless $ont->name();
        
        print STDERR "Loading ontology ",$ont->name(),":\n\t... terms\n";

        # in order to allow callbacks to the user and generally a
        # better ability to interfere with and customize the upload
        # process, we load all terms first here instead of simply
        # going for the relationships

        foreach my $term ($ont->get_all_terms()) {
            # call the persistence handler - there is only one right now
            persist_term('-term' => $term, @persist_args);
        }

        # after all terms have been processed, we run through the relationships
        # more or less non-interactively (i.e., without invoking a callback)
        
        print STDERR "\t... relationships\n";

        # first off, we need to delete the existing relationships in order
        # to avoid having stale ones around
        remove_all_relationships('-ontology' => $ont, @persist_args);

        # now go and insert all of them
        foreach my $rel ($ont->get_relationships()) {
            # pass on to persistence function - there's only one right now
            persist_relationship('-rel' => $rel, @persist_args);
        }
        
        # Generate the transitive closure if requested
        if($compute_tc) {
            print STDERR "\t... transitive closure\n";
            compute_tc($db, $ont, $ontin->term_factory(), $compute_tc);
        }

        print STDERR "\tDone with ".$ont->name.".\n";
    }

    # close the parser explicitly in case it needs this to be called
    $ontin->close();
}

print STDERR "Done, cleaning up.\n";

if ($db && $testonly_flag) {
    $db->get_object_adaptor("Bio::Ontology::TermI")->rollback();
}
# done!

#################################################################
# Implementation of functions                                   #
#################################################################

sub parse_code{
    my $src = shift;
    my $code;

    # file or subroutine?
    if(-r $src) {
	if(! (($code = do $src) && (ref($code) eq "CODE"))) {
	    die "error in parsing code block $src: $@" if $@;
	    die "unable to read file $src: $!" if $!;
	    die "failed to run $src, or it failed to return a closure";
	}
    } else {
	$code = eval $src;