App-Glacier

 view release on metacpan or  search on metacpan

Changes  view on Meta::CPAN

2.10 2019-01-02

* The "periodic" subcommand

The new subcommand "periodic" is responsible for maintaining the job database.
It finishes up pending glacier jobs, dowloading the results (both inventory
listings and requested files) and removes expired jobs. It should be run as a
cronjob.

* When run on an EC2 instance, credentials can be obtained from the EC2 metadata

* Directory validity is controlled using the LastInventoryDate field

2.00 2018-03-08

* Redo the class system.

1.00 2018-02-24

* Initial release.

lib/App/Glacier.pm  view on Meta::CPAN

The following sections are recognized:

=over 4

=item B<[glacier]>

Configures access to the Glacier service.  The following keywords are defined:
    
=over 8

=item B<credentials => I<FILE>

Sets the name of the credentials file.  See below for a detailed discussion.
    
=item B<access => I<KEYNAME>

Defines Amazon access key or access ID for look up in the credentials file.
    
=item B<secret => I<SECRET>

Sets the secret key.  The use of this statement is discouraged for
security reason.    

=item B<region => I<NAME>

Sets the Amazon region.  If this setting is absent, B<glacier> will attempt
to retrieve the region from the instance store (assuming it is run on an EC2
AWS instance).    
    
=back

If either of B<access> or B<secret> is not supplied, B<glacier> attemtps to
obtain access and secret keys from the file named in the B<credentials>
setting (if it is defined). If unable to find credentials, B<glacier> attempts
to get credentials from the instance store, assuming it is run on an EC2
instance. It will exit if this attempt fails.    

The credentials file allows you to store all security sensitive data in a
single place and to tighten permissions accordingly. In the simplest case,
this file contains a single line with your access and secret keys separated
by a semicolon, e.g.:

    AEBRGYTEBRET:RTFERYABNERTYR4HDDHEYRTWW

Additionally, the default region can be specified after a second semicolon:

    AEBRGYTEBRET:RTFERYABNERTYR4HDDHEYRTWW:us-west-1
    
If you have several accounts, you can list their credentials on separate lines.
In that case, B<glacier> will select the account with the access key supplied
by the B<access> configuration statement, or the B<--account> command line
option.  If neither of these are supplied, the first account in the file will
be used.

To further facilitate selection of the credential pair, each line can be tagged
with the line B<#:I<NAME>> immediately preceding it.  In that case, the I<NAME>
can be used to select it using the B<--account> option or B<access> configuration statement.

Apart from these constructs, the credentials file can contain empty lines and
comments (lines beginning with B<#> followed by any character, except B<:> ),
which are ignored.

=item B<[transfer]>

Configures transfer values.  The section name can be optionally followed
by B<upload> or B<download> to indicate that it applies only to transfers
in that particular direction.    
    
=over 8

lib/App/Glacier/Bre.pm  view on Meta::CPAN

    my ($class, %opts) = @_;
    my $region = (delete $opts{region} || _get_instore_region())
	or return $class->new_failed('availability region not supplied');
    my $access = delete $opts{access};
    my ($secret,$token);
    if (defined($access)) {
	$secret = delete $opts{secret}
	  	or $class->new_failed('secret not supplied');
    } else {
	($access, $secret, $token) = _get_instore_creds()
	    or return $class->new_failed('no credentials supplied');
    }
    my $self = $class->SUPER::new($region, $access, $secret);
    if ($token) {
	# Overwrite the 'sig' attribute.
	# FIXME: The attribute itself is not documented, so this 
	# method may fail if the internals of the base class change
	# in its future releases.
	# This approach works with Net::Amazon::Glacier 0.15
	$self->{sig} = new App::Glacier::Signature($self->{sig}, $token);
    }
    return $self;
}

sub new_failed {
    my ($class, $message) = @_;
    bless { _error => $message }, $class;
}

my $istore_base_url = "http://169.254.169.254/latest/";
my $istore_document_path = "dynamic/instance-identity/document";
my $istore_credentials_path = "meta-data/iam/security-credentials/";

sub _get_instore_region {
    my $ua = LWP::UserAgent->new(timeout => 10);
    my $response = $ua->get($istore_base_url . $istore_document_path);
    unless ($response->is_success) {
	return undef;
    }
    my $doc = JSON->new->decode($response->decoded_content);
    return $doc->{region};
}

sub _get_instore_creds {
    my $ua = LWP::UserAgent->new(timeout => 10);
    my $url = $istore_base_url . $istore_credentials_path;
    my $response = $ua->get($url);
    unless ($response->is_success) {
	return undef;
    }
    chomp(my $name = $response->decoded_content);
    $url .= $name;
    $response = $ua->get($url);
    unless ($response->is_success) {
	return undef;
    }

lib/App/Glacier/Command.pm  view on Meta::CPAN

	}
	$$vref = $size;
    } else {
	return 'invalid size specification';
    }
}

my %parameters = (
    glacier => {
	section => {
	    credentials => 1,
	    access => 1,
	    secret => 1,
	    region => 1,
	}
    },
    transfer => {
	section => {
	    'single-part-size' => { default => 100*MB, check => \&ck_size },
	    'jobs' => { default => 16, check => \&ck_number },
	    'retries' => { default => 10, check => \&ck_number },

lib/App/Glacier/Command.pm  view on Meta::CPAN

    App::Glacier::Roster->configtest($self->cfget(qw(database job backend)),
				     $self->config, 'database', 'job')
	or exit(EX_CONFIG);
    App::Glacier::Directory->configtest($self->cfget(qw(database inv backend)),
					$self->config, 'database', 'inv')
	or exit(EX_CONFIG);

    unless ($self->{_config}->isset(qw(glacier access))
	    && $self->{_config}->isset(qw(glacier secret))) {
	if ($self->{_config}->isset(qw(glacier credentials))) {
	    my $creds = new App::Glacier::EclatCreds($self->{_config}->get(qw(glacier credentials)));
	    $account = $self->{_config}->get(qw(glacier access))
		unless defined $account;
	    if ($creds->has_key($account)) {
		$self->{_config}->set(qw(glacier access),
				      $creds->access_key($account));
		$self->{_config}->set(qw(glacier secret),
				      $creds->secret_key($account));
		$region = $creds->region($account) unless defined $region;
	    }
	}

lib/App/Glacier/EclatCreds.pm  view on Meta::CPAN

use strict;
use warnings;
use Carp;

require Exporter;
our @ISA = qw(Exporter);

sub _parse_access_file {
    my ($self, $file) = @_;
#    debug(1, "Looking for authentication credentials in $file");
    open(my $fd, '<', $file) or do {
	croak "$file: $!";
	return;
    };
    my $key;
    while (<$fd>) {
	chomp;
	s/^\s+//;
	if (/^#:\s*(.+?)\s*$/) {
	    $key = $1;