Amazon-S3
view release on metacpan or search on metacpan
Thu Jul 14 06:34:56 2022 Rob Lauer <rlauer6@comcast.net>>
[0.55 - use XML::LibXML]:
* VERSION: bump
* src/main/perl/lib/Amazon/S3.pm.in: use XML::LibXML, not XML::Simple
- perlcritic cleanups
- pod cleanup
(new)
- cache_signer
- encrypt credentials
(get_default_region): new
(get_aws_access_key_id): new
(get_aws_secret_access_key): new
(get_token): new
(_decrypt): new
(_encrypt): new
(signer)
- accesses _signer now
- set default region to caller's value or default
(buckets): set region to us-east-1 temporarily
- don't treat warnings as errors
- output package version
- check for rpmbuild, pod2markdown, scandeps-static.pl
- update dependencies
* src/main/perl/lib/Amazon/S3.pm.in
- new logging option
- mv'd from dist/lib/Amazon/S3.pm
- documenation updates
- refactoring, perlcritic cleanups
- use Amazon::S3::Constants
- allow passing credentials class
(_make_request): check dns_bucket_names option
(dns_bucket_names): new
(get_logger): new
(level): new
(get_credentials): new
(new)
- secure defaults to true
(region): new
(list_bucket_v2): new
(list_bucket_all_v2): new
(last_response): new
* src/main/perl/lib/Amazon/S3/Bucket.pm.in
- mv'd from dist/lib/Amazon/S3/Bucket.pm
- revert using XML::Simple
- use Amazon::S3::Constants
* autotools/ax_distcheck_hack.m4: new
* autotools/ax_rpmbuild_check.m4: new
Revision history for Perl module Amazon::S3:
0.48 Sep 17 2021
- remove leading '/' from bucket name
- modify tests for not s3.amazonaws.com hosts (e.g. minio)
0.47 Feb 4 2019
- update dependencies (XML::LibXML)
0.46 Jan 19 2019
- use temporary credentials
- error string
-
0.45 Aug 14 2009
- Applied patch to support any S3 end points (Tokuhiro Matsuno)
- Applied patches to not die when 0 or 1 buckets are return also (Tokuhiro Matsuno)
0.441 Jun 16 2008
- Added handling to get_key_filename where the key is used as the file
name if one is not provided.
0.44 Jun 08 2008
- Applied patch for fixing non-xml error parsing (Harold Sinclair)
README-TESTING.md view on Meta::CPAN
```
# Credentials for Testing
You should set the environment variables `AWS_ACCESS_KEY_ID` and
`AWS_ACCESS_SECRET_KEY` to your AWS credential values that have the
ability to create and write to buckets.
If you set environment variable `AMAZON_S3_CREDENTIAL` to any value,
the tests will use the `Amazon::Credentials` module to look for valid
credentials in your environment, your credentials files or the
instance role if you are running on an EC2.
# Using S3 Mocking Services
If you want to test *some* parts of this module but don't want to
spend a few pennies (or don't have access to AWS credentials) you can
try one of the S3 mocking services. The two of the most popular services
seem to be:
* [LocalStack](https://localstack.io)
* [minio](https://min.io)
Both of these implement a subset of the S3 API. __Note that Some tests will fail
on both services (as of the writing of this document).__ To make it
through the tests, try setting one or more of the environment
variables above which will selectively skip some test.
# Table of Contents
* [NAME](#name)
* [SYNOPSIS](#synopsis)
* [DESCRIPTION](#description)
* [AUTHENTICATION AND CREDENTIALS](#authentication-and-credentials)
* [CHECKSUMS](#checksums)
* [Uploads](#uploads)
* [Downloads](#downloads)
* [WORKING WITH BUCKETS AND OBJECTS](#working-with-buckets-and-objects)
* [Creating and Representing Buckets](#creating-and-representing-buckets)
* [LISTING OBJECTS](#listing-objects)
* [Choosing a Listing API](#choosing-a-listing-api)
* [Prefixes and Delimiters](#prefixes-and-delimiters)
* [Pagination](#pagination)
* [Listing Results](#listing-results)
* [DIRECTORY BUCKETS](#directory-buckets)
* [ERROR HANDLING](#error-handling)
* [METHODS AND SUBROUTINES](#methods-and-subroutines)
* [CONSTRUCTOR](#constructor)
* [new](#new)
* [ACCESSORS](#accessors)
* [buffer\_size](#buffer\size)
* [cache\_signer](#cache\signer)
* [checksum\_algorithm](#checksum\algorithm)
* [checksum\_types](#checksum\types)
* [credentials](#credentials)
* [dns\_bucket\_names](#dns\bucket\names)
* [err](#err)
* [error](#error)
* [errstr](#errstr)
* [host](#host)
* [last\_request](#last\request)
* [last\_response](#last\response)
* [logger](#logger)
* [retry](#retry)
* [secure](#secure)
* [timeout](#timeout)
* [verify\_checksums](#verify\checksums)
* [AUTHENTICATION AND CONFIGURATION METHODS](#authentication-and-configuration-methods)
* [get\_credentials](#get\credentials)
* [get\_default\_region](#get\default\region)
* [get\_logger](#get\logger)
* [level](#level)
* [region](#region)
* [signer](#signer)
* [BUCKET MANAGEMENT](#bucket-management)
* [add\_bucket](#add\bucket)
* [bucket](#bucket)
* [buckets](#buckets)
* [bucketv2](#bucketv2)
# NAME
Amazon::S3 - A Perl client library for working with and managing
Amazon S3 buckets and objects.
# SYNOPSIS
use Amazon::S3;
my $s3 = Amazon::S3->new(
{ credentials => $credentials,
region => 'us-east-1',
}
);
my $bucket = $s3->bucket('example-bucket');
$bucket->add_key(
'testing.txt',
'T',
{ content_type => 'text/plain',
# DESCRIPTION
`Amazon::S3` provides a Perl interface to Amazon Simple Storage
Service (S3).
The distribution separates account-level S3 operations from
bucket and object operations.
`Amazon::S3` represents the S3 client and AWS account context. It
manages credentials, request signing, regions, service endpoints,
bucket creation and discovery, and account-level listing operations.
[Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) represents an individual bucket. Object
operations such as uploads, downloads, deletes, ACLs, multipart
uploads, and bucket-scoped listing operations are provided primarily
through that class.
[Amazon::S3::BucketV2](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucketV2) is a subclass of [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) that
provides a more general interface to S3 APIs. It accepts API
parameters, headers, URI parameters, and request payload structures
distributions have diverged substantially. Current versions should
not be considered interchangeable.
Version 2.1.0 adds modern S3 checksum support while preserving the
existing `Amazon::S3` and [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) interfaces. Managed
uploads use CRC64NVME by default, and downloads can request and verify
checksum metadata returned by S3.
# AUTHENTICATION AND CREDENTIALS
`Amazon::S3` supports either explicit AWS access keys or a credentials
object.
Explicit credentials may be supplied to the constructor:
my $s3 = Amazon::S3->new(
{ aws_access_key_id => $aws_access_key_id,
aws_secret_access_key => $aws_secret_access_key,
token => $session_token,
}
);
The `token` option is required only when temporary AWS credentials
include a session token.
For applications that obtain credentials dynamically, a credentials
object is preferred:
my $s3 = Amazon::S3->new({ credentials => $credentials } );
The credentials object must provide:
get_aws_access_key_id()
get_aws_secret_access_key()
get_token()
[Amazon::Credentials](https://metacpan.org/pod/Amazon%3A%3ACredentials) is one implementation of this interface.
Using a credentials object allows the credential provider to manage
credential discovery and refresh independently of `Amazon::S3`.
`Amazon::S3` uses Signature Version 4 for AWS API requests.
The signer normally uses the credentials associated with the
`Amazon::S3` object. A signer may also be supplied to the constructor
using the `signer` option.
By default, signers are not cached. Setting `cache_signer` to true
causes `Amazon::S3` to retain and reuse the signer object.
Applications should avoid dumping `Amazon::S3`, credential, or signer
objects to logs. These objects participate in request authentication
and may contain or provide access to sensitive authentication
material.
# DIRECTORY BUCKETS
`Amazon::S3` provides limited support for Amazon S3 directory
buckets.
Directory buckets use the S3 Express One Zone storage class.
`Amazon::S3` can currently create and list directory buckets.
Object access within directory buckets is not yet supported because
that requires creating an S3 Express session and using the resulting
temporary credentials when signing requests to the directory bucket's
Zonal endpoint.
A directory bucket can be created by supplying an availability zone
to `add_bucket()`:
my $bucket = $s3->add_bucket(
{ bucket => $bucket_name,
availability_zone => 'use1-az5',
}
);
my $response = $s3->buckets;
if (!$response) {
die $s3->err . ': ' . $s3->errstr;
}
Applications that prefer request failures to throw exceptions can
enable `raise_error` when constructing the client:
my $s3 = Amazon::S3->new(
credentials => $credentials,
raise\_error => 1,
);
With `raise_error` enabled, S3 request failures that would normally
return a failure value instead throw an exception. The error state is
still recorded in `err()`, `errstr()`, and `error()` before the
exception is raised.
When available, the exception includes the HTTP status together with
the S3 error code and message.
my $s3 = Amazon::S3->new(\%options);
Creates and returns a new `Amazon::S3` client object.
The constructor accepts either a list of key/value pairs or a hash
reference.
At least one of the following credential configurations is required:
- A `credentials` object that provides `get_aws_access_key_id()`,
`get_aws_secret_access_key()`, and `get_token()`.
- Both `aws_access_key_id` and `aws_secret_access_key`.
The following options are supported:
- aws\_access\_key\_id
AWS access key ID.
This option is required when a `credentials` object is not supplied.
When explicit credentials are supplied, `Amazon::S3` stores them
internally for use when signing requests. Applications should avoid
dumping the client object to logs.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
- aws\_secret\_access\_key
AWS secret access key.
This option is required when a `credentials` object is not supplied.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
- buffer\_size
Default buffer size, in bytes, used by operations that stream object
data.
The default is 4096.
- cache\_signer
When true, retain and reuse the Signature Version 4 signer.
When false, construct a signer when one is needed.
The default is false.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
- checksum\_algorithm
Checksum algorithm used when `Amazon::S3` supplies a checksum with an
upload.
The default is `crc64nvme`.
Recognized S3 checksum algorithm names are validated by the
constructor. Local checksum implementations provided by this release
are described in ["CHECKSUMS"](#checksums).
- credentials
Credentials provider object.
The object must provide:
get_aws_access_key_id()
get_aws_secret_access_key()
get_token()
[Amazon::Credentials](https://metacpan.org/pod/Amazon%3A%3ACredentials) is one implementation of this interface.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
- debug
Compatibility option that sets the default logger level to `debug`.
Applications should normally use `level` instead.
This option affects the internally created logger only.
- dns\_bucket\_names
When true, use HTTPS when communicating with the service.
The default is true.
- signer
Optional Signature Version 4 signer object.
When supplied, this signer is used instead of constructing one from
the configured credentials.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
- timeout
HTTP request timeout in seconds.
The default is 30.
- token
Optional AWS session token used with temporary credentials.
- verify\_checksums
Controls checksum verification when downloading objects.
The default is true.
See ["CHECKSUMS"](#checksums).
If neither a credentials provider nor both explicit access key values
are supplied, the constructor throws an exception.
An invalid `checksum_algorithm` also causes the constructor to throw
an exception.
On success, returns the new `Amazon::S3` object.
## ACCESSORS
### buffer\_size
### cache\_signer
my $cache_signer = $s3->cache_signer;
$s3->cache_signer($boolean);
Gets or sets whether a generated request signer is retained for reuse.
The constructor default is false.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
### checksum\_algorithm
my $algorithm = $s3->checksum_algorithm;
$s3->checksum_algorithm('sha256');
Gets or sets the checksum algorithm selected for uploads.
The constructor default is `crc64nvme`.
Returns the checksum implementations initialized for this client.
The value is a hash reference keyed by algorithm name.
This accessor is intended for introspection. The internal checksum
implementation entries are not a public plugin interface in this
release.
See ["CHECKSUMS"](#checksums).
### credentials
my $credentials = $s3->credentials;
$s3->credentials($credentials);
Gets or sets the credentials provider object.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
### dns\_bucket\_names
my $enabled = $s3->dns_bucket_names;
$s3->dns_bucket_names($boolean);
Gets or sets whether virtual-hosted-style bucket addressing is used
when possible.
Gets or sets whether supported checksums returned for downloaded
objects are verified.
The constructor default is true.
See ["CHECKSUMS"](#checksums).
## AUTHENTICATION AND CONFIGURATION METHODS
### get\_credentials
my ( $access_key_id, $secret_access_key, $token )
= $s3->get_credentials;
Returns the credentials used by the client.
When a `credentials` provider is configured, this method obtains the
three values from that provider.
Otherwise it returns the credentials stored by the `Amazon::S3`
object.
The return values, in order, are:
1. AWS access key ID.
2. AWS secret access key.
3. Session token, or `undef` when no token is configured.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
### get\_default\_region
my $region = $s3->get_default_region;
Attempts to determine the default AWS region.
The method checks, in order:
1. `AWS_REGION`.
The constructor default is `us-east-1`.
### signer
my $signer = $s3->signer;
Returns the Signature Version 4 signer used for requests.
If a signer was supplied to the constructor, that signer is returned.
Otherwise a signer is constructed from the current credentials,
region, and session token.
When `cache_signer` is true, a generated signer is retained and
reused. When it is false, a signer can be generated as needed.
This method does not accept a signer argument. Supply a custom signer
using the `signer` constructor option.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
## BUCKET MANAGEMENT
### add\_bucket
my $bucket = $s3->add_bucket(\%configuration);
Creates a bucket.
The argument is a hash reference containing the bucket configuration.
Bug reports and feature requests should be submitted through the
project issue tracker.
When reporting a problem, include the `Amazon::S3` version, Perl
version, operating system, and enough information to reproduce the
behavior.
For request or protocol problems, debug or trace logging may also be
useful. Review logs before sharing them to ensure that they do not
contain credentials, authorization information, or sensitive object
data.
# REPOSITORY
The source repository, issue tracker, and development history are
available at:
[https://github.com/rlauer6/Amazon-S3](https://github.com/rlauer6/Amazon-S3)
# AUTHOR
lib/Amazon/S3.pm view on Meta::CPAN
__PACKAGE__->mk_accessors(
qw(
aws_access_key_id
aws_secret_access_key
token
buffer_size
cache_signer
checksum_types
checksum_algorithm
credentials
dns_bucket_names
digest
endpoint_url
err
errstr
error
express
host
last_request
last_response
lib/Amazon/S3.pm view on Meta::CPAN
if ( $safe_options{aws_secret_access_key} ) {
$safe_options{aws_secret_access_key} = '****';
$safe_options{aws_access_key_id} = '****';
}
return Dumper( [ options => \%safe_options ] );
},
);
if ( !$self->credentials ) {
croak 'No aws_access_key_id'
if !$self->aws_access_key_id;
croak 'No aws_secret_access_key'
if !$self->aws_secret_access_key;
# encrypt credentials
$self->aws_access_key_id( _encrypt( $self->aws_access_key_id ) );
$self->aws_secret_access_key( _encrypt( $self->aws_secret_access_key ) );
$self->token( _encrypt( $self->token ) );
}
my $ua;
if ( $self->retry ) {
$ua = LWP::UserAgent::Determined->new(
keep_alive => $KEEP_ALIVE_CACHESIZE,
lib/Amazon/S3.pm view on Meta::CPAN
response => $r,
],
);
},
);
return $r;
}
########################################################################
sub get_credentials {
########################################################################
my ($self) = @_;
my $aws_access_key_id;
my $aws_secret_access_key;
my $token;
if ( $self->credentials ) {
$aws_access_key_id = $self->credentials->get_aws_access_key_id;
$aws_secret_access_key = $self->credentials->get_aws_secret_access_key;
$token = $self->credentials->get_token;
}
else {
$aws_access_key_id = $self->aws_access_key_id;
$aws_secret_access_key = $self->aws_secret_access_key;
$token = $self->token;
}
return ( $aws_access_key_id, $aws_secret_access_key, $token );
}
lib/Amazon/S3.pm view on Meta::CPAN
}
########################################################################
sub signer {
########################################################################
my ($self) = @_;
return $self->_signer
if $self->_signer;
my $creds = $self->credentials ? $self->credentials : $self;
my $express = $self->express;
my $signer = Amazon::S3::Signature::V4->new(
{ access_key_id => $creds->get_aws_access_key_id,
secret => $creds->get_aws_secret_access_key,
region => $self->region || $self->get_default_region,
service => $express ? 's3express' : 's3',
security_token => $creds->get_token,
},
);
lib/Amazon/S3.pm view on Meta::CPAN
return $TRUE;
}
# Deprecated - this adds a header for the old V2 auth signatures
########################################################################
sub _add_auth_header { ## no critic (ProhibitUnusedPrivateSubroutines)
########################################################################
my ( $self, $headers, $method, $path ) = @_;
my ( $aws_access_key_id, $aws_secret_access_key, $token ) = $self->get_credentials;
if ( not $headers->header('Date') ) {
$headers->header( Date => time2str(time) );
}
if ($token) {
$headers->header( $AMAZON_HEADER_PREFIX . 'security-token' => $token );
}
my $canonical_string = $self->_canonical_string( $method, $path, $headers );
lib/Amazon/S3.pm view on Meta::CPAN
=head1 NAME
Amazon::S3 - A Perl client library for working with and managing
Amazon S3 buckets and objects.
=head1 SYNOPSIS
use Amazon::S3;
my $s3 = Amazon::S3->new(
{ credentials => $credentials,
region => 'us-east-1',
}
);
my $bucket = $s3->bucket('example-bucket');
$bucket->add_key(
'testing.txt',
'T',
{ content_type => 'text/plain',
lib/Amazon/S3.pm view on Meta::CPAN
=head1 DESCRIPTION
C<Amazon::S3> provides a Perl interface to Amazon Simple Storage
Service (S3).
The distribution separates account-level S3 operations from
bucket and object operations.
C<Amazon::S3> represents the S3 client and AWS account context. It
manages credentials, request signing, regions, service endpoints,
bucket creation and discovery, and account-level listing operations.
L<Amazon::S3::Bucket> represents an individual bucket. Object
operations such as uploads, downloads, deletes, ACLs, multipart
uploads, and bucket-scoped listing operations are provided primarily
through that class.
L<Amazon::S3::BucketV2> is a subclass of L<Amazon::S3::Bucket> that
provides a more general interface to S3 APIs. It accepts API
parameters, headers, URI parameters, and request payload structures
lib/Amazon/S3.pm view on Meta::CPAN
distributions have diverged substantially. Current versions should
not be considered interchangeable.
Version 2.1.0 adds modern S3 checksum support while preserving the
existing C<Amazon::S3> and L<Amazon::S3::Bucket> interfaces. Managed
uploads use CRC64NVME by default, and downloads can request and verify
checksum metadata returned by S3.
=head1 AUTHENTICATION AND CREDENTIALS
C<Amazon::S3> supports either explicit AWS access keys or a credentials
object.
Explicit credentials may be supplied to the constructor:
my $s3 = Amazon::S3->new(
{ aws_access_key_id => $aws_access_key_id,
aws_secret_access_key => $aws_secret_access_key,
token => $session_token,
}
);
The C<token> option is required only when temporary AWS credentials
include a session token.
For applications that obtain credentials dynamically, a credentials
object is preferred:
my $s3 = Amazon::S3->new({ credentials => $credentials } );
The credentials object must provide:
get_aws_access_key_id()
get_aws_secret_access_key()
get_token()
L<Amazon::Credentials> is one implementation of this interface.
Using a credentials object allows the credential provider to manage
credential discovery and refresh independently of C<Amazon::S3>.
C<Amazon::S3> uses Signature Version 4 for AWS API requests.
The signer normally uses the credentials associated with the
C<Amazon::S3> object. A signer may also be supplied to the constructor
using the C<signer> option.
By default, signers are not cached. Setting C<cache_signer> to true
causes C<Amazon::S3> to retain and reuse the signer object.
Applications should avoid dumping C<Amazon::S3>, credential, or signer
objects to logs. These objects participate in request authentication
and may contain or provide access to sensitive authentication
material.
lib/Amazon/S3.pm view on Meta::CPAN
=head1 DIRECTORY BUCKETS
C<Amazon::S3> provides limited support for Amazon S3 directory
buckets.
Directory buckets use the S3 Express One Zone storage class.
C<Amazon::S3> can currently create and list directory buckets.
Object access within directory buckets is not yet supported because
that requires creating an S3 Express session and using the resulting
temporary credentials when signing requests to the directory bucket's
Zonal endpoint.
A directory bucket can be created by supplying an availability zone
to C<add_bucket()>:
my $bucket = $s3->add_bucket(
{ bucket => $bucket_name,
availability_zone => 'use1-az5',
}
);
lib/Amazon/S3.pm view on Meta::CPAN
my $response = $s3->buckets;
if (!$response) {
die $s3->err . ': ' . $s3->errstr;
}
Applications that prefer request failures to throw exceptions can
enable C<raise_error> when constructing the client:
my $s3 = Amazon::S3->new(
credentials => $credentials,
raise_error => 1,
);
With C<raise_error> enabled, S3 request failures that would normally
return a failure value instead throw an exception. The error state is
still recorded in C<err()>, C<errstr()>, and C<error()> before the
exception is raised.
When available, the exception includes the HTTP status together with
the S3 error code and message.
lib/Amazon/S3.pm view on Meta::CPAN
The constructor accepts either a list of key/value pairs or a hash
reference.
At least one of the following credential configurations is required:
=over 4
=item *
A C<credentials> object that provides C<get_aws_access_key_id()>,
C<get_aws_secret_access_key()>, and C<get_token()>.
=item *
Both C<aws_access_key_id> and C<aws_secret_access_key>.
=back
The following options are supported:
=over 4
=item aws_access_key_id
AWS access key ID.
This option is required when a C<credentials> object is not supplied.
When explicit credentials are supplied, C<Amazon::S3> stores them
internally for use when signing requests. Applications should avoid
dumping the client object to logs.
See L</AUTHENTICATION AND CREDENTIALS>.
=item aws_secret_access_key
AWS secret access key.
This option is required when a C<credentials> object is not supplied.
See L</AUTHENTICATION AND CREDENTIALS>.
=item buffer_size
Default buffer size, in bytes, used by operations that stream object
data.
The default is 4096.
lib/Amazon/S3.pm view on Meta::CPAN
Checksum algorithm used when C<Amazon::S3> supplies a checksum with an
upload.
The default is C<crc64nvme>.
Recognized S3 checksum algorithm names are validated by the
constructor. Local checksum implementations provided by this release
are described in L</CHECKSUMS>.
=item credentials
Credentials provider object.
The object must provide:
get_aws_access_key_id()
get_aws_secret_access_key()
get_token()
L<Amazon::Credentials> is one implementation of this interface.
lib/Amazon/S3.pm view on Meta::CPAN
When true, use HTTPS when communicating with the service.
The default is true.
=item signer
Optional Signature Version 4 signer object.
When supplied, this signer is used instead of constructing one from
the configured credentials.
See L</AUTHENTICATION AND CREDENTIALS>.
=item timeout
HTTP request timeout in seconds.
The default is 30.
=item token
Optional AWS session token used with temporary credentials.
=item verify_checksums
Controls checksum verification when downloading objects.
The default is true.
See L</CHECKSUMS>.
=back
If neither a credentials provider nor both explicit access key values
are supplied, the constructor throws an exception.
An invalid C<checksum_algorithm> also causes the constructor to throw
an exception.
On success, returns the new C<Amazon::S3> object.
=head2 ACCESSORS
=head3 buffer_size
lib/Amazon/S3.pm view on Meta::CPAN
Returns the checksum implementations initialized for this client.
The value is a hash reference keyed by algorithm name.
This accessor is intended for introspection. The internal checksum
implementation entries are not a public plugin interface in this
release.
See L</CHECKSUMS>.
=head3 credentials
my $credentials = $s3->credentials;
$s3->credentials($credentials);
Gets or sets the credentials provider object.
See L</AUTHENTICATION AND CREDENTIALS>.
=head3 dns_bucket_names
my $enabled = $s3->dns_bucket_names;
$s3->dns_bucket_names($boolean);
Gets or sets whether virtual-hosted-style bucket addressing is used
lib/Amazon/S3.pm view on Meta::CPAN
Gets or sets whether supported checksums returned for downloaded
objects are verified.
The constructor default is true.
See L</CHECKSUMS>.
=head2 AUTHENTICATION AND CONFIGURATION METHODS
=head3 get_credentials
my ( $access_key_id, $secret_access_key, $token )
= $s3->get_credentials;
Returns the credentials used by the client.
When a C<credentials> provider is configured, this method obtains the
three values from that provider.
Otherwise it returns the credentials stored by the C<Amazon::S3>
object.
The return values, in order, are:
=over 4
=item 1.
AWS access key ID.
lib/Amazon/S3.pm view on Meta::CPAN
The constructor default is C<us-east-1>.
=head3 signer
my $signer = $s3->signer;
Returns the Signature Version 4 signer used for requests.
If a signer was supplied to the constructor, that signer is returned.
Otherwise a signer is constructed from the current credentials,
region, and session token.
When C<cache_signer> is true, a generated signer is retained and
reused. When it is false, a signer can be generated as needed.
This method does not accept a signer argument. Supply a custom signer
using the C<signer> constructor option.
See L</AUTHENTICATION AND CREDENTIALS>.
lib/Amazon/S3.pm view on Meta::CPAN
Bug reports and feature requests should be submitted through the
project issue tracker.
When reporting a problem, include the C<Amazon::S3> version, Perl
version, operating system, and enough information to reproduce the
behavior.
For request or protocol problems, debug or trace logging may also be
useful. Review logs before sharing them to ensure that they do not
contain credentials, authorization information, or sensitive object
data.
=head1 REPOSITORY
The source repository, issue tracker, and development history are
available at:
L<https://github.com/rlauer6/Amazon-S3>
=head1 AUTHOR
lib/Amazon/S3/Bucket.pm view on Meta::CPAN
=head1 NAME
Amazon::S3::Bucket - An Amazon S3 bucket and object interface
=head1 SYNOPSIS
use Amazon::S3;
my $s3 = Amazon::S3->new(
{ credentials => $credentials,
region => 'us-east-1',
}
);
my $bucket = $s3->bucket('example-bucket');
$bucket->add_key(
'example.txt',
'hello world',
{ content_type => 'text/plain',
lib/Amazon/S3/Bucket.pm view on Meta::CPAN
=head1 DESCRIPTION
C<Amazon::S3::Bucket> represents an Amazon S3 bucket and provides
bucket-scoped object operations.
Instances are normally created by L<Amazon::S3/bucket> or
L<Amazon::S3/bucketv2> rather than by calling C<new()> directly.
This document is primarily a method reference. For broader discussion
of credentials, checksums, object listing, multipart uploads, error
handling, and directory buckets, see L<Amazon::S3>.
=head1 METHODS AND SUBROUTINES
=head2 CONSTRUCTOR
=head3 new
my $bucket = Amazon::S3::Bucket->new(%options);
lib/Amazon/S3/BucketV2.pm view on Meta::CPAN
=head1 NAME
Amazon::S3::BucketV2 - Interface to additional Amazon S3 bucket operations
=head1 SYNOPSIS
use Amazon::S3;
my $s3 = Amazon::S3->new(
{ credentials => $credentials,
region => 'us-east-1',
}
);
my $bucket = $s3->bucketv2(
{ bucket => 'example-bucket',
}
);
my $result = $bucket->GetBucketVersioning();
lib/Amazon/S3/CLI.pm view on Meta::CPAN
$self->set_ascii_table(
eval {
require Text::ASCIITable;
1;
}
);
die "ERROR: --format must be one of json, text or table\n"
if none { $self->get_format eq $_ } qw(json text table);
my %credentials;
my $has_amazon_credentials = eval {
require Amazon::Credentials;
1;
};
if ($has_amazon_credentials) {
$credentials{credentials} = Amazon::Credentials->new(
{ $self->get_profile
? ( profile => $self->get_profile )
: (),
}
);
}
else {
die "ERROR: --profile requires Amazon::Credentials\n"
if $self->get_profile;
$credentials{aws_access_key_id} = $ENV{AWS_ACCESS_KEY_ID};
$credentials{aws_secret_access_key} = $ENV{AWS_SECRET_ACCESS_KEY};
$credentials{token} = $ENV{AWS_SESSION_TOKEN};
}
my $s3 = Amazon::S3->new(
{ %credentials,
debug => $ENV{DEBUG},
raise_error => $TRUE,
region => $self->get_region,
logger => $self->get_logger,
dns_bucket_names => $self->get_dns_bucket_names // $FALSE,
%endpoint,
defined $self->get_secure ? ( secure => $self->get_secure ) : (),
}
);
lib/Amazon/S3/CLI.pm view on Meta::CPAN
Lists all objects in a bucket using the ListObjectsV2 API.
A prefix may alternatively be supplied with C<--prefix>.
C<list-keys> is an alias for C<list-bucket-keys>.
=item B<list-buckets>
amzn-s3-cli list-buckets
Lists general-purpose S3 buckets available to the current credentials.
=item B<list-directory-buckets>
amzn-s3-cli list-directory-buckets
Lists S3 Express One Zone directory buckets.
=item B<list-object-versions>
amzn-s3-cli list-object-versions bucket-name [prefix]
lib/Amazon/S3/CLI.pm view on Meta::CPAN
=back
=head1 AUTHENTICATION
If L<Amazon::Credentials> is installed, C<amzn-s3-cli> uses it for
credential discovery and supports named profiles:
amzn-s3-cli list-buckets --profile sandbox
Without L<Amazon::Credentials>, credentials are read from:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_SESSION_TOKEN
=head1 S3-COMPATIBLE SERVICES
An alternate S3 endpoint may be selected with C<--endpoint-url>:
amzn-s3-cli list-buckets --endpoint-url http://localhost:4566 --profile localstack
share/README.md view on Meta::CPAN
# Table of Contents
* [NAME](#name)
* [SYNOPSIS](#synopsis)
* [DESCRIPTION](#description)
* [AUTHENTICATION AND CREDENTIALS](#authentication-and-credentials)
* [CHECKSUMS](#checksums)
* [Uploads](#uploads)
* [Downloads](#downloads)
* [WORKING WITH BUCKETS AND OBJECTS](#working-with-buckets-and-objects)
* [Creating and Representing Buckets](#creating-and-representing-buckets)
* [LISTING OBJECTS](#listing-objects)
* [Choosing a Listing API](#choosing-a-listing-api)
* [Prefixes and Delimiters](#prefixes-and-delimiters)
* [Pagination](#pagination)
* [Listing Results](#listing-results)
share/README.md view on Meta::CPAN
* [DIRECTORY BUCKETS](#directory-buckets)
* [ERROR HANDLING](#error-handling)
* [METHODS AND SUBROUTINES](#methods-and-subroutines)
* [CONSTRUCTOR](#constructor)
* [new](#new)
* [ACCESSORS](#accessors)
* [buffer\_size](#buffer\size)
* [cache\_signer](#cache\signer)
* [checksum\_algorithm](#checksum\algorithm)
* [checksum\_types](#checksum\types)
* [credentials](#credentials)
* [dns\_bucket\_names](#dns\bucket\names)
* [err](#err)
* [error](#error)
* [errstr](#errstr)
* [host](#host)
* [last\_request](#last\request)
* [last\_response](#last\response)
* [logger](#logger)
* [retry](#retry)
* [secure](#secure)
* [timeout](#timeout)
* [verify\_checksums](#verify\checksums)
* [AUTHENTICATION AND CONFIGURATION METHODS](#authentication-and-configuration-methods)
* [get\_credentials](#get\credentials)
* [get\_default\_region](#get\default\region)
* [get\_logger](#get\logger)
* [level](#level)
* [region](#region)
* [signer](#signer)
* [BUCKET MANAGEMENT](#bucket-management)
* [add\_bucket](#add\bucket)
* [bucket](#bucket)
* [buckets](#buckets)
* [bucketv2](#bucketv2)
share/README.md view on Meta::CPAN
# NAME
Amazon::S3 - A Perl client library for working with and managing
Amazon S3 buckets and objects.
# SYNOPSIS
use Amazon::S3;
my $s3 = Amazon::S3->new(
{ credentials => $credentials,
region => 'us-east-1',
}
);
my $bucket = $s3->bucket('example-bucket');
$bucket->add_key(
'testing.txt',
'T',
{ content_type => 'text/plain',
share/README.md view on Meta::CPAN
# DESCRIPTION
`Amazon::S3` provides a Perl interface to Amazon Simple Storage
Service (S3).
The distribution separates account-level S3 operations from
bucket and object operations.
`Amazon::S3` represents the S3 client and AWS account context. It
manages credentials, request signing, regions, service endpoints,
bucket creation and discovery, and account-level listing operations.
[Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) represents an individual bucket. Object
operations such as uploads, downloads, deletes, ACLs, multipart
uploads, and bucket-scoped listing operations are provided primarily
through that class.
[Amazon::S3::BucketV2](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucketV2) is a subclass of [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) that
provides a more general interface to S3 APIs. It accepts API
parameters, headers, URI parameters, and request payload structures
share/README.md view on Meta::CPAN
distributions have diverged substantially. Current versions should
not be considered interchangeable.
Version 2.1.0 adds modern S3 checksum support while preserving the
existing `Amazon::S3` and [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) interfaces. Managed
uploads use CRC64NVME by default, and downloads can request and verify
checksum metadata returned by S3.
# AUTHENTICATION AND CREDENTIALS
`Amazon::S3` supports either explicit AWS access keys or a credentials
object.
Explicit credentials may be supplied to the constructor:
my $s3 = Amazon::S3->new(
{ aws_access_key_id => $aws_access_key_id,
aws_secret_access_key => $aws_secret_access_key,
token => $session_token,
}
);
The `token` option is required only when temporary AWS credentials
include a session token.
For applications that obtain credentials dynamically, a credentials
object is preferred:
my $s3 = Amazon::S3->new({ credentials => $credentials } );
The credentials object must provide:
get_aws_access_key_id()
get_aws_secret_access_key()
get_token()
[Amazon::Credentials](https://metacpan.org/pod/Amazon%3A%3ACredentials) is one implementation of this interface.
Using a credentials object allows the credential provider to manage
credential discovery and refresh independently of `Amazon::S3`.
`Amazon::S3` uses Signature Version 4 for AWS API requests.
The signer normally uses the credentials associated with the
`Amazon::S3` object. A signer may also be supplied to the constructor
using the `signer` option.
By default, signers are not cached. Setting `cache_signer` to true
causes `Amazon::S3` to retain and reuse the signer object.
Applications should avoid dumping `Amazon::S3`, credential, or signer
objects to logs. These objects participate in request authentication
and may contain or provide access to sensitive authentication
material.
share/README.md view on Meta::CPAN
# DIRECTORY BUCKETS
`Amazon::S3` provides limited support for Amazon S3 directory
buckets.
Directory buckets use the S3 Express One Zone storage class.
`Amazon::S3` can currently create and list directory buckets.
Object access within directory buckets is not yet supported because
that requires creating an S3 Express session and using the resulting
temporary credentials when signing requests to the directory bucket's
Zonal endpoint.
A directory bucket can be created by supplying an availability zone
to `add_bucket()`:
my $bucket = $s3->add_bucket(
{ bucket => $bucket_name,
availability_zone => 'use1-az5',
}
);
share/README.md view on Meta::CPAN
my $response = $s3->buckets;
if (!$response) {
die $s3->err . ': ' . $s3->errstr;
}
Applications that prefer request failures to throw exceptions can
enable `raise_error` when constructing the client:
my $s3 = Amazon::S3->new(
credentials => $credentials,
raise\_error => 1,
);
With `raise_error` enabled, S3 request failures that would normally
return a failure value instead throw an exception. The error state is
still recorded in `err()`, `errstr()`, and `error()` before the
exception is raised.
When available, the exception includes the HTTP status together with
the S3 error code and message.
share/README.md view on Meta::CPAN
my $s3 = Amazon::S3->new(\%options);
Creates and returns a new `Amazon::S3` client object.
The constructor accepts either a list of key/value pairs or a hash
reference.
At least one of the following credential configurations is required:
- A `credentials` object that provides `get_aws_access_key_id()`,
`get_aws_secret_access_key()`, and `get_token()`.
- Both `aws_access_key_id` and `aws_secret_access_key`.
The following options are supported:
- aws\_access\_key\_id
AWS access key ID.
This option is required when a `credentials` object is not supplied.
When explicit credentials are supplied, `Amazon::S3` stores them
internally for use when signing requests. Applications should avoid
dumping the client object to logs.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
- aws\_secret\_access\_key
AWS secret access key.
This option is required when a `credentials` object is not supplied.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
- buffer\_size
Default buffer size, in bytes, used by operations that stream object
data.
The default is 4096.
- cache\_signer
When true, retain and reuse the Signature Version 4 signer.
When false, construct a signer when one is needed.
The default is false.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
- checksum\_algorithm
Checksum algorithm used when `Amazon::S3` supplies a checksum with an
upload.
The default is `crc64nvme`.
Recognized S3 checksum algorithm names are validated by the
constructor. Local checksum implementations provided by this release
are described in ["CHECKSUMS"](#checksums).
- credentials
Credentials provider object.
The object must provide:
get_aws_access_key_id()
get_aws_secret_access_key()
get_token()
[Amazon::Credentials](https://metacpan.org/pod/Amazon%3A%3ACredentials) is one implementation of this interface.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
- debug
Compatibility option that sets the default logger level to `debug`.
Applications should normally use `level` instead.
This option affects the internally created logger only.
- dns\_bucket\_names
share/README.md view on Meta::CPAN
When true, use HTTPS when communicating with the service.
The default is true.
- signer
Optional Signature Version 4 signer object.
When supplied, this signer is used instead of constructing one from
the configured credentials.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
- timeout
HTTP request timeout in seconds.
The default is 30.
- token
Optional AWS session token used with temporary credentials.
- verify\_checksums
Controls checksum verification when downloading objects.
The default is true.
See ["CHECKSUMS"](#checksums).
If neither a credentials provider nor both explicit access key values
are supplied, the constructor throws an exception.
An invalid `checksum_algorithm` also causes the constructor to throw
an exception.
On success, returns the new `Amazon::S3` object.
## ACCESSORS
### buffer\_size
share/README.md view on Meta::CPAN
### cache\_signer
my $cache_signer = $s3->cache_signer;
$s3->cache_signer($boolean);
Gets or sets whether a generated request signer is retained for reuse.
The constructor default is false.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
### checksum\_algorithm
my $algorithm = $s3->checksum_algorithm;
$s3->checksum_algorithm('sha256');
Gets or sets the checksum algorithm selected for uploads.
The constructor default is `crc64nvme`.
share/README.md view on Meta::CPAN
Returns the checksum implementations initialized for this client.
The value is a hash reference keyed by algorithm name.
This accessor is intended for introspection. The internal checksum
implementation entries are not a public plugin interface in this
release.
See ["CHECKSUMS"](#checksums).
### credentials
my $credentials = $s3->credentials;
$s3->credentials($credentials);
Gets or sets the credentials provider object.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
### dns\_bucket\_names
my $enabled = $s3->dns_bucket_names;
$s3->dns_bucket_names($boolean);
Gets or sets whether virtual-hosted-style bucket addressing is used
when possible.
share/README.md view on Meta::CPAN
Gets or sets whether supported checksums returned for downloaded
objects are verified.
The constructor default is true.
See ["CHECKSUMS"](#checksums).
## AUTHENTICATION AND CONFIGURATION METHODS
### get\_credentials
my ( $access_key_id, $secret_access_key, $token )
= $s3->get_credentials;
Returns the credentials used by the client.
When a `credentials` provider is configured, this method obtains the
three values from that provider.
Otherwise it returns the credentials stored by the `Amazon::S3`
object.
The return values, in order, are:
1. AWS access key ID.
2. AWS secret access key.
3. Session token, or `undef` when no token is configured.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
### get\_default\_region
my $region = $s3->get_default_region;
Attempts to determine the default AWS region.
The method checks, in order:
1. `AWS_REGION`.
share/README.md view on Meta::CPAN
The constructor default is `us-east-1`.
### signer
my $signer = $s3->signer;
Returns the Signature Version 4 signer used for requests.
If a signer was supplied to the constructor, that signer is returned.
Otherwise a signer is constructed from the current credentials,
region, and session token.
When `cache_signer` is true, a generated signer is retained and
reused. When it is false, a signer can be generated as needed.
This method does not accept a signer argument. Supply a custom signer
using the `signer` constructor option.
See ["AUTHENTICATION AND CREDENTIALS"](#authentication-and-credentials).
## BUCKET MANAGEMENT
### add\_bucket
my $bucket = $s3->add_bucket(\%configuration);
Creates a bucket.
The argument is a hash reference containing the bucket configuration.
share/README.md view on Meta::CPAN
Bug reports and feature requests should be submitted through the
project issue tracker.
When reporting a problem, include the `Amazon::S3` version, Perl
version, operating system, and enough information to reproduce the
behavior.
For request or protocol problems, debug or trace logging may also be
useful. Review logs before sharing them to ensure that they do not
contain credentials, authorization information, or sensitive object
data.
# REPOSITORY
The source repository, issue tracker, and development history are
available at:
[https://github.com/rlauer6/Amazon-S3](https://github.com/rlauer6/Amazon-S3)
# AUTHOR
t/lib/S3TestUtils.pm view on Meta::CPAN
die "ERROR: usage: get_s3_service(host)\n"
if !$host;
my $s3 = eval {
if ( $ENV{AMAZON_S3_CREDENTIALS} ) {
require Amazon::Credentials;
return Amazon::S3->new(
{ credentials => Amazon::Credentials->new,
host => $host,
secure => is_aws(),
dns_bucket_names => $ENV{AMAZON_S3_DNS_BUCKET_NAMES},
level => $ENV{DEBUG} ? 'trace' : 'error',
raise_error => $raise_error,
}
);
}
else {
( run in 2.593 seconds using v1.01-cache-2.11-cpan-3f962061493 )