Amazon-S3
view release on metacpan or search on metacpan
Thu Jul 14 06:34:56 2022 Rob Lauer <rlauer6@comcast.net>>
[0.55 - use XML::LibXML]:
* VERSION: bump
* src/main/perl/lib/Amazon/S3.pm.in: use XML::LibXML, not XML::Simple
- perlcritic cleanups
- pod cleanup
(new)
- cache_signer
- encrypt credentials
(get_default_region): new
(get_aws_access_key_id): new
(get_aws_secret_access_key): new
(get_token): new
(_decrypt): new
(_encrypt): new
(signer)
- accesses _signer now
- set default region to caller's value or default
(buckets): set region to us-east-1 temporarily
- don't treat warnings as errors
- output package version
- check for rpmbuild, pod2markdown, scandeps-static.pl
- update dependencies
* src/main/perl/lib/Amazon/S3.pm.in
- new logging option
- mv'd from dist/lib/Amazon/S3.pm
- documenation updates
- refactoring, perlcritic cleanups
- use Amazon::S3::Constants
- allow passing credentials class
(_make_request): check dns_bucket_names option
(dns_bucket_names): new
(get_logger): new
(level): new
(get_credentials): new
(new)
- secure defaults to true
(region): new
(list_bucket_v2): new
(list_bucket_all_v2): new
(last_response): new
* src/main/perl/lib/Amazon/S3/Bucket.pm.in
- mv'd from dist/lib/Amazon/S3/Bucket.pm
- revert using XML::Simple
- use Amazon::S3::Constants
* autotools/ax_distcheck_hack.m4: new
* autotools/ax_rpmbuild_check.m4: new
Revision history for Perl module Amazon::S3:
0.48 Sep 17 2021
- remove leading '/' from bucket name
- modify tests for not s3.amazonaws.com hosts (e.g. minio)
0.47 Feb 4 2019
- update dependencies (XML::LibXML)
0.46 Jan 19 2019
- use temporary credentials
- error string
-
0.45 Aug 14 2009
- Applied patch to support any S3 end points (Tokuhiro Matsuno)
- Applied patches to not die when 0 or 1 buckets are return also (Tokuhiro Matsuno)
0.441 Jun 16 2008
- Added handling to get_key_filename where the key is used as the file
name if one is not provided.
0.44 Jun 08 2008
- Applied patch for fixing non-xml error parsing (Harold Sinclair)
README-TESTING.md view on Meta::CPAN
```
# Credentials for Testing
You should set the environment variables `AWS_ACCESS_KEY_ID` and
`AWS_ACCESS_SECRET_KEY` to your AWS credential values that have the
ability to create and write to buckets.
If you set environment variable `AMAZON_S3_CREDENTIAL` to any value,
the tests will use the `Amazon::Credentials` module to look for valid
credentials in your environment, your credentials files or the
instance role if you are running on an EC2.
# Using S3 Mocking Services
If you want to test *some* parts of this module but don't want to
spend a few pennies (or don't have access to AWS credentials) you can
try one of the S3 mocking services. The two of the most popular services
seem to be:
* [LocalStack](https://localstack.io)
* [minio](https://min.io)
Both of these implement a subset of the S3 API. __Note that Some tests will fail
on both services (as of the writing of this document).__ To make it
through the tests, try setting one or more of the environment
variables above which will selectively skip some test.
For more information regarding multipart uploads visit the link below.
[https://docs.aws.amazon.com/AmazonS3/latest/API/API\_CreateMultipartUpload.html](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateMultipartUpload.html)
# METHODS AND SUBROUTINES
## new
Create a new S3 client object. Takes some arguments:
- credentials (optional)
Reference to a class (like `Amazon::Credentials`) that can provide
credentials via the methods:
get_aws_access_key_id()
get_aws_secret_access_key()
get_token()
If you do not provide a credential class you must provide the keys
when you instantiate the object. See below.
_You are strongly encourage to use a class that provides getters. If
you choose to provide your credentials to this class then they will be
stored in this object. If you dump the class you will likely expose
those credentials._
- aws\_access\_key\_id
Use your Access Key ID as the value of the AWSAccessKeyId parameter
in requests you send to Amazon Web Services (when required). Your
Access Key ID identifies you as the party responsible for the
request.
- aws\_secret\_access\_key
Since your Access Key ID is not encrypted in requests to AWS, it
could be discovered and used by anyone. Services that are not free
require you to provide additional information, a request signature,
to verify that a request containing your unique Access Key ID could
only have come from you.
**DO NOT INCLUDE THIS IN SCRIPTS OR APPLICATIONS YOU
DISTRIBUTE. YOU'LL BE SORRY.**
_Consider using a credential class as described above to provide
credentials, otherwise this class will store your credentials for
signing the requests. If you dump this object to logs your credentials
could be discovered._
- token
An optional temporary token that will be inserted in the request along
with your access and secret key. A token is used in conjunction with
temporary credentials when your EC2 instance has
assumed a role and you've scraped the temporary credentials from
_http://169.254.169.254/latest/meta-data/iam/security-credentials_
- secure
Set this to a true value if you want to use SSL-encrypted connections
when connecting to S3. Starting in version 0.49, the default is true.
default: true
- timeout
- buffer\_size
The default buffer size when reading or writing files.
default: 4096
## signer
Sets or retrieves the signer object. API calls must be signed using
your AWS credentials. By default, starting with version 0.54 the
module will use [Net::Amazon::Signature::V4](https://metacpan.org/pod/Net%3A%3AAmazon%3A%3ASignature%3A%3AV4) as the signer and
instantiate a signer object in the constructor. Note however, that
signers need your credentials and they _will_ get stored by that
class, making them susceptible to inadvertant exfiltration. You have a
few options here:
- 1. Use your own signer.
You may have noticed that you can also provide your own credentials
object forcing this module to use your object for retrieving
credentials. Likewise, you can use your own signer so that this
module's signer never sees or stores those credentials.
- 2. Pass the credentials object and set `cache_signer` to a
false value.
If you pass a credentials object and set `cache_signer` to a false
value, the module will use the credentials object to retrieve
credentials and create a new signer each time an API call is made that
requires signing. This prevents your credentials from being stored
inside of the signer class.
_Note that using your own credentials object that stores your
credentials in plaintext is also going to expose your credentials when
someone dumps the class._
- 3. Pass credentials, set `cache_signer` to a false value.
Unfortunately, while this will prevent [Net::Amazon::Signature::V4](https://metacpan.org/pod/Net%3A%3AAmazon%3A%3ASignature%3A%3AV4)
from hanging on to your credentials, you credentials will be stored in
the `Amazon::S3` object.
Starting with version 0.55 of this module, if you have installed
[Crypt::CBC](https://metacpan.org/pod/Crypt%3A%3ACBC) and [Crypt::Blowfish](https://metacpan.org/pod/Crypt%3A%3ABlowfish), your credentials will be
encrypted using a random key created when the class is
instantiated. While this is more secure than leaving them in
plaintext, if the key is discovered (the key however is not stored in
the object's hash) and the object is dumped, your _encrypted_
credentials can be exposed.
- 4. Use very granular credentials for bucket access only.
Use credentials that only allow access to a bucket or portions of a
bucket required for your application. This will at least limit the
_blast radius_ of any potential security breach.
- 5. Do nothing...send the credentials, use the default signer.
In this case, both the `Amazon::S3` class and the
[Net::Amazon::Signature::V4](https://metacpan.org/pod/Net%3A%3AAmazon%3A%3ASignature%3A%3AV4) have your credentials. Caveat Emptor.
See also [Amazon::Credentials](https://metacpan.org/pod/Amazon%3A%3ACredentials) for more information about safely
storing your credentials and preventing exfiltration.
## region
Sets the region for the API calls. This will also be the
default when instantiating the bucket object unless you pass the
region parameter in the `bucket` method or use the `verify_region`
flag that will _always_ verify the region of the bucket using the
`get_location_constraint` method.
default: us-east-1
S3TestUtils.pm view on Meta::CPAN
sub get_s3_service {
########################################################################
my ($host) = @_;
my $s3 = eval {
if ( $ENV{AMAZON_S3_CREDENTIALS} ) {
require Amazon::Credentials;
return Amazon::S3->new(
{ credentials => Amazon::Credentials->new,
host => $host,
secure => is_aws(),
dns_bucket_names => $ENV{AMAZON_S3_DNS_BUCKET_NAMES},
level => $ENV{DEBUG} ? 'trace' : 'error',
}
);
}
else {
return Amazon::S3->new(
lib/Amazon/S3.pm view on Meta::CPAN
use parent qw(Class::Accessor::Fast);
__PACKAGE__->mk_accessors(
qw(
aws_access_key_id
aws_secret_access_key
token
buffer_size
cache_signer
credentials
dns_bucket_names
digest
err
errstr
error
host
last_request
last_response
logger
log_level
lib/Amazon/S3.pm view on Meta::CPAN
if ( $safe_options{aws_secret_access_key} ) {
$safe_options{aws_secret_access_key} = '****';
$safe_options{aws_access_key_id} = '****';
}
return Dumper( [ options => \%safe_options ] );
},
);
if ( !$self->credentials ) {
croak 'No aws_access_key_id'
if !$self->aws_access_key_id;
croak 'No aws_secret_access_key'
if !$self->aws_secret_access_key;
# encrypt credentials
$self->aws_access_key_id( _encrypt( $self->aws_access_key_id ) );
$self->aws_secret_access_key( _encrypt( $self->aws_secret_access_key ) );
$self->token( _encrypt( $self->token ) );
}
my $ua;
if ( $self->retry ) {
$ua = LWP::UserAgent::Determined->new(
keep_alive => $KEEP_ALIVE_CACHESIZE,
lib/Amazon/S3.pm view on Meta::CPAN
last if !$response->{is_truncated};
}
delete $all->{is_truncated};
delete $all->{next_marker};
return $all;
}
########################################################################
sub get_credentials {
########################################################################
my ($self) = @_;
my $aws_access_key_id;
my $aws_secret_access_key;
my $token;
if ( $self->credentials ) {
$aws_access_key_id = $self->credentials->get_aws_access_key_id;
$aws_secret_access_key = $self->credentials->get_aws_secret_access_key;
$token = $self->credentials->get_token;
}
else {
$aws_access_key_id = $self->aws_access_key_id;
$aws_secret_access_key = $self->aws_secret_access_key;
$token = $self->token;
}
return ( $aws_access_key_id, $aws_secret_access_key, $token );
}
lib/Amazon/S3.pm view on Meta::CPAN
}
########################################################################
sub signer {
########################################################################
my ($self) = @_;
return $self->_signer
if $self->_signer;
my $creds = $self->credentials ? $self->credentials : $self;
my $signer = Amazon::S3::Signature::V4->new(
{ access_key_id => $creds->get_aws_access_key_id,
secret => $creds->get_aws_secret_access_key,
region => $self->region || $self->get_default_region,
service => 's3',
security_token => $creds->get_token,
},
);
lib/Amazon/S3.pm view on Meta::CPAN
#
# Deprecated - this adds a header for the old V2 auth signatures
#
########################################################################
sub _add_auth_header { ## no critic (ProhibitUnusedPrivateSubroutines)
########################################################################
my ( $self, $headers, $method, $path ) = @_;
my ( $aws_access_key_id, $aws_secret_access_key, $token )
= $self->get_credentials;
if ( not $headers->header('Date') ) {
$headers->header( Date => time2str(time) );
}
if ($token) {
$headers->header( $AMAZON_HEADER_PREFIX . 'security-token', $token );
}
my $canonical_string = $self->_canonical_string( $method, $path, $headers );
lib/Amazon/S3.pm view on Meta::CPAN
=back
=head1 METHODS AND SUBROUTINES
=head2 new
Create a new S3 client object. Takes some arguments:
=over
=item credentials (optional)
Reference to a class (like C<Amazon::Credentials>) that can provide
credentials via the methods:
get_aws_access_key_id()
get_aws_secret_access_key()
get_token()
If you do not provide a credential class you must provide the keys
when you instantiate the object. See below.
I<You are strongly encourage to use a class that provides getters. If
you choose to provide your credentials to this class then they will be
stored in this object. If you dump the class you will likely expose
those credentials.>
=item aws_access_key_id
Use your Access Key ID as the value of the AWSAccessKeyId parameter
in requests you send to Amazon Web Services (when required). Your
Access Key ID identifies you as the party responsible for the
request.
=item aws_secret_access_key
Since your Access Key ID is not encrypted in requests to AWS, it
could be discovered and used by anyone. Services that are not free
require you to provide additional information, a request signature,
to verify that a request containing your unique Access Key ID could
only have come from you.
B<DO NOT INCLUDE THIS IN SCRIPTS OR APPLICATIONS YOU
DISTRIBUTE. YOU'LL BE SORRY.>
I<Consider using a credential class as described above to provide
credentials, otherwise this class will store your credentials for
signing the requests. If you dump this object to logs your credentials
could be discovered.>
=item token
An optional temporary token that will be inserted in the request along
with your access and secret key. A token is used in conjunction with
temporary credentials when your EC2 instance has
assumed a role and you've scraped the temporary credentials from
I<http://169.254.169.254/latest/meta-data/iam/security-credentials>
=item secure
Set this to a true value if you want to use SSL-encrypted connections
when connecting to S3. Starting in version 0.49, the default is true.
default: true
=item timeout
lib/Amazon/S3.pm view on Meta::CPAN
The default buffer size when reading or writing files.
default: 4096
=back
=head2 signer
Sets or retrieves the signer object. API calls must be signed using
your AWS credentials. By default, starting with version 0.54 the
module will use L<Net::Amazon::Signature::V4> as the signer and
instantiate a signer object in the constructor. Note however, that
signers need your credentials and they I<will> get stored by that
class, making them susceptible to inadvertant exfiltration. You have a
few options here:
=over 5
=item 1. Use your own signer.
You may have noticed that you can also provide your own credentials
object forcing this module to use your object for retrieving
credentials. Likewise, you can use your own signer so that this
module's signer never sees or stores those credentials.
=item 2. Pass the credentials object and set C<cache_signer> to a
false value.
If you pass a credentials object and set C<cache_signer> to a false
value, the module will use the credentials object to retrieve
credentials and create a new signer each time an API call is made that
requires signing. This prevents your credentials from being stored
inside of the signer class.
I<Note that using your own credentials object that stores your
credentials in plaintext is also going to expose your credentials when
someone dumps the class.>
=item 3. Pass credentials, set C<cache_signer> to a false value.
Unfortunately, while this will prevent L<Net::Amazon::Signature::V4>
from hanging on to your credentials, you credentials will be stored in
the C<Amazon::S3> object.
Starting with version 0.55 of this module, if you have installed
L<Crypt::CBC> and L<Crypt::Blowfish>, your credentials will be
encrypted using a random key created when the class is
instantiated. While this is more secure than leaving them in
plaintext, if the key is discovered (the key however is not stored in
the object's hash) and the object is dumped, your I<encrypted>
credentials can be exposed.
=item 4. Use very granular credentials for bucket access only.
Use credentials that only allow access to a bucket or portions of a
bucket required for your application. This will at least limit the
I<blast radius> of any potential security breach.
=item 5. Do nothing...send the credentials, use the default signer.
In this case, both the C<Amazon::S3> class and the
L<Net::Amazon::Signature::V4> have your credentials. Caveat Emptor.
See also L<Amazon::Credentials> for more information about safely
storing your credentials and preventing exfiltration.
=back
=head2 region
Sets the region for the API calls. This will also be the
default when instantiating the bucket object unless you pass the
region parameter in the C<bucket> method or use the C<verify_region>
flag that will I<always> verify the region of the bucket using the
C<get_location_constraint> method.
( run in 0.270 second using v1.01-cache-2.11-cpan-a5abf4f5562 )