view release on metacpan or search on metacpan
* .gitignore: modified
* bin/amzn-s3-cli.in: new
* bin/run-test.in: deleted
* requires: JSON::PP
* suggests: +File::MimeInfo::Magic, +Text::ASCIITable, -Text::ASCIITable::EasyTable
* deps.mk: generated
* cpanfile: likewise
* Makefile: updated by CPAN::Maker::Bootstrapper
* lib/Amazon/S3/CLI.pm.in: new
* lib/Amazon/S3.pm.in
- new option endpoint_url
- pod updates
(new)
- handle new endpoint_url option
(_parse_endpoint): new
(region)
- only add region if host was not populated from endpoint_url
(empty_bucket): new
(list_bucket_all)
- change $bucket to $bucket_name to avoid confusion
(is_json_response): new
(decode_response)
- decode JSON payloads with fallback
- fix call to _handle_response_error
* lib/Amazon/S3/Bucket.pm.in
- pod updates
(_add_key): new
(add_key_filename): likewise
(_state_upload_source): new
* lib/Amazon/S3/BucketV2.pm.in
- pod updates
(@LIST_BUCKET_METHODS): new
(@DELETE_OBJECT_METHODS)
- +abort_multipart_upload
* lib/Amazon/S3/CLI.pm.in: modified
* requires: modified
* suggests: modified
* t/10-endpoint-url.t
- new subtest 'host as url'
* t/lib/S3TestUtils.pm
(set_s3_host)
- allow passing of host
- set host from environment first
- check LocalStack services if LocalStack
(get_s3_service)
- add raise_error parameter
- require host
- die if error creating service
* VERSION: bump
* README.md: generated
* buildspec.yml
- moved S3TestUtils.pm
- renamed git repository
* lib/Amazon/S3.pm.in
- complete POD rewrite
- add checksum_types, checksum_algorithm, verify_checksums accessors, raise_error
(new)
- set default options for new accessors
- support endpoint_url
(use_express_one_zone): set secure
(_decode_response)
- use _handle_response_error to support raise_error
(send_request_expect_nothing_: likewise
(send_request_expect_nothing_probed): likewise
(_croak_if_response_error): likewise
(_handle_response_error): new
(_init_checksum_types): new
(add_bucket): _send_request_expect_nothing => send_request_expect_nothing
(delete_bucket): likewise
t/00-amazon-s3.t
t/01-api.t
t/02-logger.t
t/03-region.t
t/04-list-buckets.t
t/05-multipart-upload.t
t/06-list-multipart-uploads.t
t/07-crc64nvme.t
t/08-checksums.t
t/09-bucket-v2.t
t/10-endpoint-url.t
t/11-empty-bucket.t
t/12-add-key-sources.t
t/13-response-decode.t
t/lib/S3TestUtils.pm
META.yml Module YAML meta-data (added by MakeMaker)
META.json Module JSON meta-data (added by MakeMaker)
# DESCRIPTION
`Amazon::S3` provides a Perl interface to Amazon Simple Storage
Service (S3).
The distribution separates account-level S3 operations from
bucket and object operations.
`Amazon::S3` represents the S3 client and AWS account context. It
manages credentials, request signing, regions, service endpoints,
bucket creation and discovery, and account-level listing operations.
[Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) represents an individual bucket. Object
operations such as uploads, downloads, deletes, ACLs, multipart
uploads, and bucket-scoped listing operations are provided primarily
through that class.
[Amazon::S3::BucketV2](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucketV2) is a subclass of [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) that
provides a more general interface to S3 APIs. It accepts API
parameters, headers, URI parameters, and request payload structures
`Amazon::S3` provides limited support for Amazon S3 directory
buckets.
Directory buckets use the S3 Express One Zone storage class.
`Amazon::S3` can currently create and list directory buckets.
Object access within directory buckets is not yet supported because
that requires creating an S3 Express session and using the resulting
temporary credentials when signing requests to the directory bucket's
Zonal endpoint.
A directory bucket can be created by supplying an availability zone
to `add_bucket()`:
my $bucket = $s3->add_bucket(
{ bucket => $bucket_name,
availability_zone => 'use1-az5',
}
);
the S3 error code and message.
Some request, protocol, multipart, checksum verification, and
validation failures always throw exceptions because the operation
cannot safely continue.
The most recent HTTP request and response can be inspected using
`last_request()` and `last_response()`.
These accessors are especially useful when diagnosing signing,
endpoint, header, or protocol problems.
`raise_error` defaults to false to preserve the historical
`Amazon::S3` interface. New applications may prefer to enable it when
they want request failures to be impossible to overlook.
# METHODS AND SUBROUTINES
This section documents methods provided directly by `Amazon::S3`.
Methods implemented by [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) or
- dns\_bucket\_names
Controls whether virtual-hosted-style bucket names are used when
possible.
The default is true.
A bucket name that cannot be used as a DNS subdomain is placed in the
request path instead.
- endpoint\_url
Optional explicit S3 service endpoint.
When `endpoint_url` is supplied, `Amazon::S3` uses that endpoint as
specified and does not rewrite the host when the configured region
changes.
The `region` setting still controls the AWS signing region.
This is useful for S3-compatible services, local test environments, and
other cases where the caller must select the service endpoint explicitly.
For example:
my $s3 = Amazon::S3->new(
{ endpoint_url => 'http://localhost:4566',
region => 'us-east-1',
...
}
);
When `endpoint_url` is not supplied, `Amazon::S3` may derive the
standard AWS S3 endpoint from the configured region.
`endpoint_url` and `host` may not both be supplied.
- host
S3 service endpoint.
The default is `s3.amazonaws.com`.
When `region()` is set and the host is a standard Amazon S3 endpoint,
`Amazon::S3` adjusts the host for the configured region.
This option can also be used with S3-compatible and local testing
services.
- level
Logging level used when `Amazon::S3` creates its default logger.
The default is `error`.
### errstr
Returns the most recent human-readable error message.
See ["ERROR HANDLING"](#error-handling).
### host
my $host = $s3->host;
$s3->host($endpoint);
Gets or sets the configured S3 endpoint.
The constructor default is `s3.amazonaws.com`.
See ["S3-COMPATIBLE SERVICES"](#s3-compatible-services).
### last\_request
Returns the most recent [HTTP::Request](https://metacpan.org/pod/HTTP%3A%3ARequest) generated by `Amazon::S3`.
See ["ERROR HANDLING"](#error-handling).
### get\_default\_region
my $region = $s3->get_default_region;
Attempts to determine the default AWS region.
The method checks, in order:
1. `AWS_REGION`.
2. `AWS_DEFAULT_REGION`.
3. The EC2 instance metadata availability-zone endpoint.
When an availability zone is obtained from instance metadata, the
zone suffix is removed to derive the region.
If no region can be determined, `us-east-1` is returned.
### get\_logger
my $logger = $s3->get_logger;
$s3->region('us-west-2');
Gets or sets the region used by the client.
The region is used for account-level requests and as the default
region assigned to bucket objects when no bucket-specific region is
supplied.
When the configured host uses the standard `s3.amazonaws.com` form,
setting the region also adjusts the host to the regional Amazon S3
endpoint.
The constructor default is `us-east-1`.
### signer
my $signer = $s3->signer;
Returns the Signature Version 4 signer used for requests.
If a signer was supplied to the constructor, that signer is returned.
my $response = $s3->list_directory_buckets;
my $response = $s3->list_directory_buckets(
{ uri_params => \%params,
}
);
Lists directory buckets owned by the account.
The optional `uri_params` hash reference is passed as URI parameters
to the S3 Express control endpoint.
The method temporarily switches the client to the S3 Express control
endpoint for the request and restores the previous express-mode state
afterward.
On success, returns the parsed S3 response.
On failure, returns `undef` and records error information on the
client.
See ["DIRECTORY BUCKETS"](#directory-buckets).
## OBJECT LISTING AND VERSIONING
At `trace` level, HTTP request and response information may also be
logged.
Applications should review trace output before retaining or sharing
it. Request and response data may contain sensitive application
information even when authentication values are sanitized.
# S3-COMPATIBLE SERVICES
`Amazon::S3` can be used with S3-compatible services and local S3
implementations by configuring the service endpoint and related
connection options.
The `host`, `secure`, and `dns_bucket_names` settings are commonly
relevant when using a non-AWS endpoint.
S3-compatible implementations may differ from AWS in supported APIs,
request validation, checksum behavior, or edge cases.
The integration tests used during development include LocalStack, but
applications targeting another S3-compatible implementation should
test against that implementation directly.
# COMPARISON TO OTHER PERL S3 MODULES
## Directory Buckets
Directory bucket support is currently limited to account-level create
and list operations.
See ["DIRECTORY BUCKETS"](#directory-buckets).
# TESTING
The distribution includes unit tests and integration tests that
exercise behavior requiring an S3 endpoint.
Run the normal distribution test suite with:
make test
Integration testing during development includes LocalStack.
See `README-TESTING.md` in the distribution root for test
environment setup, integration-test requirements, and additional
testing instructions.
lib/Amazon/S3.pm view on Meta::CPAN
aws_access_key_id
aws_secret_access_key
token
buffer_size
cache_signer
checksum_types
checksum_algorithm
credentials
dns_bucket_names
digest
endpoint_url
err
errstr
error
express
host
last_request
last_response
logger
log_level
raise_error
lib/Amazon/S3.pm view on Meta::CPAN
my %options = ref $args[0] ? %{ $args[0] } : @args;
$options{timeout} //= $DEFAULT_TIMEOUT;
$options{cache_signer} //= $FALSE;
$options{retry} //= $FALSE;
$options{express} //= $FALSE;
$options{verify_checksums} //= $TRUE;
$options{checksum_algorithm} //= 'crc64nvme';
$options{raise_error} //= $FALSE;
croak "ERROR: pass endpoint_url or host but not both\n"
if $options{endpoint_url} && $options{host};
if ( my $endpoint_url = $options{endpoint_url} ) {
@options{qw(secure host)} = _parse_endpoint($endpoint_url);
}
elsif ( $options{host} ) {
if ( $options{host} =~ m{\Ahttps?://}xsm ) {
warn "host containing a URL is deprecated; use endpoint_url instead\n";
@options{qw(secure host)} = _parse_endpoint( $options{host} );
}
else {
$options{secure} //= $TRUE;
}
}
else {
$options{secure} = $TRUE;
$options{host} = $DEFAULT_HOST;
}
lib/Amazon/S3.pm view on Meta::CPAN
}
$self->turn_on_special_retry();
$self->_init_checksum_types;
return $self;
}
########################################################################
sub _parse_endpoint {
########################################################################
my ($endpoint_url) = @_;
my $uri = URI->new($endpoint_url);
my $scheme = $uri->scheme;
croak "ERROR: endpoint_url must include a host\n"
if !defined $uri->host || !length $uri->host;
croak "ERROR: endpoint_url must use http or https\n"
if !defined $scheme || ( $scheme ne 'http' && $scheme ne 'https' );
croak "ERROR: endpoint_url must not contain a query or fragment\n"
if defined $uri->query || defined $uri->fragment;
croak "ERROR: endpoint_url must not contain a path\n"
if $uri->path && $uri->path ne $SLASH;
my $secure = $scheme eq 'https' ? $TRUE : $FALSE;
my $host = $uri->host_port;
return ( $secure, $host );
}
########################################################################
sub _init_checksum_types {
lib/Amazon/S3.pm view on Meta::CPAN
if (@args) {
$self->_region( $args[0] );
}
$self->get_logger->debug( sub { return 'region: ' . ( $self->_region // $EMPTY ) } );
if ( $self->_region ) {
my $host = $self->host;
$self->get_logger->debug( sub { return 'host: ' . $self->host } );
# only add region if host was not populated from endpoint_url
if ( !$self->endpoint_url && $host =~ /\As3[.](.*)?amazonaws/xsm ) {
$self->host( sprintf 's3.%s.amazonaws.com', $self->_region );
}
}
return $self->_region;
}
########################################################################
sub buckets {
########################################################################
my ( $self, $verify_region ) = @_;
# The "default" region for Amazon is us-east-1
# This is the region to set it to for listing buckets
# You may need to reset the signer's endpoint to 'us-east-1'
# temporarily cache signer
my $region = $self->_region;
my $bucket_list;
$self->reset_signer_region($DEFAULT_REGION); # default region for buckets op
my $r = $self->send_request(
{ method => 'GET',
path => $EMPTY,
lib/Amazon/S3.pm view on Meta::CPAN
}
my $request = HTTP::Request->new( $method, $url, $http_headers );
$self->last_request($request);
if ($data) {
$request->content($data);
}
$self->signer->region($region); # always set regional endpoint for signing
$self->signer->sign($request);
return $request;
}
########################################################################
sub send_request {
########################################################################
my ( $self, @args ) = @_;
lib/Amazon/S3.pm view on Meta::CPAN
return $TRUE
if $response->is_success();
# If the error is due to the wrong region, then we will get
# back a block of XML with the details
return $FALSE
if !is_xml_response($response);
my $error_hash = $self->_xpc_of_content( $response->content );
my ( $endpoint, $code, $region, $message )
= @{$error_hash}{qw(Endpoint Code Region Message)};
my $condition = eval {
return 'PermanentRedirect'
if $code eq 'PermanentRedirect' && $endpoint;
return 'AuthorizationHeaderMalformed'
if $code eq 'AuthorizationHeaderMalformed' && $region;
return 'IllegalLocationConstraintException'
if $code eq 'IllegalLocationConstraintException';
return 'Other';
};
my %error_handlers = (
PermanentRedirect => sub {
# Don't recurse through multiple redirects
return $FALSE
if $called_from_redirect;
# With a permanent redirect error, they are telling us the explicit
# host to use. The endpoint will be in the form of bucket.host
my $host = $endpoint;
# Remove the bucket name from the front of the host name
# All the requests will need to be of the form https://host/bucket
$host =~ s/\A$bucket[.]//xsm;
$self->host($host);
# We will need to call ourselves again in order to trigger the
# AuthorizationHeaderMalformed error in order to get the region
return $self->adjust_region( $bucket, $TRUE );
},
AuthorizationHeaderMalformed => sub {
# Set the signer to use the correct reader evermore
$self->{signer}->{endpoint} = $region;
# Only change the host if we haven't been called as a redirect
# where an exact host has been given
if ( !$called_from_redirect ) {
$self->host( sprintf 's3-%s-amazonaws.com', $region );
}
return $TRUE;
},
IllegalLocationConstraintException => sub {
# This is hackish; but in this case the region name only appears in the message
if ( $message =~ /The (\S+) location/xsm ) {
my $new_region = $1;
# Correct the region for the signer
$self->{signer}->{endpoint} = $new_region;
# Set the proper host for the region
$self->host( sprintf 's3.%s.amazonaws.com', $new_region );
return $TRUE;
}
},
'Other' => sub {
# Some other error
$self->_remember_errors( $response->content, 1 );
lib/Amazon/S3.pm view on Meta::CPAN
=head1 DESCRIPTION
C<Amazon::S3> provides a Perl interface to Amazon Simple Storage
Service (S3).
The distribution separates account-level S3 operations from
bucket and object operations.
C<Amazon::S3> represents the S3 client and AWS account context. It
manages credentials, request signing, regions, service endpoints,
bucket creation and discovery, and account-level listing operations.
L<Amazon::S3::Bucket> represents an individual bucket. Object
operations such as uploads, downloads, deletes, ACLs, multipart
uploads, and bucket-scoped listing operations are provided primarily
through that class.
L<Amazon::S3::BucketV2> is a subclass of L<Amazon::S3::Bucket> that
provides a more general interface to S3 APIs. It accepts API
parameters, headers, URI parameters, and request payload structures
lib/Amazon/S3.pm view on Meta::CPAN
C<Amazon::S3> provides limited support for Amazon S3 directory
buckets.
Directory buckets use the S3 Express One Zone storage class.
C<Amazon::S3> can currently create and list directory buckets.
Object access within directory buckets is not yet supported because
that requires creating an S3 Express session and using the resulting
temporary credentials when signing requests to the directory bucket's
Zonal endpoint.
A directory bucket can be created by supplying an availability zone
to C<add_bucket()>:
my $bucket = $s3->add_bucket(
{ bucket => $bucket_name,
availability_zone => 'use1-az5',
}
);
lib/Amazon/S3.pm view on Meta::CPAN
the S3 error code and message.
Some request, protocol, multipart, checksum verification, and
validation failures always throw exceptions because the operation
cannot safely continue.
The most recent HTTP request and response can be inspected using
C<last_request()> and C<last_response()>.
These accessors are especially useful when diagnosing signing,
endpoint, header, or protocol problems.
C<raise_error> defaults to false to preserve the historical
C<Amazon::S3> interface. New applications may prefer to enable it when
they want request failures to be impossible to overlook.
=head1 METHODS AND SUBROUTINES
This section documents methods provided directly by C<Amazon::S3>.
Methods implemented by L<Amazon::S3::Bucket> or
lib/Amazon/S3.pm view on Meta::CPAN
=item dns_bucket_names
Controls whether virtual-hosted-style bucket names are used when
possible.
The default is true.
A bucket name that cannot be used as a DNS subdomain is placed in the
request path instead.
=item endpoint_url
Optional explicit S3 service endpoint.
When C<endpoint_url> is supplied, C<Amazon::S3> uses that endpoint as
specified and does not rewrite the host when the configured region
changes.
The C<region> setting still controls the AWS signing region.
This is useful for S3-compatible services, local test environments, and
other cases where the caller must select the service endpoint explicitly.
For example:
my $s3 = Amazon::S3->new(
{ endpoint_url => 'http://localhost:4566',
region => 'us-east-1',
...
}
);
When C<endpoint_url> is not supplied, C<Amazon::S3> may derive the
standard AWS S3 endpoint from the configured region.
C<endpoint_url> and C<host> may not both be supplied.
=item host
S3 service endpoint.
The default is C<s3.amazonaws.com>.
When C<region()> is set and the host is a standard Amazon S3 endpoint,
C<Amazon::S3> adjusts the host for the configured region.
This option can also be used with S3-compatible and local testing
services.
=item level
Logging level used when C<Amazon::S3> creates its default logger.
The default is C<error>.
lib/Amazon/S3.pm view on Meta::CPAN
=head3 errstr
Returns the most recent human-readable error message.
See L</ERROR HANDLING>.
=head3 host
my $host = $s3->host;
$s3->host($endpoint);
Gets or sets the configured S3 endpoint.
The constructor default is C<s3.amazonaws.com>.
See L</S3-COMPATIBLE SERVICES>.
=head3 last_request
Returns the most recent L<HTTP::Request> generated by C<Amazon::S3>.
See L</ERROR HANDLING>.
lib/Amazon/S3.pm view on Meta::CPAN
=item 1.
C<AWS_REGION>.
=item 2.
C<AWS_DEFAULT_REGION>.
=item 3.
The EC2 instance metadata availability-zone endpoint.
=back
When an availability zone is obtained from instance metadata, the
zone suffix is removed to derive the region.
If no region can be determined, C<us-east-1> is returned.
=head3 get_logger
lib/Amazon/S3.pm view on Meta::CPAN
$s3->region('us-west-2');
Gets or sets the region used by the client.
The region is used for account-level requests and as the default
region assigned to bucket objects when no bucket-specific region is
supplied.
When the configured host uses the standard C<s3.amazonaws.com> form,
setting the region also adjusts the host to the regional Amazon S3
endpoint.
The constructor default is C<us-east-1>.
=head3 signer
my $signer = $s3->signer;
Returns the Signature Version 4 signer used for requests.
If a signer was supplied to the constructor, that signer is returned.
lib/Amazon/S3.pm view on Meta::CPAN
my $response = $s3->list_directory_buckets;
my $response = $s3->list_directory_buckets(
{ uri_params => \%params,
}
);
Lists directory buckets owned by the account.
The optional C<uri_params> hash reference is passed as URI parameters
to the S3 Express control endpoint.
The method temporarily switches the client to the S3 Express control
endpoint for the request and restores the previous express-mode state
afterward.
On success, returns the parsed S3 response.
On failure, returns C<undef> and records error information on the
client.
See L</DIRECTORY BUCKETS>.
=head2 OBJECT LISTING AND VERSIONING
lib/Amazon/S3.pm view on Meta::CPAN
At C<trace> level, HTTP request and response information may also be
logged.
Applications should review trace output before retaining or sharing
it. Request and response data may contain sensitive application
information even when authentication values are sanitized.
=head1 S3-COMPATIBLE SERVICES
C<Amazon::S3> can be used with S3-compatible services and local S3
implementations by configuring the service endpoint and related
connection options.
The C<host>, C<secure>, and C<dns_bucket_names> settings are commonly
relevant when using a non-AWS endpoint.
S3-compatible implementations may differ from AWS in supported APIs,
request validation, checksum behavior, or edge cases.
The integration tests used during development include LocalStack, but
applications targeting another S3-compatible implementation should
test against that implementation directly.
=head1 COMPARISON TO OTHER PERL S3 MODULES
lib/Amazon/S3.pm view on Meta::CPAN
=head2 Directory Buckets
Directory bucket support is currently limited to account-level create
and list operations.
See L</DIRECTORY BUCKETS>.
=head1 TESTING
The distribution includes unit tests and integration tests that
exercise behavior requiring an S3 endpoint.
Run the normal distribution test suite with:
make test
Integration testing during development includes LocalStack.
See F<README-TESTING.md> in the distribution root for test
environment setup, integration-test requirements, and additional
testing instructions.
lib/Amazon/S3/CLI.pm view on Meta::CPAN
}
return $SUCCESS;
}
########################################################################
sub init {
########################################################################
my ($self) = @_;
my %endpoint = choose {
return ( host => $self->get_host )
if defined $self->get_host;
return ( endpoint_url => $self->get_endpoint_url )
if $self->get_endpoint_url;
return ();
};
$self->set_ascii_table(
eval {
require Text::ASCIITable;
1;
}
);
lib/Amazon/S3/CLI.pm view on Meta::CPAN
$credentials{token} = $ENV{AWS_SESSION_TOKEN};
}
my $s3 = Amazon::S3->new(
{ %credentials,
debug => $ENV{DEBUG},
raise_error => $TRUE,
region => $self->get_region,
logger => $self->get_logger,
dns_bucket_names => $self->get_dns_bucket_names // $FALSE,
%endpoint,
defined $self->get_secure ? ( secure => $self->get_secure ) : (),
}
);
$self->set_s3($s3);
return;
}
########################################################################
lib/Amazon/S3/CLI.pm view on Meta::CPAN
region => 'us-east-1',
format => 'text',
);
my @option_specs = qw(
availability-zone=s
bucket-name|b=s
content-type=s
debug
dns-bucket-names
endpoint-url|u=s
force|f
format|F=s
help|h
host|H=s
key|k=s
modified-since|m=s
output|o=s
prefix=s
profile|p=s
range|R=s
lib/Amazon/S3/CLI.pm view on Meta::CPAN
=head1 SYNOPSIS
# list buckets in tabular format
amzn-s3-cli list-buckets --profile sandbox --format table
# list keys in JSON format
amzn-s3-cli list-keys test-bucket --format json
# create a new bucket on LocalStack
amzn-s3-cli create-bucket my-bucket --endpoint-url http://localhost:4566 --profile localstack
# add a key using the filename as the object name
amzn-s3-cli add-key --bucket-name my-bucket some-file
# add a key with a different object name
amzn-s3-cli add-key --bucket-name my-bucket some-file project-alpha/some-file
# retrieve an object
amzn-s3-cli get-key my-bucket project-alpha/some-file
lib/Amazon/S3/CLI.pm view on Meta::CPAN
=head1 DESCRIPTION
C<Amazon::S3::CLI> provides the C<amzn-s3-cli> command for performing
common Amazon S3 operations using L<Amazon::S3>.
It is intended as a convenient command-line companion to the library,
not as a replacement for the complete AWS CLI or C<aws s3api> command
surface.
The CLI supports common bucket and object operations and can also be
used with S3-compatible services by specifying an alternate endpoint.
=head2 Commands
=over
=item B<add-key>
amzn-s3-cli add-key bucket-name filename [object-name]
Adds a file to a bucket.
lib/Amazon/S3/CLI.pm view on Meta::CPAN
Most commands accept the bucket name positionally or through
C<--bucket-name>.
Output-producing commands generally support C<text>, C<json>, and
C<table> formats. Table output requires L<Text::ASCIITable>.
Credentials are discovered through L<Amazon::Credentials> when that
module is installed. Otherwise the CLI uses the standard AWS credential
environment variables.
Use C<--endpoint-url> when working with an alternate S3-compatible
service such as LocalStack.
=head1 OPTIONS
=over
=item B<--availability-zone>
Specifies an Availability Zone when creating an S3 Express One Zone
directory bucket.
lib/Amazon/S3/CLI.pm view on Meta::CPAN
Specifies the content type used when adding an object.
=item B<--debug>
Enables debug logging.
=item B<--dns-bucket-names>
Enables DNS-style bucket addressing.
=item B<--endpoint-url>, B<-u>
Specifies the complete S3 service endpoint URL.
For example:
--endpoint-url http://localhost:4566
The URL scheme determines whether HTTP or HTTPS is used.
=item B<--force>, B<-f>
Suppresses confirmation prompts for destructive operations.
=item B<--format>, B<-F>
Selects the output format.
lib/Amazon/S3/CLI.pm view on Meta::CPAN
=item B<--help>, B<-h>
Displays command help.
=item B<--host>, B<-H>
Specifies the S3 service host using the legacy host configuration
mechanism.
C<--endpoint-url> is preferred for alternate endpoints.
=item B<--key>, B<-k>
Specifies an object key.
=item B<--modified-since>, B<-m>
Supplies an C<If-Modified-Since> condition to C<get-key>.
=item B<--output>, B<-o>
lib/Amazon/S3/CLI.pm view on Meta::CPAN
--range 0-1023
=item B<--region>, B<-r>
Specifies the AWS region.
The default is C<us-east-1>.
=item B<--secure>, B<-s>
Controls secure transport when host-based endpoint configuration is
used.
When C<--endpoint-url> is supplied, the URL scheme determines the
transport.
=item B<--version-id>
Specifies an object version ID for C<get-key> or C<delete-key>.
=back
=head1 AUTHENTICATION
lib/Amazon/S3/CLI.pm view on Meta::CPAN
amzn-s3-cli list-buckets --profile sandbox
Without L<Amazon::Credentials>, credentials are read from:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_SESSION_TOKEN
=head1 S3-COMPATIBLE SERVICES
An alternate S3 endpoint may be selected with C<--endpoint-url>:
amzn-s3-cli list-buckets --endpoint-url http://localhost:4566 --profile localstack
This is useful for LocalStack and other S3-compatible services.
=head1 OPTIONAL DEPENDENCIES
=over
=item L<Amazon::Credentials>
Provides credential discovery and named profile support.
lib/Amazon/S3/Signature/V4.pm view on Meta::CPAN
use parent qw{Net::Amazon::Signature::V4};
########################################################################
sub new {
########################################################################
my ( $class, @args ) = @_;
my %options;
if ( !ref $args[0] ) {
@options{qw{access_key_id secret endpoint service}} = @args;
}
else {
%options = %{ $args[0] };
}
my $region = delete $options{region};
$options{endpoint} //= $region;
my $self = $class->SUPER::new( \%options );
return $self;
}
########################################################################
sub region {
########################################################################
my ( $self, @args ) = @_;
if (@args) {
$self->{endpoint} = $args[0];
}
return $self->{endpoint};
}
1;
share/README.md view on Meta::CPAN
# DESCRIPTION
`Amazon::S3` provides a Perl interface to Amazon Simple Storage
Service (S3).
The distribution separates account-level S3 operations from
bucket and object operations.
`Amazon::S3` represents the S3 client and AWS account context. It
manages credentials, request signing, regions, service endpoints,
bucket creation and discovery, and account-level listing operations.
[Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) represents an individual bucket. Object
operations such as uploads, downloads, deletes, ACLs, multipart
uploads, and bucket-scoped listing operations are provided primarily
through that class.
[Amazon::S3::BucketV2](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucketV2) is a subclass of [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) that
provides a more general interface to S3 APIs. It accepts API
parameters, headers, URI parameters, and request payload structures
share/README.md view on Meta::CPAN
`Amazon::S3` provides limited support for Amazon S3 directory
buckets.
Directory buckets use the S3 Express One Zone storage class.
`Amazon::S3` can currently create and list directory buckets.
Object access within directory buckets is not yet supported because
that requires creating an S3 Express session and using the resulting
temporary credentials when signing requests to the directory bucket's
Zonal endpoint.
A directory bucket can be created by supplying an availability zone
to `add_bucket()`:
my $bucket = $s3->add_bucket(
{ bucket => $bucket_name,
availability_zone => 'use1-az5',
}
);
share/README.md view on Meta::CPAN
the S3 error code and message.
Some request, protocol, multipart, checksum verification, and
validation failures always throw exceptions because the operation
cannot safely continue.
The most recent HTTP request and response can be inspected using
`last_request()` and `last_response()`.
These accessors are especially useful when diagnosing signing,
endpoint, header, or protocol problems.
`raise_error` defaults to false to preserve the historical
`Amazon::S3` interface. New applications may prefer to enable it when
they want request failures to be impossible to overlook.
# METHODS AND SUBROUTINES
This section documents methods provided directly by `Amazon::S3`.
Methods implemented by [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) or
share/README.md view on Meta::CPAN
- dns\_bucket\_names
Controls whether virtual-hosted-style bucket names are used when
possible.
The default is true.
A bucket name that cannot be used as a DNS subdomain is placed in the
request path instead.
- endpoint\_url
Optional explicit S3 service endpoint.
When `endpoint_url` is supplied, `Amazon::S3` uses that endpoint as
specified and does not rewrite the host when the configured region
changes.
The `region` setting still controls the AWS signing region.
This is useful for S3-compatible services, local test environments, and
other cases where the caller must select the service endpoint explicitly.
For example:
my $s3 = Amazon::S3->new(
{ endpoint_url => 'http://localhost:4566',
region => 'us-east-1',
...
}
);
When `endpoint_url` is not supplied, `Amazon::S3` may derive the
standard AWS S3 endpoint from the configured region.
`endpoint_url` and `host` may not both be supplied.
- host
S3 service endpoint.
The default is `s3.amazonaws.com`.
When `region()` is set and the host is a standard Amazon S3 endpoint,
`Amazon::S3` adjusts the host for the configured region.
This option can also be used with S3-compatible and local testing
services.
- level
Logging level used when `Amazon::S3` creates its default logger.
The default is `error`.
share/README.md view on Meta::CPAN
### errstr
Returns the most recent human-readable error message.
See ["ERROR HANDLING"](#error-handling).
### host
my $host = $s3->host;
$s3->host($endpoint);
Gets or sets the configured S3 endpoint.
The constructor default is `s3.amazonaws.com`.
See ["S3-COMPATIBLE SERVICES"](#s3-compatible-services).
### last\_request
Returns the most recent [HTTP::Request](https://metacpan.org/pod/HTTP%3A%3ARequest) generated by `Amazon::S3`.
See ["ERROR HANDLING"](#error-handling).
share/README.md view on Meta::CPAN
### get\_default\_region
my $region = $s3->get_default_region;
Attempts to determine the default AWS region.
The method checks, in order:
1. `AWS_REGION`.
2. `AWS_DEFAULT_REGION`.
3. The EC2 instance metadata availability-zone endpoint.
When an availability zone is obtained from instance metadata, the
zone suffix is removed to derive the region.
If no region can be determined, `us-east-1` is returned.
### get\_logger
my $logger = $s3->get_logger;
share/README.md view on Meta::CPAN
$s3->region('us-west-2');
Gets or sets the region used by the client.
The region is used for account-level requests and as the default
region assigned to bucket objects when no bucket-specific region is
supplied.
When the configured host uses the standard `s3.amazonaws.com` form,
setting the region also adjusts the host to the regional Amazon S3
endpoint.
The constructor default is `us-east-1`.
### signer
my $signer = $s3->signer;
Returns the Signature Version 4 signer used for requests.
If a signer was supplied to the constructor, that signer is returned.
share/README.md view on Meta::CPAN
my $response = $s3->list_directory_buckets;
my $response = $s3->list_directory_buckets(
{ uri_params => \%params,
}
);
Lists directory buckets owned by the account.
The optional `uri_params` hash reference is passed as URI parameters
to the S3 Express control endpoint.
The method temporarily switches the client to the S3 Express control
endpoint for the request and restores the previous express-mode state
afterward.
On success, returns the parsed S3 response.
On failure, returns `undef` and records error information on the
client.
See ["DIRECTORY BUCKETS"](#directory-buckets).
## OBJECT LISTING AND VERSIONING
share/README.md view on Meta::CPAN
At `trace` level, HTTP request and response information may also be
logged.
Applications should review trace output before retaining or sharing
it. Request and response data may contain sensitive application
information even when authentication values are sanitized.
# S3-COMPATIBLE SERVICES
`Amazon::S3` can be used with S3-compatible services and local S3
implementations by configuring the service endpoint and related
connection options.
The `host`, `secure`, and `dns_bucket_names` settings are commonly
relevant when using a non-AWS endpoint.
S3-compatible implementations may differ from AWS in supported APIs,
request validation, checksum behavior, or edge cases.
The integration tests used during development include LocalStack, but
applications targeting another S3-compatible implementation should
test against that implementation directly.
# COMPARISON TO OTHER PERL S3 MODULES
share/README.md view on Meta::CPAN
## Directory Buckets
Directory bucket support is currently limited to account-level create
and list operations.
See ["DIRECTORY BUCKETS"](#directory-buckets).
# TESTING
The distribution includes unit tests and integration tests that
exercise behavior requiring an S3 endpoint.
Run the normal distribution test suite with:
make test
Integration testing during development includes LocalStack.
See `README-TESTING.md` in the distribution root for test
environment setup, integration-test requirements, and additional
testing instructions.
t/10-endpoint-url.t view on Meta::CPAN
use English qw(-no_match_vars);
use Test::More;
our $TRUE = 1;
our $FALSE = 0;
our $DEFAULT_HOST = 's3.amazonaws.com';
use_ok(qw(Amazon::S3));
########################################################################
subtest 'endpoint_url' => sub {
########################################################################
my %options = (
endpoint_url => 'http://localhost:4566',
aws_access_key_id => 'test',
aws_secret_access_key => 'test',
);
my $s3 = Amazon::S3->new(%options);
is( $s3->host, 'localhost:4566', 'host' );
ok( !$s3->secure, 'secure' );
};
########################################################################
subtest 'endpoint_url and host' => sub {
########################################################################
my %options = (
endpoint_url => 'http://localhost:4566',
aws_access_key_id => 'test',
aws_secret_access_key => 'test',
host => 'localstack:4566',
);
my $s3 = eval { Amazon::S3->new(%options); };
like( $EVAL_ERROR, qr/error/i, 'throws' );
};