Amazon-S3

 view release on metacpan or  search on metacpan

ChangeLog  view on Meta::CPAN

	* .gitignore: modified
	* bin/amzn-s3-cli.in: new
	* bin/run-test.in: deleted
	* requires: JSON::PP
	* suggests: +File::MimeInfo::Magic, +Text::ASCIITable, -Text::ASCIITable::EasyTable
	* deps.mk: generated
	* cpanfile: likewise
	* Makefile: updated by CPAN::Maker::Bootstrapper
	* lib/Amazon/S3/CLI.pm.in: new
	* lib/Amazon/S3.pm.in
	- new option endpoint_url
	- pod updates
	(new)
	- handle new endpoint_url option
	(_parse_endpoint): new
	(region)
	- only add region if host was not populated from endpoint_url
	(empty_bucket): new
	(list_bucket_all)
	- change $bucket to $bucket_name to avoid confusion
	(is_json_response): new
	(decode_response)
	- decode JSON payloads with fallback
	- fix call to _handle_response_error
	* lib/Amazon/S3/Bucket.pm.in
	- pod updates
	(_add_key): new

ChangeLog  view on Meta::CPAN

	(add_key_filename): likewise
	(_state_upload_source): new
	* lib/Amazon/S3/BucketV2.pm.in
	- pod updates
	(@LIST_BUCKET_METHODS): new
	(@DELETE_OBJECT_METHODS)
	- +abort_multipart_upload
	* lib/Amazon/S3/CLI.pm.in: modified
	* requires: modified
	* suggests: modified
	* t/10-endpoint-url.t
	- new subtest 'host as url'
	* t/lib/S3TestUtils.pm
	(set_s3_host)
	- allow passing of host
	- set host from environment first
	- check LocalStack services if LocalStack
	(get_s3_service)
	- add raise_error parameter
	- require host
	- die if error creating service

ChangeLog  view on Meta::CPAN

	* VERSION: bump
	* README.md: generated
	* buildspec.yml
	- moved S3TestUtils.pm
	- renamed git repository
	* lib/Amazon/S3.pm.in
	- complete POD rewrite
	- add checksum_types, checksum_algorithm, verify_checksums accessors, raise_error
	(new)
	- set default options for new accessors
	- support endpoint_url
	(use_express_one_zone): set secure
	(_decode_response)
	- use _handle_response_error to support raise_error
	(send_request_expect_nothing_: likewise
	(send_request_expect_nothing_probed): likewise
	(_croak_if_response_error): likewise
	(_handle_response_error): new
	(_init_checksum_types): new
	(add_bucket): _send_request_expect_nothing => send_request_expect_nothing
	(delete_bucket): likewise

MANIFEST  view on Meta::CPAN

t/00-amazon-s3.t
t/01-api.t
t/02-logger.t
t/03-region.t
t/04-list-buckets.t
t/05-multipart-upload.t
t/06-list-multipart-uploads.t
t/07-crc64nvme.t
t/08-checksums.t
t/09-bucket-v2.t
t/10-endpoint-url.t
t/11-empty-bucket.t
t/12-add-key-sources.t
t/13-response-decode.t
t/lib/S3TestUtils.pm
META.yml                                 Module YAML meta-data (added by MakeMaker)
META.json                                Module JSON meta-data (added by MakeMaker)

README.md  view on Meta::CPAN


# DESCRIPTION

`Amazon::S3` provides a Perl interface to Amazon Simple Storage
Service (S3).

The distribution separates account-level S3 operations from
bucket and object operations.

`Amazon::S3` represents the S3 client and AWS account context. It
manages credentials, request signing, regions, service endpoints,
bucket creation and discovery, and account-level listing operations.

[Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) represents an individual bucket. Object
operations such as uploads, downloads, deletes, ACLs, multipart
uploads, and bucket-scoped listing operations are provided primarily
through that class.

[Amazon::S3::BucketV2](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucketV2) is a subclass of [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) that
provides a more general interface to S3 APIs. It accepts API
parameters, headers, URI parameters, and request payload structures

README.md  view on Meta::CPAN


`Amazon::S3` provides limited support for Amazon S3 directory
buckets.

Directory buckets use the S3 Express One Zone storage class.

`Amazon::S3` can currently create and list directory buckets.
Object access within directory buckets is not yet supported because
that requires creating an S3 Express session and using the resulting
temporary credentials when signing requests to the directory bucket's
Zonal endpoint.

A directory bucket can be created by supplying an availability zone
to `add_bucket()`:

    my $bucket = $s3->add_bucket(
      { bucket            => $bucket_name,
        availability_zone => 'use1-az5',
      }
    );

README.md  view on Meta::CPAN

the S3 error code and message.

Some request, protocol, multipart, checksum verification, and
validation failures always throw exceptions because the operation
cannot safely continue.

The most recent HTTP request and response can be inspected using
`last_request()` and `last_response()`.

These accessors are especially useful when diagnosing signing,
endpoint, header, or protocol problems.

`raise_error` defaults to false to preserve the historical
`Amazon::S3` interface. New applications may prefer to enable it when
they want request failures to be impossible to overlook.

# METHODS AND SUBROUTINES

This section documents methods provided directly by `Amazon::S3`.

Methods implemented by [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) or

README.md  view on Meta::CPAN

- dns\_bucket\_names

    Controls whether virtual-hosted-style bucket names are used when
    possible.

    The default is true.

    A bucket name that cannot be used as a DNS subdomain is placed in the
    request path instead.

- endpoint\_url

    Optional explicit S3 service endpoint.

    When `endpoint_url` is supplied, `Amazon::S3` uses that endpoint as
    specified and does not rewrite the host when the configured region
    changes.

    The `region` setting still controls the AWS signing region.

    This is useful for S3-compatible services, local test environments, and
    other cases where the caller must select the service endpoint explicitly.

    For example:

        my $s3 = Amazon::S3->new(
          { endpoint_url => 'http://localhost:4566',
            region       => 'us-east-1',
            ...
          }
        );

    When `endpoint_url` is not supplied, `Amazon::S3` may derive the
    standard AWS S3 endpoint from the configured region.

    `endpoint_url` and `host` may not both be supplied.

- host

    S3 service endpoint.

    The default is `s3.amazonaws.com`.

    When `region()` is set and the host is a standard Amazon S3 endpoint,
    `Amazon::S3` adjusts the host for the configured region.

    This option can also be used with S3-compatible and local testing
    services.

- level

    Logging level used when `Amazon::S3` creates its default logger.

    The default is `error`.

README.md  view on Meta::CPAN

### errstr

Returns the most recent human-readable error message.

See ["ERROR HANDLING"](#error-handling).

### host

    my $host = $s3->host;

    $s3->host($endpoint);

Gets or sets the configured S3 endpoint.

The constructor default is `s3.amazonaws.com`.

See ["S3-COMPATIBLE SERVICES"](#s3-compatible-services).

### last\_request

Returns the most recent [HTTP::Request](https://metacpan.org/pod/HTTP%3A%3ARequest) generated by `Amazon::S3`.

See ["ERROR HANDLING"](#error-handling).

README.md  view on Meta::CPAN

### get\_default\_region

    my $region = $s3->get_default_region;

Attempts to determine the default AWS region.

The method checks, in order:

1. `AWS_REGION`.
2. `AWS_DEFAULT_REGION`.
3. The EC2 instance metadata availability-zone endpoint.

When an availability zone is obtained from instance metadata, the
zone suffix is removed to derive the region.

If no region can be determined, `us-east-1` is returned.

### get\_logger

    my $logger = $s3->get_logger;

README.md  view on Meta::CPAN

    $s3->region('us-west-2');

Gets or sets the region used by the client.

The region is used for account-level requests and as the default
region assigned to bucket objects when no bucket-specific region is
supplied.

When the configured host uses the standard `s3.amazonaws.com` form,
setting the region also adjusts the host to the regional Amazon S3
endpoint.

The constructor default is `us-east-1`.

### signer

    my $signer = $s3->signer;

Returns the Signature Version 4 signer used for requests.

If a signer was supplied to the constructor, that signer is returned.

README.md  view on Meta::CPAN

    my $response = $s3->list_directory_buckets;

    my $response = $s3->list_directory_buckets(
      { uri_params => \%params,
      }
    );

Lists directory buckets owned by the account.

The optional `uri_params` hash reference is passed as URI parameters
to the S3 Express control endpoint.

The method temporarily switches the client to the S3 Express control
endpoint for the request and restores the previous express-mode state
afterward.

On success, returns the parsed S3 response.

On failure, returns `undef` and records error information on the
client.

See ["DIRECTORY BUCKETS"](#directory-buckets).

## OBJECT LISTING AND VERSIONING

README.md  view on Meta::CPAN

At `trace` level, HTTP request and response information may also be
logged.

Applications should review trace output before retaining or sharing
it. Request and response data may contain sensitive application
information even when authentication values are sanitized.

# S3-COMPATIBLE SERVICES

`Amazon::S3` can be used with S3-compatible services and local S3
implementations by configuring the service endpoint and related
connection options.

The `host`, `secure`, and `dns_bucket_names` settings are commonly
relevant when using a non-AWS endpoint.

S3-compatible implementations may differ from AWS in supported APIs,
request validation, checksum behavior, or edge cases.

The integration tests used during development include LocalStack, but
applications targeting another S3-compatible implementation should
test against that implementation directly.

# COMPARISON TO OTHER PERL S3 MODULES

README.md  view on Meta::CPAN

## Directory Buckets

Directory bucket support is currently limited to account-level create
and list operations.

See ["DIRECTORY BUCKETS"](#directory-buckets).

# TESTING

The distribution includes unit tests and integration tests that
exercise behavior requiring an S3 endpoint.

Run the normal distribution test suite with:

    make test

Integration testing during development includes LocalStack.

See `README-TESTING.md` in the distribution root for test
environment setup, integration-test requirements, and additional
testing instructions.

lib/Amazon/S3.pm  view on Meta::CPAN

    aws_access_key_id
    aws_secret_access_key
    token
    buffer_size
    cache_signer
    checksum_types
    checksum_algorithm
    credentials
    dns_bucket_names
    digest
    endpoint_url
    err
    errstr
    error
    express
    host
    last_request
    last_response
    logger
    log_level
    raise_error

lib/Amazon/S3.pm  view on Meta::CPAN

  my %options = ref $args[0] ? %{ $args[0] } : @args;

  $options{timeout}            //= $DEFAULT_TIMEOUT;
  $options{cache_signer}       //= $FALSE;
  $options{retry}              //= $FALSE;
  $options{express}            //= $FALSE;
  $options{verify_checksums}   //= $TRUE;
  $options{checksum_algorithm} //= 'crc64nvme';
  $options{raise_error}        //= $FALSE;

  croak "ERROR: pass endpoint_url or host but not both\n"
    if $options{endpoint_url} && $options{host};

  if ( my $endpoint_url = $options{endpoint_url} ) {
    @options{qw(secure host)} = _parse_endpoint($endpoint_url);
  }
  elsif ( $options{host} ) {
    if ( $options{host} =~ m{\Ahttps?://}xsm ) {
      warn "host containing a URL is deprecated; use endpoint_url instead\n";

      @options{qw(secure host)} = _parse_endpoint( $options{host} );
    }
    else {
      $options{secure} //= $TRUE;
    }
  }
  else {
    $options{secure} = $TRUE;
    $options{host}   = $DEFAULT_HOST;
  }

lib/Amazon/S3.pm  view on Meta::CPAN

  }

  $self->turn_on_special_retry();

  $self->_init_checksum_types;

  return $self;
}

########################################################################
sub _parse_endpoint {
########################################################################
  my ($endpoint_url) = @_;

  my $uri = URI->new($endpoint_url);

  my $scheme = $uri->scheme;

  croak "ERROR: endpoint_url must include a host\n"
    if !defined $uri->host || !length $uri->host;

  croak "ERROR: endpoint_url must use http or https\n"
    if !defined $scheme || ( $scheme ne 'http' && $scheme ne 'https' );

  croak "ERROR: endpoint_url must not contain a query or fragment\n"
    if defined $uri->query || defined $uri->fragment;

  croak "ERROR: endpoint_url must not contain a path\n"
    if $uri->path && $uri->path ne $SLASH;

  my $secure = $scheme eq 'https' ? $TRUE : $FALSE;
  my $host   = $uri->host_port;

  return ( $secure, $host );
}

########################################################################
sub _init_checksum_types {

lib/Amazon/S3.pm  view on Meta::CPAN

  if (@args) {
    $self->_region( $args[0] );
  }

  $self->get_logger->debug( sub { return 'region: ' . ( $self->_region // $EMPTY ) } );

  if ( $self->_region ) {
    my $host = $self->host;
    $self->get_logger->debug( sub { return 'host: ' . $self->host } );

    # only add region if host was not populated from endpoint_url
    if ( !$self->endpoint_url && $host =~ /\As3[.](.*)?amazonaws/xsm ) {
      $self->host( sprintf 's3.%s.amazonaws.com', $self->_region );
    }
  }

  return $self->_region;
}

########################################################################
sub buckets {
########################################################################
  my ( $self, $verify_region ) = @_;

  # The "default" region for Amazon is us-east-1
  # This is the region to set it to for listing buckets
  # You may need to reset the signer's endpoint to 'us-east-1'

  # temporarily cache signer
  my $region = $self->_region;
  my $bucket_list;

  $self->reset_signer_region($DEFAULT_REGION);  # default region for buckets op

  my $r = $self->send_request(
    { method  => 'GET',
      path    => $EMPTY,

lib/Amazon/S3.pm  view on Meta::CPAN

  }

  my $request = HTTP::Request->new( $method, $url, $http_headers );

  $self->last_request($request);

  if ($data) {
    $request->content($data);
  }

  $self->signer->region($region);  # always set regional endpoint for signing

  $self->signer->sign($request);

  return $request;
}

########################################################################
sub send_request {
########################################################################
  my ( $self, @args ) = @_;

lib/Amazon/S3.pm  view on Meta::CPAN

  return $TRUE
    if $response->is_success();

  # If the error is due to the wrong region, then we will get
  # back a block of XML with the details
  return $FALSE
    if !is_xml_response($response);

  my $error_hash = $self->_xpc_of_content( $response->content );

  my ( $endpoint, $code, $region, $message )
    = @{$error_hash}{qw(Endpoint Code Region Message)};

  my $condition = eval {
    return 'PermanentRedirect'
      if $code eq 'PermanentRedirect' && $endpoint;

    return 'AuthorizationHeaderMalformed'
      if $code eq 'AuthorizationHeaderMalformed' && $region;

    return 'IllegalLocationConstraintException'
      if $code eq 'IllegalLocationConstraintException';

    return 'Other';
  };

  my %error_handlers = (
    PermanentRedirect => sub {
      # Don't recurse through multiple redirects
      return $FALSE
        if $called_from_redirect;

      # With a permanent redirect error, they are telling us the explicit
      # host to use.  The endpoint will be in the form of bucket.host
      my $host = $endpoint;

      # Remove the bucket name from the front of the host name
      # All the requests will need to be of the form https://host/bucket
      $host =~ s/\A$bucket[.]//xsm;
      $self->host($host);

      # We will need to call ourselves again in order to trigger the
      # AuthorizationHeaderMalformed error in order to get the region
      return $self->adjust_region( $bucket, $TRUE );
    },
    AuthorizationHeaderMalformed => sub {
      # Set the signer to use the correct reader evermore
      $self->{signer}->{endpoint} = $region;

      # Only change the host if we haven't been called as a redirect
      # where an exact host has been given
      if ( !$called_from_redirect ) {
        $self->host( sprintf 's3-%s-amazonaws.com', $region );
      }

      return $TRUE;
    },
    IllegalLocationConstraintException => sub {
      # This is hackish; but in this case the region name only appears in the message
      if ( $message =~ /The (\S+) location/xsm ) {
        my $new_region = $1;

        # Correct the region for the signer
        $self->{signer}->{endpoint} = $new_region;

        # Set the proper host for the region
        $self->host( sprintf 's3.%s.amazonaws.com', $new_region );

        return $TRUE;
      }
    },
    'Other' => sub {
      # Some other error
      $self->_remember_errors( $response->content, 1 );

lib/Amazon/S3.pm  view on Meta::CPAN


=head1 DESCRIPTION

C<Amazon::S3> provides a Perl interface to Amazon Simple Storage
Service (S3).

The distribution separates account-level S3 operations from
bucket and object operations.

C<Amazon::S3> represents the S3 client and AWS account context. It
manages credentials, request signing, regions, service endpoints,
bucket creation and discovery, and account-level listing operations.

L<Amazon::S3::Bucket> represents an individual bucket. Object
operations such as uploads, downloads, deletes, ACLs, multipart
uploads, and bucket-scoped listing operations are provided primarily
through that class.

L<Amazon::S3::BucketV2> is a subclass of L<Amazon::S3::Bucket> that
provides a more general interface to S3 APIs. It accepts API
parameters, headers, URI parameters, and request payload structures

lib/Amazon/S3.pm  view on Meta::CPAN


C<Amazon::S3> provides limited support for Amazon S3 directory
buckets.

Directory buckets use the S3 Express One Zone storage class.

C<Amazon::S3> can currently create and list directory buckets.
Object access within directory buckets is not yet supported because
that requires creating an S3 Express session and using the resulting
temporary credentials when signing requests to the directory bucket's
Zonal endpoint.

A directory bucket can be created by supplying an availability zone
to C<add_bucket()>:

  my $bucket = $s3->add_bucket(
    { bucket            => $bucket_name,
      availability_zone => 'use1-az5',
    }
  );

lib/Amazon/S3.pm  view on Meta::CPAN

the S3 error code and message.

Some request, protocol, multipart, checksum verification, and
validation failures always throw exceptions because the operation
cannot safely continue.

The most recent HTTP request and response can be inspected using
C<last_request()> and C<last_response()>.

These accessors are especially useful when diagnosing signing,
endpoint, header, or protocol problems.

C<raise_error> defaults to false to preserve the historical
C<Amazon::S3> interface. New applications may prefer to enable it when
they want request failures to be impossible to overlook.

=head1 METHODS AND SUBROUTINES

This section documents methods provided directly by C<Amazon::S3>.

Methods implemented by L<Amazon::S3::Bucket> or

lib/Amazon/S3.pm  view on Meta::CPAN

=item dns_bucket_names

Controls whether virtual-hosted-style bucket names are used when
possible.

The default is true.

A bucket name that cannot be used as a DNS subdomain is placed in the
request path instead.

=item endpoint_url

Optional explicit S3 service endpoint.

When C<endpoint_url> is supplied, C<Amazon::S3> uses that endpoint as
specified and does not rewrite the host when the configured region
changes.

The C<region> setting still controls the AWS signing region.

This is useful for S3-compatible services, local test environments, and
other cases where the caller must select the service endpoint explicitly.

For example:

  my $s3 = Amazon::S3->new(
    { endpoint_url => 'http://localhost:4566',
      region       => 'us-east-1',
      ...
    }
  );

When C<endpoint_url> is not supplied, C<Amazon::S3> may derive the
standard AWS S3 endpoint from the configured region.

C<endpoint_url> and C<host> may not both be supplied.

=item host

S3 service endpoint.

The default is C<s3.amazonaws.com>.

When C<region()> is set and the host is a standard Amazon S3 endpoint,
C<Amazon::S3> adjusts the host for the configured region.

This option can also be used with S3-compatible and local testing
services.

=item level

Logging level used when C<Amazon::S3> creates its default logger.

The default is C<error>.

lib/Amazon/S3.pm  view on Meta::CPAN

=head3 errstr

Returns the most recent human-readable error message.

See L</ERROR HANDLING>.

=head3 host

  my $host = $s3->host;

  $s3->host($endpoint);

Gets or sets the configured S3 endpoint.

The constructor default is C<s3.amazonaws.com>.

See L</S3-COMPATIBLE SERVICES>.

=head3 last_request

Returns the most recent L<HTTP::Request> generated by C<Amazon::S3>.

See L</ERROR HANDLING>.

lib/Amazon/S3.pm  view on Meta::CPAN

=item 1.

C<AWS_REGION>.

=item 2.

C<AWS_DEFAULT_REGION>.

=item 3.

The EC2 instance metadata availability-zone endpoint.

=back

When an availability zone is obtained from instance metadata, the
zone suffix is removed to derive the region.

If no region can be determined, C<us-east-1> is returned.

=head3 get_logger

lib/Amazon/S3.pm  view on Meta::CPAN

  $s3->region('us-west-2');

Gets or sets the region used by the client.

The region is used for account-level requests and as the default
region assigned to bucket objects when no bucket-specific region is
supplied.

When the configured host uses the standard C<s3.amazonaws.com> form,
setting the region also adjusts the host to the regional Amazon S3
endpoint.

The constructor default is C<us-east-1>.

=head3 signer

  my $signer = $s3->signer;

Returns the Signature Version 4 signer used for requests.

If a signer was supplied to the constructor, that signer is returned.

lib/Amazon/S3.pm  view on Meta::CPAN

  my $response = $s3->list_directory_buckets;

  my $response = $s3->list_directory_buckets(
    { uri_params => \%params,
    }
  );

Lists directory buckets owned by the account.

The optional C<uri_params> hash reference is passed as URI parameters
to the S3 Express control endpoint.

The method temporarily switches the client to the S3 Express control
endpoint for the request and restores the previous express-mode state
afterward.

On success, returns the parsed S3 response.

On failure, returns C<undef> and records error information on the
client.

See L</DIRECTORY BUCKETS>.

=head2 OBJECT LISTING AND VERSIONING

lib/Amazon/S3.pm  view on Meta::CPAN

At C<trace> level, HTTP request and response information may also be
logged.

Applications should review trace output before retaining or sharing
it. Request and response data may contain sensitive application
information even when authentication values are sanitized.

=head1 S3-COMPATIBLE SERVICES

C<Amazon::S3> can be used with S3-compatible services and local S3
implementations by configuring the service endpoint and related
connection options.

The C<host>, C<secure>, and C<dns_bucket_names> settings are commonly
relevant when using a non-AWS endpoint.

S3-compatible implementations may differ from AWS in supported APIs,
request validation, checksum behavior, or edge cases.

The integration tests used during development include LocalStack, but
applications targeting another S3-compatible implementation should
test against that implementation directly.

=head1 COMPARISON TO OTHER PERL S3 MODULES

lib/Amazon/S3.pm  view on Meta::CPAN

=head2 Directory Buckets

Directory bucket support is currently limited to account-level create
and list operations.

See L</DIRECTORY BUCKETS>.

=head1 TESTING

The distribution includes unit tests and integration tests that
exercise behavior requiring an S3 endpoint.

Run the normal distribution test suite with:

  make test

Integration testing during development includes LocalStack.

See F<README-TESTING.md> in the distribution root for test
environment setup, integration-test requirements, and additional
testing instructions.

lib/Amazon/S3/CLI.pm  view on Meta::CPAN

  }

  return $SUCCESS;
}

########################################################################
sub init {
########################################################################
  my ($self) = @_;

  my %endpoint = choose {
    return ( host => $self->get_host )
      if defined $self->get_host;

    return ( endpoint_url => $self->get_endpoint_url )
      if $self->get_endpoint_url;

    return ();
  };

  $self->set_ascii_table(
    eval {
      require Text::ASCIITable;
      1;
    }
  );

lib/Amazon/S3/CLI.pm  view on Meta::CPAN

    $credentials{token}                 = $ENV{AWS_SESSION_TOKEN};
  }

  my $s3 = Amazon::S3->new(
    { %credentials,
      debug            => $ENV{DEBUG},
      raise_error      => $TRUE,
      region           => $self->get_region,
      logger           => $self->get_logger,
      dns_bucket_names => $self->get_dns_bucket_names // $FALSE,
      %endpoint,
      defined $self->get_secure ? ( secure => $self->get_secure ) : (),
    }
  );

  $self->set_s3($s3);

  return;
}

########################################################################

lib/Amazon/S3/CLI.pm  view on Meta::CPAN

    region => 'us-east-1',
    format => 'text',
  );

  my @option_specs = qw(
    availability-zone=s
    bucket-name|b=s
    content-type=s
    debug
    dns-bucket-names
    endpoint-url|u=s
    force|f
    format|F=s
    help|h
    host|H=s
    key|k=s
    modified-since|m=s
    output|o=s
    prefix=s
    profile|p=s
    range|R=s

lib/Amazon/S3/CLI.pm  view on Meta::CPAN


=head1 SYNOPSIS

  # list buckets in tabular format
  amzn-s3-cli list-buckets --profile sandbox --format table

  # list keys in JSON format
  amzn-s3-cli list-keys test-bucket --format json

  # create a new bucket on LocalStack
  amzn-s3-cli create-bucket my-bucket --endpoint-url http://localhost:4566 --profile localstack

  # add a key using the filename as the object name
  amzn-s3-cli add-key --bucket-name my-bucket some-file

  # add a key with a different object name
  amzn-s3-cli add-key --bucket-name my-bucket some-file project-alpha/some-file

  # retrieve an object
  amzn-s3-cli get-key my-bucket project-alpha/some-file

lib/Amazon/S3/CLI.pm  view on Meta::CPAN

=head1 DESCRIPTION

C<Amazon::S3::CLI> provides the C<amzn-s3-cli> command for performing
common Amazon S3 operations using L<Amazon::S3>.

It is intended as a convenient command-line companion to the library,
not as a replacement for the complete AWS CLI or C<aws s3api> command
surface.

The CLI supports common bucket and object operations and can also be
used with S3-compatible services by specifying an alternate endpoint.

=head2 Commands

=over

=item B<add-key>

  amzn-s3-cli add-key bucket-name filename [object-name]

Adds a file to a bucket.

lib/Amazon/S3/CLI.pm  view on Meta::CPAN

Most commands accept the bucket name positionally or through
C<--bucket-name>.

Output-producing commands generally support C<text>, C<json>, and
C<table> formats. Table output requires L<Text::ASCIITable>.

Credentials are discovered through L<Amazon::Credentials> when that
module is installed. Otherwise the CLI uses the standard AWS credential
environment variables.

Use C<--endpoint-url> when working with an alternate S3-compatible
service such as LocalStack.

=head1 OPTIONS

=over

=item B<--availability-zone>

Specifies an Availability Zone when creating an S3 Express One Zone
directory bucket.

lib/Amazon/S3/CLI.pm  view on Meta::CPAN

Specifies the content type used when adding an object.

=item B<--debug>

Enables debug logging.

=item B<--dns-bucket-names>

Enables DNS-style bucket addressing.

=item B<--endpoint-url>, B<-u>

Specifies the complete S3 service endpoint URL.

For example:

  --endpoint-url http://localhost:4566

The URL scheme determines whether HTTP or HTTPS is used.

=item B<--force>, B<-f>

Suppresses confirmation prompts for destructive operations.

=item B<--format>, B<-F>

Selects the output format.

lib/Amazon/S3/CLI.pm  view on Meta::CPAN


=item B<--help>, B<-h>

Displays command help.

=item B<--host>, B<-H>

Specifies the S3 service host using the legacy host configuration
mechanism.

C<--endpoint-url> is preferred for alternate endpoints.

=item B<--key>, B<-k>

Specifies an object key.

=item B<--modified-since>, B<-m>

Supplies an C<If-Modified-Since> condition to C<get-key>.

=item B<--output>, B<-o>

lib/Amazon/S3/CLI.pm  view on Meta::CPAN

  --range 0-1023

=item B<--region>, B<-r>

Specifies the AWS region.

The default is C<us-east-1>.

=item B<--secure>, B<-s>

Controls secure transport when host-based endpoint configuration is
used.

When C<--endpoint-url> is supplied, the URL scheme determines the
transport.

=item B<--version-id>

Specifies an object version ID for C<get-key> or C<delete-key>.

=back

=head1 AUTHENTICATION

lib/Amazon/S3/CLI.pm  view on Meta::CPAN

  amzn-s3-cli list-buckets --profile sandbox

Without L<Amazon::Credentials>, credentials are read from:

  AWS_ACCESS_KEY_ID
  AWS_SECRET_ACCESS_KEY
  AWS_SESSION_TOKEN

=head1 S3-COMPATIBLE SERVICES

An alternate S3 endpoint may be selected with C<--endpoint-url>:

  amzn-s3-cli list-buckets --endpoint-url http://localhost:4566 --profile localstack

This is useful for LocalStack and other S3-compatible services.

=head1 OPTIONAL DEPENDENCIES

=over

=item L<Amazon::Credentials>

Provides credential discovery and named profile support.

lib/Amazon/S3/Signature/V4.pm  view on Meta::CPAN

use parent qw{Net::Amazon::Signature::V4};

########################################################################
sub new {
########################################################################
  my ( $class, @args ) = @_;

  my %options;

  if ( !ref $args[0] ) {
    @options{qw{access_key_id secret endpoint service}} = @args;
  }
  else {
    %options = %{ $args[0] };
  }

  my $region = delete $options{region};
  $options{endpoint} //= $region;

  my $self = $class->SUPER::new( \%options );

  return $self;
}

########################################################################
sub region {
########################################################################
  my ( $self, @args ) = @_;

  if (@args) {
    $self->{endpoint} = $args[0];
  }

  return $self->{endpoint};
}

1;

share/README.md  view on Meta::CPAN


# DESCRIPTION

`Amazon::S3` provides a Perl interface to Amazon Simple Storage
Service (S3).

The distribution separates account-level S3 operations from
bucket and object operations.

`Amazon::S3` represents the S3 client and AWS account context. It
manages credentials, request signing, regions, service endpoints,
bucket creation and discovery, and account-level listing operations.

[Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) represents an individual bucket. Object
operations such as uploads, downloads, deletes, ACLs, multipart
uploads, and bucket-scoped listing operations are provided primarily
through that class.

[Amazon::S3::BucketV2](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucketV2) is a subclass of [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) that
provides a more general interface to S3 APIs. It accepts API
parameters, headers, URI parameters, and request payload structures

share/README.md  view on Meta::CPAN


`Amazon::S3` provides limited support for Amazon S3 directory
buckets.

Directory buckets use the S3 Express One Zone storage class.

`Amazon::S3` can currently create and list directory buckets.
Object access within directory buckets is not yet supported because
that requires creating an S3 Express session and using the resulting
temporary credentials when signing requests to the directory bucket's
Zonal endpoint.

A directory bucket can be created by supplying an availability zone
to `add_bucket()`:

    my $bucket = $s3->add_bucket(
      { bucket            => $bucket_name,
        availability_zone => 'use1-az5',
      }
    );

share/README.md  view on Meta::CPAN

the S3 error code and message.

Some request, protocol, multipart, checksum verification, and
validation failures always throw exceptions because the operation
cannot safely continue.

The most recent HTTP request and response can be inspected using
`last_request()` and `last_response()`.

These accessors are especially useful when diagnosing signing,
endpoint, header, or protocol problems.

`raise_error` defaults to false to preserve the historical
`Amazon::S3` interface. New applications may prefer to enable it when
they want request failures to be impossible to overlook.

# METHODS AND SUBROUTINES

This section documents methods provided directly by `Amazon::S3`.

Methods implemented by [Amazon::S3::Bucket](https://metacpan.org/pod/Amazon%3A%3AS3%3A%3ABucket) or

share/README.md  view on Meta::CPAN

- dns\_bucket\_names

    Controls whether virtual-hosted-style bucket names are used when
    possible.

    The default is true.

    A bucket name that cannot be used as a DNS subdomain is placed in the
    request path instead.

- endpoint\_url

    Optional explicit S3 service endpoint.

    When `endpoint_url` is supplied, `Amazon::S3` uses that endpoint as
    specified and does not rewrite the host when the configured region
    changes.

    The `region` setting still controls the AWS signing region.

    This is useful for S3-compatible services, local test environments, and
    other cases where the caller must select the service endpoint explicitly.

    For example:

        my $s3 = Amazon::S3->new(
          { endpoint_url => 'http://localhost:4566',
            region       => 'us-east-1',
            ...
          }
        );

    When `endpoint_url` is not supplied, `Amazon::S3` may derive the
    standard AWS S3 endpoint from the configured region.

    `endpoint_url` and `host` may not both be supplied.

- host

    S3 service endpoint.

    The default is `s3.amazonaws.com`.

    When `region()` is set and the host is a standard Amazon S3 endpoint,
    `Amazon::S3` adjusts the host for the configured region.

    This option can also be used with S3-compatible and local testing
    services.

- level

    Logging level used when `Amazon::S3` creates its default logger.

    The default is `error`.

share/README.md  view on Meta::CPAN

### errstr

Returns the most recent human-readable error message.

See ["ERROR HANDLING"](#error-handling).

### host

    my $host = $s3->host;

    $s3->host($endpoint);

Gets or sets the configured S3 endpoint.

The constructor default is `s3.amazonaws.com`.

See ["S3-COMPATIBLE SERVICES"](#s3-compatible-services).

### last\_request

Returns the most recent [HTTP::Request](https://metacpan.org/pod/HTTP%3A%3ARequest) generated by `Amazon::S3`.

See ["ERROR HANDLING"](#error-handling).

share/README.md  view on Meta::CPAN

### get\_default\_region

    my $region = $s3->get_default_region;

Attempts to determine the default AWS region.

The method checks, in order:

1. `AWS_REGION`.
2. `AWS_DEFAULT_REGION`.
3. The EC2 instance metadata availability-zone endpoint.

When an availability zone is obtained from instance metadata, the
zone suffix is removed to derive the region.

If no region can be determined, `us-east-1` is returned.

### get\_logger

    my $logger = $s3->get_logger;

share/README.md  view on Meta::CPAN

    $s3->region('us-west-2');

Gets or sets the region used by the client.

The region is used for account-level requests and as the default
region assigned to bucket objects when no bucket-specific region is
supplied.

When the configured host uses the standard `s3.amazonaws.com` form,
setting the region also adjusts the host to the regional Amazon S3
endpoint.

The constructor default is `us-east-1`.

### signer

    my $signer = $s3->signer;

Returns the Signature Version 4 signer used for requests.

If a signer was supplied to the constructor, that signer is returned.

share/README.md  view on Meta::CPAN

    my $response = $s3->list_directory_buckets;

    my $response = $s3->list_directory_buckets(
      { uri_params => \%params,
      }
    );

Lists directory buckets owned by the account.

The optional `uri_params` hash reference is passed as URI parameters
to the S3 Express control endpoint.

The method temporarily switches the client to the S3 Express control
endpoint for the request and restores the previous express-mode state
afterward.

On success, returns the parsed S3 response.

On failure, returns `undef` and records error information on the
client.

See ["DIRECTORY BUCKETS"](#directory-buckets).

## OBJECT LISTING AND VERSIONING

share/README.md  view on Meta::CPAN

At `trace` level, HTTP request and response information may also be
logged.

Applications should review trace output before retaining or sharing
it. Request and response data may contain sensitive application
information even when authentication values are sanitized.

# S3-COMPATIBLE SERVICES

`Amazon::S3` can be used with S3-compatible services and local S3
implementations by configuring the service endpoint and related
connection options.

The `host`, `secure`, and `dns_bucket_names` settings are commonly
relevant when using a non-AWS endpoint.

S3-compatible implementations may differ from AWS in supported APIs,
request validation, checksum behavior, or edge cases.

The integration tests used during development include LocalStack, but
applications targeting another S3-compatible implementation should
test against that implementation directly.

# COMPARISON TO OTHER PERL S3 MODULES

share/README.md  view on Meta::CPAN

## Directory Buckets

Directory bucket support is currently limited to account-level create
and list operations.

See ["DIRECTORY BUCKETS"](#directory-buckets).

# TESTING

The distribution includes unit tests and integration tests that
exercise behavior requiring an S3 endpoint.

Run the normal distribution test suite with:

    make test

Integration testing during development includes LocalStack.

See `README-TESTING.md` in the distribution root for test
environment setup, integration-test requirements, and additional
testing instructions.

t/10-endpoint-url.t  view on Meta::CPAN

use English qw(-no_match_vars);
use Test::More;

our $TRUE         = 1;
our $FALSE        = 0;
our $DEFAULT_HOST = 's3.amazonaws.com';

use_ok(qw(Amazon::S3));

########################################################################
subtest 'endpoint_url' => sub {
########################################################################
  my %options = (
    endpoint_url          => 'http://localhost:4566',
    aws_access_key_id     => 'test',
    aws_secret_access_key => 'test',
  );

  my $s3 = Amazon::S3->new(%options);

  is( $s3->host, 'localhost:4566', 'host' );

  ok( !$s3->secure, 'secure' );
};

########################################################################
subtest 'endpoint_url and host' => sub {
########################################################################
  my %options = (
    endpoint_url          => 'http://localhost:4566',
    aws_access_key_id     => 'test',
    aws_secret_access_key => 'test',
    host                  => 'localstack:4566',
  );

  my $s3 = eval { Amazon::S3->new(%options); };

  like( $EVAL_ERROR, qr/error/i, 'throws' );
};



( run in 4.517 seconds using v1.01-cache-2.11-cpan-1bcd28dbaf1 )