AWS-Signature-V4
view release on metacpan or search on metacpan
# AWS::Signature::V4
A user-agent agnostic implementation of AWS Signature Version 4, for both
the credentials and the X.509 (IAM Roles Anywhere) variants. Pure Perl
5.24 with Moo; `Dist::Zilla` (`[@Milla]`) builds the distribution.
## Running the tests
The dependencies are installed in `local/` by carton, so they have to be
put on the include path:
```shell
prove -l -Ilocal/lib/perl5 t/ # or: carton exec prove -l t/
```
## Conventions
- `unless` is used **only as a statement modifier**. Write the block form
as `if (!$x) { ... }`: the negated block is harder to read and ages
badly when an `else` has to be added later.
- Errors are raised with `fail` from `AWS::Signature::V4::Error`, which
throws an `Ouch`. Code **400** means the caller got the input wrong.
- Examples are self-contained: core modules plus this distribution's own
dependencies, nothing more. They honour `DRY_RUN=1` (print the signed
request, send nothing), take credentials from `AWS_ACCESS_KEY_ID`,
`AWS_SECRET_ACCESS_KEY` and optionally `AWS_SESSION_TOKEN`, ask
`HTTP::Tiny` for `verify_SSL => 1`, and remove the `host` header
because `HTTP::Tiny` insists on setting it itself.
- Anything pasted into a URL's host name — a bucket, a region — is
validated before use: a `/` in it moves the host elsewhere and the
signed request, session token included, would go there.
## Where the plan lives
`TODO.md` carries the work programme and the reasoning behind decisions
{
"abstract" : "User-Agent agnostic AWS Signatures V4 for credentials and X509",
"author" : [
"Flavio Poletti (flavio@polettix.it)"
],
"dynamic_config" : 0,
"generated_by" : "Dist::Milla version v1.0.22, Dist::Zilla version 6.037, CPAN::Meta::Converter version 2.150010",
"license" : [
"apache_2_0"
],
"meta-spec" : {
"url" : "http://search.cpan.org/perldoc?CPAN::Meta::Spec",
---
abstract: 'User-Agent agnostic AWS Signatures V4 for credentials and X509'
author:
- 'Flavio Poletti (flavio@polettix.it)'
build_requires:
File::Temp: '0'
Test2::V0: '0'
Test::Pod: '0'
configure_requires:
Module::Build::Tiny: '0.034'
dynamic_config: 0
generated_by: 'Dist::Milla version v1.0.22, Dist::Zilla version 6.037, CPAN::Meta::Converter version 2.150010'
NAME
AWS::Signature::V4 - User-Agent agnostic AWS Signatures V4 for
credentials and X509
VERSION
This document describes AWS::Signature::V4 version 0.001.
SYNOPSIS
use AWS::Signature::V4;
# traditional variant, based on credentials
my $s = AWS::Signature::V4->new(
service => 'iam', region => 'us-east-1',
credentials => {
access_key_id => $key_id,
secret_access_key => $secret,
session_token => $token, # optional
},
);
# certificate-based variant (IAM Roles Anywhere)
my $x = AWS::Signature::V4->new(
service => 'rolesanywhere', region => 'eu-west-1',
x509 => {
url => 'https://iam.amazonaws.com/?Action=ListUsers',
headers => { 'Content-Type' => 'application/json' },
body => $payload,
);
$ua_request->header($_ => $r->{headers}{$_}) for keys $r->{headers}->%*;
# presigned URL, i.e. signature in the query string; the signer must
# be for the service of the URL, S3 here
my $s3 = AWS::Signature::V4->new(
service => 's3', region => 'us-east-1',
credentials => {
access_key_id => $key_id,
secret_access_key => $secret,
},
);
my $p = $s3->presign(
url => 'https://bucket.s3.amazonaws.com/key',
expires => 3600,
);
my $url = $p->{url};
DESCRIPTION
This module implements the AWS Signature Version 4 algorithm without
being tied to any specific user agent: it does not send anything, it
just takes the pieces of a request (method, URL, headers, body) and
returns what has to be added to it, so that it can be used with
whatever HTTP client is at hand.
Two variants are supported:
* the traditional one, based on credentials (algorithm
AWS4-HMAC-SHA256), with optional session token;
* the one based on X.509 certificates, as used by IAM Roles Anywhere
(algorithms AWS4-X509-RSA-SHA256 and AWS4-X509-ECDSA-SHA256). The
signature is made with the private key that goes with the
certificate, using CryptX (Crypt::PK::RSA and Crypt::PK::ECC) or a
signing function of your own.
Both header-based signing ("sign") and presigned URLs ("presign") are
available, as well as chunked and streaming uploads to S3 (see "sign"
can be overridden (see "new").
INTERFACE
new
my $s = AWS::Signature::V4->new(%args);
Create a signer. service and region are mandatory (for the X.509
variant used with IAM Roles Anywhere, the service is rolesanywhere) and
can only hold letters, digits, ., _ and -; exactly one of credentials
or x509 must be provided.
The classes of this distribution are built with Moo: the constructor
also accepts a hash reference, and every option below is available as a
read-only accessor of the same name (e.g. $s->region).
Options that are undef are treated as missing. This is important: if
you want to pass a false value in an input that accepts a boolean
value, use 0 for false.
The constructor checks the options, including that certificates and
keys can be read and loaded, and that signer is a code reference, so
problems are reported by it and not by the first signature; it does not
check that a certificate is otherwise valid, nor that a key matches it.
credentials and x509 are copied (shallowly), so changing your own hash
afterwards has no effect, but the accessors give back the copies held
by the object: leave them alone. Mind that ->credentials includes the
secret access key. private_key and private_key_password are dropped
from the copy of x509 as soon as the key is loaded, so ->x509 does not
have them. Subclasses and roles work as usual.
The two variants are implemented by two internal classes, one per
option, that this class uses on your behalf:
AWS::Signature::V4::Credentials and AWS::Signature::V4::X509. They are
documented for the record, but you do not need to know about them.
credentials
hash reference with access_key_id, secret_access_key and the optional
session_token. Other keys are an error, so that a misspelled
session_token is not silently ignored. An empty value is like a
missing one: an error for the first two, no token for session_token
(so that an empty AWS_SESSION_TOKEN can be passed as it is).
x509
hash reference with the following keys (others are an error):
the intermediate values of the algorithm, handy for debugging;
chunker
only when streaming: see below.
Chunked and streaming uploads
streaming enables the aws-chunked encoding used for uploads to S3,
where the body is sent in chunks that are signed as they go. It can be
1 or signed (each chunk is signed, credentials variant only) or
unsigned (no chunk signatures, only the request headers are signed,
also OK with X.509). The decoded_content_length, i.e. the size of the
data, is mandatory. sign sets the payload hash to
STREAMING-AWS4-HMAC-SHA256-PAYLOAD (or its variants below), adds
x-amz-decoded-content-length and aws-chunked to Content-Encoding (after
any other encoding, e.g. gzip,aws-chunked, as it is the one applied
last: S3 takes that token off the end and stores what is left, here
gzip), all signed; the Content-Length to provide is the size of the
encoded body, see "encoded_length". S3 wants all chunks but the last
one to be at least 8 KiB. body, body_fh, payload_hash and
percent-encoded (e.g. %2e%2e) are an error, as it is not certain how
AWS reads them; S3 does not normalize at all, so there they are
ordinary characters of the key and /public/%2e%2e/admin is signed as
it is.
* Do not log the results. headers and authorization can be used to
replay the request for up to 15 minutes; a presigned URL is a bearer
token until it expires; headers, canonical_request and presigned URLs
include the session token, if there is one. Treat them as secrets.
* Secrets in memory. The signer keeps the credentials (the secret
access key included) as long as it lives, and ->credentials gives
them back. The text of an X.509 private key and its password are
dropped once the key is loaded. The key derived for signed chunks,
which could sign any request for the same service, region and day, is
kept by the chunker in a closure: no accessor gives back its bytes
and dumping the chunker does not show them, but whoever holds the
chunker can still sign with it through its internals. Do not hand the
chunker, let alone the signer, to code that you would not trust with
the credentials.
* Error messages may include values that the caller provided, with
non-printable characters escaped (e.g. \x{A}), so that they cannot
forge lines in a log.
* Dependencies are listed with their versions in cpanfile.snapshot;
check them regularly against published advisories, e.g. with
CPAN::Audit.
BUGS AND LIMITATIONS
The guess was that S3 strips the `aws-chunked` token from either
position, so that both orders would be accepted and the point would
be moot; the run above confirms the stripping, at least from the
end. It does not say what the other order would do, and there is no
reason to find out while the module sends this one.
`eg/10-s3-content-encoding-probe.pl` settled it. It uploads a
gzipped object with `streaming`, plus a control without any
`Content-Encoding` so that a rejection can be told apart from a
wrong bucket, region or set of credentials; it reads both back,
checks the bytes round-trip, and deletes them again. Run it against
a bucket that can be written to:
```shell
AWS_REGION=eu-west-1 \
AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... \
./eg/10-s3-content-encoding-probe.pl my-bucket
```
It exits zero and prints `verdict: S3 accepted gzip,aws-chunked`
refused, the order is wrong there: swap the last line of the
`$streaming` branch in `lib/AWS/Signature/V4.pm` to
```perl
join ',', 'aws-chunked', @encodings;
```
update the sentence under `sign` in `V4.pod` and the two
`content-encoding` assertions in `t/streaming.t` to match, and run
the probe again to confirm. The report it prints holds no
credentials, no `Authorization` header and neither bucket nor key,
so it is safe to paste into a chat or a bug report.
abstract = User-Agent agnostic AWS Signatures V4 for credentials and X509
license = Apache_2_0
author = Flavio Poletti (flavio@polettix.it)
copyright_holder = Flavio Poletti (flavio@polettix.it)
[@Milla]
[Run::AfterBuild]
run = support/podversion.pl "%d" "%v" "%n"
; authordep Template::Perlish
[PruneFiles]
eg/01-sts-get-caller-identity.pl view on Meta::CPAN
#!/usr/bin/env perl
# The simplest case: a signed POST with credentials taken from the
# environment, sent with HTTP::Tiny (any other user agent would do).
#
# AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... ./01-sts-get-caller-identity.pl
#
# Set DRY_RUN=1 to see the signed request instead of sending it.
use v5.24;
use warnings;
use FindBin '$Bin';
use lib "$Bin/../lib";
use AWS::Signature::V4;
use HTTP::Tiny;
my $signer = AWS::Signature::V4->new(
service => 'sts',
region => 'us-east-1',
credentials => {
access_key_id => $ENV{AWS_ACCESS_KEY_ID},
secret_access_key => $ENV{AWS_SECRET_ACCESS_KEY},
session_token => $ENV{AWS_SESSION_TOKEN}, # optional
},
);
my $url = 'https://sts.amazonaws.com/';
my $body = 'Action=GetCallerIdentity&Version=2011-06-15';
my $r = $signer->sign(
method => 'POST',
eg/01-sts-get-caller-identity.pl view on Meta::CPAN
say "$_: $r->{headers}{$_}" for sort keys $r->{headers}->%*;
say "\n$body";
exit 0;
}
my $response = HTTP::Tiny->new(verify_SSL => 1)->request(POST => $url,
{headers => \%headers, content => $body});
say "$response->{status} $response->{reason}";
say $response->{content};
# the body above explains what went wrong; exit non-zero all the same, so
# that `./01-sts-get-caller-identity.pl && something-else` does the right
# thing when the credentials are not good
die "the request failed\n" unless $response->{success};
eg/02-s3-get-object.pl view on Meta::CPAN
# https://evil.example.com/x.s3... and the signed request, session token
# included, is sent there
die "invalid bucket name: 3 to 63 letters, digits, dots or dashes\n"
unless $bucket =~ m{\A[A-Za-z0-9][A-Za-z0-9.-]{1,61}[A-Za-z0-9]\z};
die "invalid region: letters, digits and dashes only\n"
unless $region =~ m{\A[a-z0-9-]+\z};
my $signer = AWS::Signature::V4->new(
service => 's3',
region => $region,
credentials => {
access_key_id => $ENV{AWS_ACCESS_KEY_ID},
secret_access_key => $ENV{AWS_SECRET_ACCESS_KEY},
session_token => $ENV{AWS_SESSION_TOKEN},
},
);
# virtual-hosted style: the bucket is part of the host name. A bucket whose
# name contains a dot does not match the *.s3.REGION.amazonaws.com
# certificate, so for those use the path-style URL instead (never turn TLS
# verification off): https://s3.$region.amazonaws.com/$bucket/$key
eg/03-s3-put-object-from-file.pl view on Meta::CPAN
# https://evil.example.com/x.s3... and the signed request, session token
# and the file being uploaded included, is sent there
die "invalid bucket name: 3 to 63 letters, digits, dots or dashes\n"
unless $bucket =~ m{\A[A-Za-z0-9][A-Za-z0-9.-]{1,61}[A-Za-z0-9]\z};
die "invalid region: letters, digits and dashes only\n"
unless $region =~ m{\A[a-z0-9-]+\z};
my $signer = AWS::Signature::V4->new(
service => 's3',
region => $region,
credentials => {
access_key_id => $ENV{AWS_ACCESS_KEY_ID},
secret_access_key => $ENV{AWS_SECRET_ACCESS_KEY},
session_token => $ENV{AWS_SESSION_TOKEN},
},
);
open my $fh, '<:raw', $file or die "open('$file'): $!\n";
my $url = "https://$bucket.s3.$region.amazonaws.com/$key";
my $r = $signer->sign(
method => 'PUT',
eg/04-s3-presigned-urls.pl view on Meta::CPAN
# https://evil.example.com/x.s3... and the URLs printed here, which are
# meant to be handed out and used, would point there instead
die "invalid bucket name: 3 to 63 letters, digits, dots or dashes\n"
unless $bucket =~ m{\A[A-Za-z0-9][A-Za-z0-9.-]{1,61}[A-Za-z0-9]\z};
die "invalid region: letters, digits and dashes only\n"
unless $region =~ m{\A[a-z0-9-]+\z};
my $signer = AWS::Signature::V4->new(
service => 's3',
region => $region,
credentials => {
access_key_id => $ENV{AWS_ACCESS_KEY_ID},
secret_access_key => $ENV{AWS_SECRET_ACCESS_KEY},
session_token => $ENV{AWS_SESSION_TOKEN},
},
);
my $url = "https://$bucket.s3.$region.amazonaws.com/$key";
for my $method (qw< GET PUT >) {
my $p = $signer->presign(
method => $method,
eg/05-s3-chunked-upload.pl view on Meta::CPAN
# https://evil.example.com/x.s3... and the signed request, session token
# and the file being uploaded included, is sent there
die "invalid bucket name: 3 to 63 letters, digits, dots or dashes\n"
unless $bucket =~ m{\A[A-Za-z0-9][A-Za-z0-9.-]{1,61}[A-Za-z0-9]\z};
die "invalid region: letters, digits and dashes only\n"
unless $region =~ m{\A[a-z0-9-]+\z};
my $signer = AWS::Signature::V4->new(
service => 's3',
region => $region,
credentials => {
access_key_id => $ENV{AWS_ACCESS_KEY_ID},
secret_access_key => $ENV{AWS_SECRET_ACCESS_KEY},
session_token => $ENV{AWS_SESSION_TOKEN},
},
);
my $chunk_size = 64 * 1024; # S3 wants at least 8 KiB, but the last one
my $size = -s $file;
open my $fh, '<:raw', $file or die "open('$file'): $!\n";
eg/06-dynamodb-json-api.pl view on Meta::CPAN
use FindBin '$Bin';
use lib "$Bin/../lib";
use AWS::Signature::V4;
use HTTP::Tiny;
use JSON::PP qw< encode_json decode_json >;
my $region = shift // $ENV{AWS_REGION} // 'us-east-1';
my $signer = AWS::Signature::V4->new(
service => 'dynamodb',
region => $region,
credentials => {
access_key_id => $ENV{AWS_ACCESS_KEY_ID},
secret_access_key => $ENV{AWS_SECRET_ACCESS_KEY},
session_token => $ENV{AWS_SESSION_TOKEN},
},
);
my $url = "https://dynamodb.$region.amazonaws.com/";
my $body = encode_json({Limit => 10}); # JSON::PP gives bytes, as required
my $r = $signer->sign(
method => 'POST',
eg/07-rolesanywhere-x509.pl view on Meta::CPAN
#!/usr/bin/env perl
# The X.509 variant: instead of a secret key, the request is signed with
# the private key of a certificate, as IAM Roles Anywhere wants, to get
# temporary credentials.
#
# CERT_FILE=cert.pem KEY_FILE=key.pem KEY_TYPE=RSA \
# TRUST_ANCHOR_ARN=... PROFILE_ARN=... ROLE_ARN=... ./07-rolesanywhere-x509.pl
#
# Optional: CHAIN_FILE (PEM bundle of intermediate CAs), KEY_PASSWORD (for
# encrypted keys), AWS_REGION. KEY_TYPE is RSA (default) or ECDSA.
#
# The session goes on the standard output and nothing else does, so that
# it can be piped into jq or saved for later use. It holds temporary
# credentials: a file it is saved in deserves the same care as a key.
#
# NOTE: check the shape of the CreateSession request (path, body) against
# the current IAM Roles Anywhere API reference before relying on it.
use v5.24;
use warnings;
use FindBin '$Bin';
use lib "$Bin/../lib";
use AWS::Signature::V4;
use HTTP::Tiny;
use JSON::PP qw< encode_json >;
eg/07-rolesanywhere-x509.pl view on Meta::CPAN
exit 0;
}
my $response = HTTP::Tiny->new(verify_SSL => 1)->request(POST => $url,
{headers => \%headers, content => $body});
say {*STDERR} "$response->{status} $response->{reason}";
# only a session reaches the standard output, so that "> session.json" or
# a pipe into jq gets JSON and nothing else. An error body is often not
# JSON at all: when the request never got to AWS the status is 599 and
# that field holds the reason as plain text ("Could not connect to ...").
# Saved in the place of a session, it would be a failure kept as if it
# were credentials, and read back as such much later
if (!$response->{success}) {
say {*STDERR} $response->{content}; # this says what went wrong
die "the request failed\n";
}
say {*STDOUT} $response->{content};
eg/09-inspect-a-signature.pl view on Meta::CPAN
# ./09-inspect-a-signature.pl
use v5.24;
use warnings;
use FindBin '$Bin';
use lib "$Bin/../lib";
use AWS::Signature::V4;
my $signer = AWS::Signature::V4->new(
service => 'iam',
region => 'us-east-1',
credentials => {
access_key_id => 'AKIDEXAMPLE',
secret_access_key => 'wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY',
},
);
my $r = $signer->sign(
method => 'GET',
url => 'https://iam.amazonaws.com/?Action=ListUsers&Version=2010-05-08',
headers => {'Content-Type' => 'application/x-www-form-urlencoded; charset=utf-8'},
time => 1440938160, # 2015-08-30T12:36:00Z
);
eg/10-s3-content-encoding-probe.pl view on Meta::CPAN
#
# With streaming, sign() has to add "aws-chunked" to whatever
# Content-Encoding the caller already set. RFC 9110 and botocore put it
# last ("gzip,aws-chunked"), because aws-chunked is applied to the
# already compressed data; the S3 documentation was read the other way
# round in an earlier review of this distribution. See TODO.md: nothing
# but a real bucket can settle it, and this is the program that asks.
#
# Two objects are uploaded: a control with no Content-Encoding of its
# own, and the real case with gzip. The control tells a wrong order apart
# from a wrong bucket, region or set of credentials. Both are deleted
# again, unless KEEP=1.
#
# The report at the end is meant to be pasted in a bug report or a chat:
# it is built from a fixed list of fields, so it carries no credentials,
# no Authorization header, no session token, and neither bucket nor key.
use v5.24;
use warnings;
use FindBin '$Bin';
use lib "$Bin/../lib";
use AWS::Signature::V4;
use HTTP::Tiny;
use IO::Compress::Gzip qw< gzip $GzipError >;
my ($bucket, $prefix) = @ARGV;
eg/10-s3-content-encoding-probe.pl view on Meta::CPAN
die "invalid bucket name: it must be 3 to 63 letters, digits, dots or dashes\n"
unless $bucket =~ m{\A[A-Za-z0-9][A-Za-z0-9.-]{1,61}[A-Za-z0-9]\z};
die "invalid region: it must be letters, digits and dashes\n"
unless $region =~ m{\A[a-z0-9-]+\z};
die "invalid key prefix: use letters, digits, dot, dash, underscore, tilde and /\n"
unless $prefix =~ m{\A[A-Za-z0-9._~/-]+\z};
my $signer = AWS::Signature::V4->new(
service => 's3',
region => $region,
credentials => {
access_key_id => $ENV{AWS_ACCESS_KEY_ID},
secret_access_key => $ENV{AWS_SECRET_ACCESS_KEY},
session_token => $ENV{AWS_SESSION_TOKEN},
},
);
# something that compresses well, so that the gzipped payload is clearly
# not the plain one and a body that came back unzipped cannot be mistaken
# for a good round trip
my $plain = "the quick brown fox jumps over the lazy dog\n" x 100;
eg/10-s3-content-encoding-probe.pl view on Meta::CPAN
return 'cannot tell whether it is free (' . status_of($head) . ')';
}
if (!$ENV{DRY_RUN}) {
for my $c (@cases) {
my $why = key_in_the_way($c->{key}) // next;
die "refusing to use the key '$c->{key}': $why.\n",
"This program overwrites and then deletes the keys it uses, so it\n",
"only touches ones that hold nothing. If that key is in fact free,\n",
"then the check itself failed: verify the bucket, the region and\n",
"the credentials, and grant s3:ListBucket on the bucket so that a\n",
"missing key answers 404 instead of 403. Otherwise pass a\n",
"KEY_PREFIX that points at empty space, e.g.:\n",
" $0 $bucket probe-", time, "/\n";
}
}
for my $c (@cases) {
my $url = $base . $c->{key};
my $r = $signer->sign(
method => 'PUT',
eg/10-s3-content-encoding-probe.pl view on Meta::CPAN
# and a failure is only an answer when S3 itself refused the request. The
# control having gone through says nothing about the one after it: a 503
# SlowDown, a 500, a dropped connection or a key that expired in between
# would otherwise be read as a verdict on the order, and this report
# would send someone to change the signing code over a bad afternoon
my $put_status = $gzip_case->{status} // 0; # unset in a dry run, and
# if the preflight stopped
my $refused = !$gzip_case->{ok}
&& $put_status >= 400 && $put_status < 500
# 401 and 403 are answers about the credentials, not about the header:
# a key that expired between the control and this upload would land
# here, and it is not the ordering that was turned down
&& $put_status != 401 && $put_status != 403
&& defined $gzip_case->{s3_code};
my $status = 0;
if ($ENV{DRY_RUN}) {
push @out, 'verdict: nothing was sent (DRY_RUN): the announced length';
push @out, ' must equal the body size in both cases above';
}
elsif (!$control->{ok}) {
push @out, 'verdict: INCONCLUSIVE. The control upload failed, so this run';
push @out, ' says nothing about the ordering: check the bucket, the';
push @out, ' region and the credentials, then run it again.';
$status = 1;
}
elsif ($confirmed) {
push @out, "verdict: S3 accepted $gzip_case->{sent}, stored it as";
push @out, " Content-Encoding: $gzip_case->{stored}, and gave the";
push @out, ' bytes back unchanged.';
}
elsif ($gzip_case->{ok}) {
push @out, "verdict: INCONCLUSIVE. S3 took $gzip_case->{sent}, but what it";
push @out, ' stored is not what this probe expects: read the stored';
eg/README.md view on Meta::CPAN
# Examples
Small, self-contained programs showing typical usage. Beside the modules
of the distribution (Moo, Ouch and namespace::clean, see `cpanfile`; with
Carton, run them as `carton exec ./eg/...`) they only need Perl core
modules (`HTTP::Tiny` needs `IO::Socket::SSL` for HTTPS), plus CryptX for
the X.509 ones. Run them from a checkout (they load `../lib`).
Set `DRY_RUN=1` to see the signed request instead of sending it. This is
handy to try them out without any AWS access, e.g. with dummy credentials:
```shell
export AWS_ACCESS_KEY_ID=AKIDEXAMPLE AWS_SECRET_ACCESS_KEY=secret DRY_RUN=1
```
Credentials come from `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and
optionally `AWS_SESSION_TOKEN`; the region from `AWS_REGION` where it
makes sense (default `us-east-1`).
`HTTP::Tiny` is used only because it is in the core: the module does not
depend on it, and any other user agent can be used the same way. Mind that
`HTTP::Tiny` wants to set the `Host` header itself, so the examples remove
it from the headers they pass (it is still part of the signature). They
also ask it to verify TLS certificates explicitly (`verify_SSL => 1`),
because versions of `HTTP::Tiny` before 0.083, bundled with older Perls,
do not do it by default: without it, anybody in the middle could read the
signed requests and, in example 07, the temporary credentials.
## 01-sts-get-caller-identity.pl
The simplest case: a signed POST with credentials taken from the
environment. It asks STS who the caller is, so it is also a quick way to
check that the credentials work: it prints the answer either way, and
exits non-zero when the request fails.
```shell
./eg/01-sts-get-caller-identity.pl
DRY_RUN=1 ./eg/01-sts-get-caller-identity.pl
```
## 02-s3-get-object.pl
eg/README.md view on Meta::CPAN
```shell
./eg/06-dynamodb-json-api.pl
./eg/06-dynamodb-json-api.pl eu-west-1
```
## 07-rolesanywhere-x509.pl
The X.509 variant: instead of a secret key, the request is signed with
the private key of a certificate, as IAM Roles Anywhere wants, to get
temporary credentials. `KEY_TYPE` is `RSA` (default) or `ECDSA`;
`CHAIN_FILE` (a PEM bundle of intermediate CAs) and `KEY_PASSWORD` (for
encrypted keys) are optional. `TRUST_ANCHOR_ARN`, `PROFILE_ARN` and
`ROLE_ARN` are required, except with `DRY_RUN=1`, where placeholders
stand in just to show the shape of the request. Beside the
`Authorization` header, the request carries the certificate in
`X-Amz-X509` and the chain, if any, in `X-Amz-X509-Chain`.
```shell
CERT_FILE=cert.pem KEY_FILE=key.pem \
TRUST_ANCHOR_ARN=arn:aws:rolesanywhere:... PROFILE_ARN=arn:aws:rolesanywhere:... \
eg/README.md view on Meta::CPAN
The session JSON is the only thing on the standard output — the
algorithm, the status line, any error body and the `DRY_RUN=1` dump of
the request all go to the standard error — so the program can be used as
`> session.json` or piped into `jq`. A dry run obtains no session and so
writes nothing there either. A failed request prints nothing at all on
the standard output, rather than
an error body where a session was expected: `HTTP::Tiny` reports a
request that never reached AWS as status 599 with the reason, as plain
text, in the body, and that saved under the name of a session would be a
failure kept as if it were credentials. The exit status is non-zero
either way. Mind that what does get saved holds temporary credentials,
so the file deserves the same care as a private key.
`AWS_REGION` is checked here too, for the same reason as in 02 to 05 and
10: it lands in the host name, and a `/` would send the certificate and
its signature somewhere else, with whatever answered read back as a
session.
Check the shape of the `CreateSession` request (path and body) against
the current IAM Roles Anywhere API reference before relying on it: it was
written from memory and has not been tried against AWS.
eg/README.md view on Meta::CPAN
## 10-s3-content-encoding-probe.pl
Not a usage example but a diagnostic probe, kept here because it is built
out of the same pieces as 05. With `streaming`, `sign` adds `aws-chunked`
to whatever `Content-Encoding` the caller already set, and the authorities
disagree on the order: this uploads a gzipped object and reports whether
S3 takes the one the module sends (see the note in `TODO.md`).
It writes two objects to the bucket it is given, a control without any
`Content-Encoding` and the real case with gzip, so that a rejection can
be told apart from a wrong bucket, region or set of credentials. It reads
both back, checks the bytes round-trip, and deletes them again unless
`KEEP=1`. The key prefix defaults to `aws-sigv4-probe/`.
Because it deletes what it uploads, it first checks that both keys hold
nothing and refuses to run otherwise, treating anything but a clear "not
there" as occupied. That check needs `s3:ListBucket` on the bucket:
without it S3 answers a HEAD on a key that does not exist with 403
rather than 404, a free key cannot be told from a forbidden one, and the
program stops and says so. The bucket, the region and the key prefix are also
checked before use: they are pasted into the URL, and a `/` in the bucket
eg/README.md view on Meta::CPAN
KEEP=1 ./eg/10-s3-content-encoding-probe.pl my-bucket
DRY_RUN=1 ./eg/10-s3-content-encoding-probe.pl my-bucket
```
A verdict is only reached when the run earns it. "Accepted" needs more
than a 200: S3 must also have stored the object as `Content-Encoding:
gzip`, having taken the `aws-chunked` token off, and have given the bytes
back unchanged. "Rejected", which tells you to change the order in the
signing code, needs S3 itself to have turned the upload down — a 4xx
carrying an S3 error code, and not one of the 401 and 403 that talk
about the credentials rather than the header. Everything else, a 503, a
dropped connection, a key that expired halfway through, is reported as
inconclusive, because the control having gone through says nothing about
the upload after it. It exits non-zero when the probe reaches no verdict,
or when the order is refused, and then says which line to change. The
report it prints is meant
to be pasted into a bug report or a chat: it is built from a fixed list of
fields, so it holds no credentials, no `Authorization` header, no session
token, and neither the bucket name nor the key.
lib/AWS/Signature/V4.pm view on Meta::CPAN
sort { $a cmp $b } @list;
}
use namespace::clean; # imported or plain functions must not become methods
has service => (is => 'ro');
has region => (is => 'ro');
# credentials: {access_key_id, secret_access_key, session_token?}
has credentials => (is => 'ro', coerce => \&_copy, isa => sub { _hash_or_undef(credentials => @_) });
# Certificate-based variant, as used by IAM Roles Anywhere.
# certificate / certificate_file: PEM or DER text, or path of a file with it
# chain / chain_files: optional arrayref of PEM/DER intermediate certificates,
# or of paths of files with them; a PEM item can be a bundle.
# chain can also be a plain string: a PEM-encoded bundle, and
# chain_files a plain string: the path of one file
# key_type: 'RSA' or 'ECDSA'
# signer: sub ($bytes_to_sign) -> signature (DER for ECDSA, PKCS#1 v1.5
# for RSA), computed with SHA-256; or, alternatively,
lib/AWS/Signature/V4.pm view on Meta::CPAN
# FIXME review this decision: "undef" in Perl has a history of meaning
# "false" but here we're saying that it means "do the default".
# an undefined option is like a missing one: the default applies
around BUILDARGS => sub ($orig, $class, @args) {
my $args = $class->$orig(@args);
defined $args->{$_} or delete $args->{$_}
for qw< double_encode normalize_path payload_header >;
return $args;
};
# The variant in use, credentials or x509, which knows how to sign and what
# goes with the request: see AWS::Signature::V4::Credentials and ::X509
has _auth => (is => 'lazy', init_arg => undef);
sub BUILD ($self, $args) {
for my $name (qw< service region >) {
my $value = $self->$name;
defined($value) || fail 400, qq{missing parameter "$name"};
fail 400, qq{invalid "$name" } . shown($value)
. ': only letters, digits, ".", "_" and "-" are allowed'
unless $value =~ m{\A[-A-Za-z0-9._]+\z};
}
# make sure the caller provides us *exactly* one of credentials/X509
my $x509_params = $self->x509;
my $credentials_params = $self->credentials;
fail 400, 'provide either "credentials" or "x509", not both'
if $credentials_params && $x509_params;
fail 400, 'provide (exactly) one of "credentials" or "x509"'
unless $credentials_params || $x509_params;
$self->_auth; # force building and fail fast
# after using the parameters, we get rid of sensitive data in the input
# hash for X509
delete($x509_params->{$_}) for qw< private_key private_key_password >;
return;
}
sub _build__auth ($self) {
return $self->credentials
? AWS::Signature::V4::Credentials->new($self->credentials->%*)
: AWS::Signature::V4::X509->new($self->x509->%*);
}
sub algorithm ($self) { $self->_auth->algorithm }
# sign(method => ..., url => ..., headers => ..., body => ..., time => ...)
# Returns a hashref:
# headers: all headers to set on the request (name => value)
# including Authorization, X-Amz-Date, Host if missing...
# authorization: value of the Authorization header
lib/AWS/Signature/V4.pm view on Meta::CPAN
my ($unsigned_chunks, @trailer_names);
my $payload_hash;
if ($streaming) {
fail 400, 'streaming must be false, 1, "signed", or "unsigned"'
unless $streaming =~ m{\A(?:1|signed|unsigned)\z};
$unsigned_chunks = $streaming eq 'unsigned';
@trailer_names = _trailer_names(\%args);
fail 400, 'unsigned streaming needs "checksum" or "trailers"'
if $unsigned_chunks && !@trailer_names;
fail 400, 'signed streaming needs the credentials variant, not x509'
if !$unsigned_chunks && !$self->_auth->can_sign_chunks;
my $len = $args{decoded_content_length}
// fail 400, 'streaming needs "decoded_content_length"';
fail 400, 'decoded_content_length must be a non-negative integer'
unless $len =~ m{\A[0-9]+\z};
$payload_hash =
$unsigned_chunks ? 'STREAMING-UNSIGNED-PAYLOAD-TRAILER'
: @trailer_names ? 'STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER'
: 'STREAMING-AWS4-HMAC-SHA256-PAYLOAD';
$h->{'x-amz-decoded-content-length'} = $len;
lib/AWS/Signature/V4.pod view on Meta::CPAN
=pod
=for vim
vim: tw=72 ts=3 sts=3 sw=3 et ai :
=encoding utf8
=head1 NAME
AWS::Signature::V4 - User-Agent agnostic AWS Signatures V4 for credentials and X509
=head1 VERSION
This document describes AWS::Signature::V4 version 0.001.
=head1 SYNOPSIS
use AWS::Signature::V4;
# traditional variant, based on credentials
my $s = AWS::Signature::V4->new(
service => 'iam', region => 'us-east-1',
credentials => {
access_key_id => $key_id,
secret_access_key => $secret,
session_token => $token, # optional
},
);
# certificate-based variant (IAM Roles Anywhere)
my $x = AWS::Signature::V4->new(
service => 'rolesanywhere', region => 'eu-west-1',
x509 => {
lib/AWS/Signature/V4.pod view on Meta::CPAN
url => 'https://iam.amazonaws.com/?Action=ListUsers',
headers => { 'Content-Type' => 'application/json' },
body => $payload,
);
$ua_request->header($_ => $r->{headers}{$_}) for keys $r->{headers}->%*;
# presigned URL, i.e. signature in the query string; the signer must
# be for the service of the URL, S3 here
my $s3 = AWS::Signature::V4->new(
service => 's3', region => 'us-east-1',
credentials => {
access_key_id => $key_id,
secret_access_key => $secret,
},
);
my $p = $s3->presign(
url => 'https://bucket.s3.amazonaws.com/key',
expires => 3600,
);
my $url = $p->{url};
lib/AWS/Signature/V4.pod view on Meta::CPAN
just takes the pieces of a request (method, URL, headers, body) and
returns what has to be added to it, so that it can be used with whatever
HTTP client is at hand.
Two variants are supported:
=over
=item *
the traditional one, based on credentials (algorithm
C<AWS4-HMAC-SHA256>), with optional session token;
=item *
the one based on X.509 certificates, as used by IAM Roles Anywhere
(algorithms C<AWS4-X509-RSA-SHA256> and C<AWS4-X509-ECDSA-SHA256>). The
signature is made with the private key that goes with the certificate,
using L<CryptX> (L<Crypt::PK::RSA> and L<Crypt::PK::ECC>) or a signing
function of your own.
lib/AWS/Signature/V4.pod view on Meta::CPAN
=head1 INTERFACE
=head2 new
my $s = AWS::Signature::V4->new(%args);
Create a signer. C<service> and C<region> are mandatory (for the
X.509 variant used with IAM Roles Anywhere, the service is
C<rolesanywhere>) and can only hold letters, digits, C<.>, C<_> and
C<->; exactly one of C<credentials> or C<x509> must be provided.
The classes of this distribution are built with L<Moo>: the constructor
also accepts a hash reference, and every option below is available as a
read-only accessor of the same name (e.g. C<< $s->region >>).
Options that are C<undef> are treated as missing. B<This is important>:
if you want to pass a false value in an input that accepts a boolean value,
use C<0> for I<false>.
The constructor checks the options, including that certificates and keys
can be read and loaded, and that C<signer> is a code reference, so
problems are reported by it and not by the first signature; it does not
check that a certificate is otherwise valid, nor that a key matches it.
C<credentials> and C<x509> are copied (shallowly), so changing your own
hash afterwards has no effect, but the accessors give back the copies
held by the object: leave them alone. Mind that C<< ->credentials >>
includes the secret access key. C<private_key> and
C<private_key_password> are dropped from the copy of C<x509> as soon as
the key is loaded, so C<< ->x509 >> does not have them. Subclasses and
roles work as usual.
The two variants are implemented by two internal classes, one per
option, that this class uses on your behalf:
L<AWS::Signature::V4::Credentials> and L<AWS::Signature::V4::X509>. They
are documented for the record, but you do not need to know about them.
=over
=item C<credentials>
hash reference with C<access_key_id>, C<secret_access_key> and the
optional C<session_token>. Other keys are an error, so that a misspelled
C<session_token> is not silently ignored. An empty value is like a
missing one: an error for the first two, no token for C<session_token>
(so that an empty C<AWS_SESSION_TOKEN> can be passed as it is).
=item C<x509>
hash reference with the following keys (others are an error):
lib/AWS/Signature/V4.pod view on Meta::CPAN
=item C<chunker>
only when streaming: see below.
=back
=head3 Chunked and streaming uploads
C<streaming> enables the C<aws-chunked> encoding used for uploads to S3,
where the body is sent in chunks that are signed as they go. It can be
C<1> or C<signed> (each chunk is signed, credentials variant only) or
C<unsigned> (no chunk signatures, only the request headers are signed,
also OK with X.509). The C<decoded_content_length>, i.e. the size of
the data, is mandatory. C<sign> sets the payload hash to
C<STREAMING-AWS4-HMAC-SHA256-PAYLOAD> (or its variants below), adds
C<x-amz-decoded-content-length> and C<aws-chunked> to C<Content-Encoding>
(after any other encoding, e.g. C<gzip,aws-chunked>, as it is the one
applied last: S3 takes that token off the end and stores what is left,
here C<gzip>), all signed; the C<Content-Length> to provide is the size
of the encoded body, see L</encoded_length>. S3 wants all chunks but the last
one to be at least 8 KiB. C<body>, C<body_fh>, C<payload_hash> and
lib/AWS/Signature/V4.pod view on Meta::CPAN
=item *
B<Do not log the results.> C<headers> and C<authorization> can be used to
replay the request for up to 15 minutes; a presigned URL is a bearer token
until it expires; C<headers>, C<canonical_request> and presigned URLs
include the session token, if there is one. Treat them as secrets.
=item *
B<Secrets in memory.> The signer keeps the credentials (the secret access
key included) as long as it lives, and C<< ->credentials >> gives them
back. The text of an X.509 private key and its password are dropped once
the key is loaded. The key derived for signed chunks, which could sign
any request for the same service, region and day, is kept by the chunker
in a closure: no accessor gives back its bytes and dumping the chunker
does not show them, but whoever holds the chunker can still sign with it
through its internals. Do not hand the chunker, let alone the signer, to
code that you would not trust with the credentials.
=item *
B<Error messages> may include values that the caller provided, with
non-printable characters escaped (e.g. C<\x{A}>), so that they cannot
forge lines in a log.
=item *
B<Dependencies> are listed with their versions in F<cpanfile.snapshot>;
lib/AWS/Signature/V4/Chunker.pod view on Meta::CPAN
its version.
=head1 SYNOPSIS
# you do not create the chunker: sign() gives it to you
use AWS::Signature::V4;
my $s = AWS::Signature::V4->new(
service => 's3', region => 'eu-west-1',
credentials => { access_key_id => $id, secret_access_key => $secret },
);
my $r = $s->sign(
method => 'PUT', url => $url, streaming => 1,
decoded_content_length => $size,
headers => { 'Content-Length' => AWS::Signature::V4->encoded_length($size, $chunk_size) },
);
my $chunker = $r->{chunker};
print {$socket} $chunker->chunk($_) for @pieces;
print {$socket} $chunker->finish;
lib/AWS/Signature/V4/Credentials.pm view on Meta::CPAN
package AWS::Signature::V4::Credentials;
use v5.24;
use Moo;
use AWS::Signature::V4::Error qw< fail shown >;
use Digest::SHA qw< hmac_sha256 hmac_sha256_hex >;
use experimental qw< signatures >;
use namespace::clean;
# The credentials-based variant (AWS4-HMAC-SHA256). The options are the ones
# of the "credentials" option of AWS::Signature::V4.
has [qw< access_key_id secret_access_key session_token >] => (is => 'ro');
sub BUILD ($self, $args) {
if (my ($name) = sort(grep { !m{\A(?:access_key_id|secret_access_key|session_token)\z} } keys $args->%*)) {
fail 400, 'unknown option "' . shown($name) . '" in credentials';
}
for my $name (qw< access_key_id secret_access_key >) {
my $value = $self->$name; # empty is like missing, AWS would refuse it
defined $value && length $value or fail 400, "missing credentials/$name";
}
return;
}
# ---- what AWS::Signature::V4 wants from a variant --------------------------
sub algorithm ($self) { 'AWS4-HMAC-SHA256' }
# identifies who signs, in the Credential part of the authorization
sub credential_id ($self) { $self->access_key_id }
lib/AWS/Signature/V4/Credentials.pod view on Meta::CPAN
=pod
=for vim
vim: tw=72 ts=3 sts=3 sw=3 et ai :
=encoding utf8
=head1 NAME
AWS::Signature::V4::Credentials - The credentials-based variant of AWS Signature V4
=head1 VERSION
This module is part of the L<AWS::Signature::V4> distribution and shares
its version.
=head1 SYNOPSIS
# you do not create it: AWS::Signature::V4 does, from "credentials"
use AWS::Signature::V4;
my $s = AWS::Signature::V4->new(
service => 'iam', region => 'us-east-1',
credentials => {
access_key_id => $id,
secret_access_key => $secret,
session_token => $token, # optional
},
);
=head1 DESCRIPTION
This class implements the traditional variant of the algorithm
(C<AWS4-HMAC-SHA256>), where the signature is an HMAC computed with a key
derived from the secret access key.
B<You are not supposed to use this module directly.>
L<AWS::Signature::V4> creates an object when it is given the
C<credentials> option, using the same keys, and calls the methods below.
The class is documented for those who work on the distribution, or want
to know how the two variants are kept apart, not as a public interface.
The constructor complains with an L<Ouch> exception (code C<400>) if the
access key or the secret are missing or empty; the session token is
optional, and an empty one is like a missing one.
=head1 THE INTERFACE OF A VARIANT
lib/AWS/Signature/V4/X509.pm view on Meta::CPAN
);
}
# No derived key, so no signed chunks. This is stated twice, on purpose:
# AWS::Signature::V4 asks can_sign_chunks() up front, to refuse the request
# before doing anything else, and signing_key() is what fails if somebody
# asks for the key anyway. If this ever changes, change BOTH, and the
# corresponding tests in t/variants.t.
sub can_sign_chunks ($self) { 0 }
sub signing_key ($self, $scope) {
fail 400, 'signed streaming needs the credentials variant, not x509';
}
1;
use Test2::V0;
use FindBin '$Bin';
use lib "$Bin/../lib";
use AWS::Signature::V4;
use File::Temp qw< tempdir >;
use Math::BigInt;
# Example from the AWS SigV4 documentation (IAM ListUsers)
my $s = AWS::Signature::V4->new(
service => 'iam', region => 'us-east-1',
credentials => {
access_key_id => 'AKIDEXAMPLE',
secret_access_key => 'wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY',
},
);
my $r = $s->sign(
method => 'GET',
url => 'https://iam.amazonaws.com/?Action=ListUsers&Version=2010-05-08',
headers => {'Content-Type' => 'application/x-www-form-urlencoded; charset=utf-8'},
time => 1440938160, # 20150830T123600Z
);
t/canonical.t view on Meta::CPAN
use v5.24;
use utf8;
use experimental 'signatures';
use Test2::V0;
use FindBin '$Bin';
use lib "$Bin/../lib";
use AWS::Signature::V4;
use Digest::SHA qw< sha256_hex >;
my %cred = (access_key_id => 'AKID', secret_access_key => 'SECRET');
my $std = AWS::Signature::V4->new(service => 'service', region => 'us-east-1', credentials => {%cred});
my $s3 = AWS::Signature::V4->new(service => 's3', region => 'us-east-1', credentials => {%cred});
sub canon ($signer, $url, %rest) {
my $r = $signer->sign(method => 'GET', url => $url, time => 1440938160, %rest);
my @lines = split /\n/, $r->{canonical_request}, -1;
return ($lines[1], $lines[2], $r); # path, query, whole result
}
sub path_of ($signer, $url) { (canon($signer, $url))[0] }
sub query_of ($signer, $url) { (canon($signer, $url))[1] }
sub req (@args) { (canon(@args))[2] }
t/canonical.t view on Meta::CPAN
ok !exists $r->{headers}{'x-a'}, 'undefined value: no header';
is $r->{headers}{'x-b'}, '1', 'undefined items of a list are skipped';
# names that differ only in case are joined in a repeatable order
is req($std, 'https://h/', headers => {'X-Foo' => 'a', 'x-foo' => 'b', 'X-FOO' => 'c'})
->{headers}{'x-foo'}, 'c,a,b', 'sorted by name as given';
};
subtest 'x-amz-* headers are always signed' => sub {
my $t = AWS::Signature::V4->new(service => 's3', region => 'r',
credentials => {%cred, session_token => 'TOK'});
my $r = $t->sign(method => 'PUT', url => 'https://b.s3.amazonaws.com/k', time => 0,
headers => {'Content-Type' => 'text/plain', 'X-Amz-Acl' => 'private'},
signed_headers => ['content-type'], body => 'abc');
is $r->{signed_headers},
'content-type;host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-security-token',
'those given and those added by sign';
is req($std, 'https://h/', signed_headers => [])->{signed_headers}, 'host;x-amz-date',
'even with an empty list';
};
t/canonical.t view on Meta::CPAN
'undefined payload_hash: the body is hashed';
$r = req($std, 'https://h/', payload_hash => 'BA7816BF8F01CFEA414140DE5DAE2223B00361A396177A9CB410FF61F20015AD');
like $r->{canonical_request}, qr/\nba7816bf[0-9a-f]{56}\z/, 'hex payload_hash is lowercased';
for my $hash ("x\r\nX-Injected: 1", 'HASH', 'a' x 63, {}) {
is dies { canon($s3, 'https://h/', payload_hash => $hash) }, bad(qr/payload_hash/),
'invalid payload_hash ' . (ref $hash || $hash =~ s/[\r\n]/?/gr);
}
};
subtest 'payload header' => sub {
my $lambda = AWS::Signature::V4->new(service => 'lambda', region => 'r', credentials => {%cred});
my $r = req($lambda, 'https://h/', body => '{}', unsigned_payload => 1);
is $r->{headers}{'x-amz-content-sha256'}, 'UNSIGNED-PAYLOAD', 'unsigned payload: header added';
like $r->{signed_headers}, qr/x-amz-content-sha256/, 'and signed';
$r = req($lambda, 'https://h/', payload_hash => 'STREAMING-X');
is $r->{headers}{'x-amz-content-sha256'}, 'STREAMING-X', 'same for any marker';
my $abc = sha256_hex('abc');
$r = req($s3, 'https://h/', headers => {'X-Amz-Content-Sha256' => $abc});
is $r->{headers}{'x-amz-content-sha256'}, $abc, 'S3: the header given is kept';
like $r->{canonical_request}, qr/\n$abc\z/, 'and used as the payload hash';
t/canonical.t view on Meta::CPAN
like $r->{canonical_request}, qr/\n$abc\z/, 'it can agree with the body';
is dies { canon($lambda, 'https://h/', body => 'abc',
headers => {'X-Amz-Content-Sha256' => 'UNSIGNED-PAYLOAD'}) },
bad(qr/does not match/), 'a header that disagrees with the body is refused';
is dies { canon($s3, 'https://h/', headers => {'X-Amz-Content-Sha256' => 'nope'}) },
bad(qr/x-amz-content-sha256/), 'an invalid header is refused';
};
subtest 'S3 under other names' => sub {
for my $name (qw< s3-object-lambda s3-outposts s3express >) {
my $t = AWS::Signature::V4->new(service => $name, region => 'r', credentials => {%cred});
my ($path, undef, $r) = canon($t, 'https://h/a%20b//c/../d');
is $path, '/a%20b//c/../d', "$name: path verbatim";
is $r->{headers}{'x-amz-content-sha256'}, sha256_hex(''), "$name: payload header";
like $t->presign(url => 'https://h/', time => 0)->{canonical_request},
qr/\nUNSIGNED-PAYLOAD\z/, "$name: presign with UNSIGNED-PAYLOAD";
}
my $t = AWS::Signature::V4->new(service => 's3x', region => 'r', credentials => {%cred});
is path_of($t, 'https://h/a/../b'), '/b', 'but not any name starting with s3';
};
subtest 'body as scalar reference' => sub {
my $body = 'some payload';
my $plain = req($std, 'https://h/', body => $body);
my $byref = req($std, 'https://h/', body => \$body);
is $byref->{signature}, $plain->{signature}, 'same signature by reference';
is $body, 'some payload', 'referenced body untouched';
t/canonical.t view on Meta::CPAN
subtest 'method, scope, token' => sub {
my $r = $std->sign(method => 'post', url => 'https://h/', time => 1440938160);
like $r->{canonical_request}, qr/\APOST\n/, 'method uppercased';
is $r->{scope}, '20150830/us-east-1/service/aws4_request', 'scope';
like $r->{authorization},
qr{^AWS4-HMAC-SHA256 Credential=AKID/20150830/us-east-1/service/aws4_request, SignedHeaders=host;x-amz-date, Signature=[0-9a-f]{64}$},
'authorization header';
my $t = AWS::Signature::V4->new(service => 'service', region => 'r',
credentials => {%cred, session_token => 'TOK'});
$r = $t->sign(method => 'GET', url => 'https://h/', time => 0);
is $r->{headers}{'x-amz-security-token'}, 'TOK', 'session token header';
like $r->{signed_headers}, qr/x-amz-security-token/, 'is signed';
like $r->{headers}{'x-amz-date'}, qr/^19700101T000000Z$/, 'epoch 0';
};
subtest 'constructor errors' => sub {
is dies { AWS::Signature::V4->new(region => 'r', credentials => {%cred}) },
bad(qr/service/), 'no service';
is dies { AWS::Signature::V4->new(service => 's', credentials => {%cred}) },
bad(qr/region/), 'no region';
is dies { AWS::Signature::V4->new(service => 's', region => 'r') },
bad(qr/credentials/), 'no credentials';
is dies { AWS::Signature::V4->new(service => 's', region => 'r', credentials => {access_key_id => 'x'}) },
bad(qr/secret_access_key/), 'missing secret';
is dies { AWS::Signature::V4->new(service => 's', region => 'r', credentials => {%cred, token => 'T'}) },
bad(qr/unknown option "token" in credentials/), 'unknown credentials option';
is dies { AWS::Signature::V4->new(service => 's', region => "us-east-1\r\nX-Evil: 1",
credentials => {%cred}) }, bad(qr/region/), 'invalid region';
is dies { AWS::Signature::V4->new(service => 'a/b', region => 'r', credentials => {%cred}) },
bad(qr/service/), 'invalid service';
};
subtest 'arguments' => sub {
is $std->sign({method => 'GET', url => 'https://h/', time => 0})->{signature},
$std->sign(method => 'GET', url => 'https://h/', time => 0)->{signature},
'a hash reference is the same as a list of pairs';
is dies { $std->sign(method => 'GET', url => 'https://h/', 'time') },
bad(qr/pairs/), 'odd list';
is dies { canon($std, 'https://h/', content => 'x') }, bad(qr/unsupported for sign: "content"/),
t/canonical.t view on Meta::CPAN
for my $url ('https://h/public/%2e%2e/admin', 'https://h/a/%2E/b', 'https://h/a/.%2e/b') {
is dies { canon($std, $url) }, bad(qr/dot segment/), "encoded dot segment: $url";
}
is path_of($s3, 'https://h/public/%2e%2e/admin'), '/public/../admin',
'S3 does not normalize: fine there';
};
subtest 'values that the module puts in headers or in the query' => sub {
my $signer = sub (%creds) {
AWS::Signature::V4->new(service => 'service', region => 'r',
credentials => {%cred, %creds});
};
my $t = $signer->(session_token => "TOKEN\n");
is dies { $t->sign(method => 'GET', url => 'https://h/') },
bad(qr/x-amz-security-token/), 'session token with a newline';
my $k = $signer->(access_key_id => "AKID\r\nX-Evil: 1");
is dies { $k->sign(method => 'GET', url => 'https://h/') },
bad(qr/authorization/), 'access key id with CR/LF';
# wide characters would reach sha256_hex, or _uri_encode in presign, and
# die there instead of being reported as bad input
use v5.24;
use Test2::V0;
use FindBin '$Bin';
use lib "$Bin/../lib";
use experimental 'signatures';
use Digest::SHA qw< sha256 >;
use MIME::Base64 qw< encode_base64 >;
use AWS::Signature::V4;
use AWS::Signature::V4::Chunker;
my %cred = (credentials => {access_key_id => 'a', secret_access_key => 'b'});
my $s = AWS::Signature::V4->new(service => 's3', region => 'r', %cred);
my $secret = 'Hunter2-TopSecret';
sub ouch_like ($code, $status, $name) {
is dies { $code->() },
object { prop blessed => 'Ouch'; call code => $status },
$name;
}
sub streaming ($signer, %opts) {
$signer->sign(method => 'PUT', url => 'http://h/', streaming => 1, %opts);
}
# errors caused by the caller are Ouch exceptions with code 400
my %caller = (
'no service' => sub { AWS::Signature::V4->new(region => 'r', %cred) },
'no region' => sub { AWS::Signature::V4->new(service => 's', %cred) },
'no credentials' => sub { AWS::Signature::V4->new(service => 's', region => 'r') },
'both variants' => sub { AWS::Signature::V4->new(service => 's', region => 'r', %cred, x509 => {}) },
'bad key type' => sub { AWS::Signature::V4->new(service => 's', region => 'r', x509 => {key_type => 'DSA'}) },
'x509 key type under its old name' =>
sub { AWS::Signature::V4->new(service => 's', region => 'r', x509 => {algorithm => 'RSA'}) },
'credentials not a hash' =>
sub { AWS::Signature::V4->new(service => 's', region => 'r', credentials => 'AKID:SECRET') },
'no method' => sub { $s->sign(url => 'http://h/') },
'non-ASCII url' => sub { $s->sign(method => 'GET', url => "http://h/\x{e9}") },
'control chars in url' => sub { $s->sign(method => 'GET', url => "http://h/a\nb") },
'empty url' => sub { $s->sign(method => 'GET', url => '') },
'no host' => sub { $s->presign(url => '/path') },
'wide body' => sub { $s->sign(method => 'PUT', url => 'http://h/', body => "\x{263a}") },
'body_fh not a handle' => sub { $s->sign(method => 'PUT', url => 'http://h/', body_fh => 'file.txt') },
'bad expires' => sub { $s->presign(url => 'http://h/', expires => 0) },
'unknown checksum' => sub { streaming($s, decoded_content_length => 1, checksum => 'md5') },
'finish twice' => sub {
like $t->sign(method => 'GET', url => 'http://h/')->{headers},
{'x-amz-content-sha256' => qr/\A[0-9a-f]{64}\z/}, 'the header is there';
my $u = AWS::Signature::V4->new(
service => 's', region => 'r', %cred,
algorithm => 'RSA', _signer => undef, _x509_serial => 'X',
);
is $u->algorithm, 'AWS4-HMAC-SHA256', 'algorithm cannot be set';
my %mine = (access_key_id => 'a', secret_access_key => 'b');
my $v = AWS::Signature::V4->new(service => 's', region => 'r', credentials => \%mine);
$mine{access_key_id} = 'EVIL';
is $v->credentials->{access_key_id}, 'a', 'credentials are copied';
# a subclass can assign $_ in its builders
{
package Sub::Signer;
use Moo;
extends 'AWS::Signature::V4';
has '+region' => (lazy => 1, builder => 1);
sub _build_region { $_ = 'eu-west-1'; return $_ }
}
is(Sub::Signer->new(service => 's', %cred)->region, 'eu-west-1', 'builders can use $_');
t/presign.t view on Meta::CPAN
use Digest::SHA qw< sha256_hex >;
use MIME::Base64 qw< encode_base64 >;
sub bad ($re = qr/./) {
object { prop blessed => 'Ouch'; call code => 400; call message => match $re };
}
# Example from the AWS docs: presigned GET on S3
my $s3 = AWS::Signature::V4->new(
service => 's3', region => 'us-east-1',
credentials => {
access_key_id => 'AKIAIOSFODNN7EXAMPLE',
secret_access_key => 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY',
},
);
my $r = $s3->presign(
url => 'https://examplebucket.s3.amazonaws.com/test.txt',
expires => 86400,
time => 1369353600, # 20130524T000000Z
);
is $r->{signature},
t/presign.t view on Meta::CPAN
. '?X-Amz-Algorithm=AWS4-HMAC-SHA256'
. '&X-Amz-Credential=AKIAIOSFODNN7EXAMPLE%2F20130524%2Fus-east-1%2Fs3%2Faws4_request'
. '&X-Amz-Date=20130524T000000Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host'
. '&X-Amz-Signature=aeeed9bbccd4d02ee5c0109b86d86835f995330da4c265957d157751f604d404',
'full URL';
is $r->{headers}, {host => 'examplebucket.s3.amazonaws.com'}, 'headers to send';
like $r->{canonical_request}, qr/\nhost\nUNSIGNED-PAYLOAD\z/, 'S3: unsigned payload';
# other services: hash of the empty body, unless told otherwise
my $iam = AWS::Signature::V4->new(service => 'iam', region => 'us-east-1',
credentials => {access_key_id => 'A', secret_access_key => 'S', session_token => 'TOK'});
$r = $iam->presign(url => 'https://iam.amazonaws.com/?Version=1&Action=X', time => 0);
like $r->{canonical_request},
qr/\ne3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\z/,
'non-S3: empty body hash';
like $r->{url}, qr{^https://iam\.amazonaws\.com/\?Action=X&Version=1&X-Amz-Algorithm=}, 'existing params kept';
like $r->{url}, qr/&X-Amz-Security-Token=TOK&/, 'session token goes in the query';
like $r->{url}, qr/&X-Amz-Signature=[0-9a-f]{64}\z/, 'signature is last';
unlike $r->{canonical_request}, qr/x-amz-security-token:/, 'not a header';
$r = $iam->presign(url => 'https://h/', unsigned_payload => 1, time => 0);
t/security.t view on Meta::CPAN
use v5.24;
use Test2::V0;
use FindBin '$Bin';
use lib "$Bin/../lib";
use experimental 'signatures';
use Data::Dumper ();
use File::Temp ();
use AWS::Signature::V4;
my %cred = (access_key_id => 'AKID', secret_access_key => 'SECRET');
my $s3 = AWS::Signature::V4->new(service => 's3', region => 'r', credentials => {%cred});
my $url = 'https://b.s3.amazonaws.com/k';
sub bad_request ($re) {
object { prop blessed => 'Ouch'; call code => 400; call message => match $re };
}
sub dump_of ($x) {
local $Data::Dumper::Deparse = 0;
Data::Dumper->new([$x])->Useqq(1)->Dump;
}
t/streaming.t view on Meta::CPAN
use experimental 'signatures';
use Test2::V0;
use FindBin '$Bin';
use lib "$Bin/../lib";
use AWS::Signature::V4;
use Digest::SHA qw< sha256_hex >;
use File::Temp qw< tempfile >;
my $s3 = AWS::Signature::V4->new(
service => 's3', region => 'us-east-1',
credentials => {
access_key_id => 'AKIAIOSFODNN7EXAMPLE',
secret_access_key => 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY',
},
);
# Example from the AWS docs: PUT with signed chunks (64 KiB, 1 KiB, final)
is(AWS::Signature::V4->encoded_length(66560, 65536), 66824, 'encoded length, as in the docs');
my $r = $s3->sign(
method => 'PUT',
t/streaming.t view on Meta::CPAN
# body_fh: the hash of the file, without loading it
my ($fh, $file) = tempfile(UNLINK => 1);
binmode $fh;
my $data = join '', map { chr($_ % 256) } 1 .. 300_000;
print {$fh} $data;
close $fh;
open my $in, '<:raw', $file or die;
seek $in, 5, 0;
my $std = AWS::Signature::V4->new(service => 'service', region => 'r',
credentials => {access_key_id => 'A', secret_access_key => 'S'});
my $viafh = $std->sign(method => 'PUT', url => 'https://h/', body_fh => $in, time => 0);
my $viabody = $std->sign(method => 'PUT', url => 'https://h/', body => substr($data, 5), time => 0);
is $viafh->{signature}, $viabody->{signature}, 'body_fh hashes from the current position to the end';
is tell($in), 5, 'and the file position is restored';
ok dies { $std->sign(method => 'PUT', url => 'https://h/', body_fh => $in, body => 'x') },
'body and body_fh together are refused';
my $pfh = $std->presign(url => 'https://h/', body_fh => $in, time => 0);
my $pbody = $std->presign(url => 'https://h/', body => substr($data, 5), time => 0);
is $pfh->{signature}, $pbody->{signature}, 'body_fh works with presign too';
t/trailer.t view on Meta::CPAN
use v5.24;
use experimental 'signatures';
use Test2::V0;
use FindBin '$Bin';
use lib "$Bin/../lib";
use AWS::Signature::V4;
use Digest::SHA qw< sha1 sha256 sha256_hex hmac_sha256 hmac_sha256_hex >;
use MIME::Base64 qw< encode_base64 >;
my $s3 = AWS::Signature::V4->new(service => 's3', region => 'us-east-1', credentials => {
access_key_id => 'AKIAIOSFODNN7EXAMPLE',
secret_access_key => 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY'});
my $x = AWS::Signature::V4->new(service => 's3', region => 'r', x509 => {
key_type => 'RSA', serial => 1, signer => sub { 'x' },
certificate => "-----BEGIN CERTIFICATE-----\nMAMCAQc=\n-----END CERTIFICATE-----\n"});
sub bad ($re) {
object { prop blessed => 'Ouch'; call code => 400; call message => match $re };
}
t/trailer.t view on Meta::CPAN
}
# --- configuration errors
ok dies { start($s3, streaming => 'unsigned') }, 'unsigned without trailers';
ok dies { start($s3, streaming => 1, checksum => 'md5') }, 'unknown checksum';
ok dies { start($s3, streaming => 1, checksum => 'crc32', trailers => ['x-amz-checksum-crc32']) }, 'same trailer twice';
ok dies { start($s3, streaming => 1, trailers => ['bad name']) }, 'invalid trailer name';
ok dies { start($s3, checksum => 'crc32') }, 'checksum without streaming';
ok dies { start($s3, trailers => ['x-foo']) }, 'trailers without streaming';
ok dies { start($s3, streaming => 'sideways') }, 'unknown streaming mode';
ok dies { start($x, streaming => 'signed', checksum => 'crc32') }, 'signed chunks need credentials';
ok lives { start($x, streaming => 'unsigned', checksum => 'crc32') }, 'unsigned works with x509';
# --- encoded_length equals the real length
my @cases = (
[signed => 1, {}],
[signed => 1, {checksum => 'crc32c'}, {streaming => 1, checksum => 'crc32c'}],
[signed => 1, {trailers => {'x-amz-checksum-crc64nvme' => 12}}, {streaming => 1, trailers => ['x-amz-checksum-crc64nvme']}, {'x-amz-checksum-crc64nvme' => 'A' x 12}],
[signed => 0, {checksum => 'sha256'}, {streaming => 'unsigned', checksum => 'sha256'}],
[signed => 0, {checksum => 'sha1', trailers => {'x-foo' => 3}}, {streaming => 'unsigned', checksum => 'sha1', trailers => ['x-foo']}, {'x-foo' => 'abc'}],
);
t/variants.t view on Meta::CPAN
use experimental 'signatures';
use MIME::Base64 qw< encode_base64 >;
use AWS::Signature::V4;
use AWS::Signature::V4::Credentials;
use AWS::Signature::V4::X509;
sub bad_request ($name) {
object { prop blessed => 'Ouch'; call code => 400; call message => match qr/\Q$name\E/ };
}
subtest 'credentials' => sub {
my $c = AWS::Signature::V4::Credentials->new(
access_key_id => 'AKIDEXAMPLE',
secret_access_key => 'wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY',
);
is $c->algorithm, 'AWS4-HMAC-SHA256', 'algorithm';
is $c->credential_id, 'AKIDEXAMPLE', 'credential id is the access key';
ok $c->can_sign_chunks, 'it can sign chunks';
is [$c->extra_fields], [], 'no token, no extra fields';
my $scope = '20150830/us-east-1/iam/aws4_request';
t/variants.t view on Meta::CPAN
'signing key, from the AWS documentation';
like $c->signature($scope, 'anything'), qr/\A[0-9a-f]{64}\z/, 'hex signature';
my $t = AWS::Signature::V4::Credentials->new(
access_key_id => 'a', secret_access_key => 'b', session_token => 'TOKEN');
is [$t->extra_fields], ['X-Amz-Security-Token' => 'TOKEN'], 'the session token';
$t = AWS::Signature::V4::Credentials->new(
access_key_id => 'a', secret_access_key => 'b', session_token => '');
is [$t->extra_fields], [], 'an empty session token is no token';
my $s = AWS::Signature::V4->new(service => 'sts', region => 'us-east-1',
credentials => {access_key_id => 'a', secret_access_key => 'b', session_token => ''});
my $r = $s->sign(method => 'GET', url => 'https://sts.amazonaws.com/', time => 0);
ok !exists $r->{headers}{'x-amz-security-token'}, 'no empty token header';
unlike $r->{signed_headers}, qr/security-token/, 'no empty token signed';
unlike $s->presign(url => 'https://sts.amazonaws.com/', time => 0)->{url},
qr/Security-Token/, 'no empty token presigned';
is dies { AWS::Signature::V4::Credentials->new(secret_access_key => 'b') },
bad_request('credentials/access_key_id'), 'access key is needed';
is dies { AWS::Signature::V4::Credentials->new(access_key_id => 'a') },
bad_request('credentials/secret_access_key'), 'secret is needed';
is dies { AWS::Signature::V4::Credentials->new(access_key_id => '', secret_access_key => 'b') },
bad_request('credentials/access_key_id'), 'access key cannot be empty';
is dies { AWS::Signature::V4::Credentials->new(access_key_id => 'a', secret_access_key => '') },
bad_request('credentials/secret_access_key'), 'secret cannot be empty';
};
subtest 'x509' => sub {
my $der = "\x30\x03\x02\x01\x05"; # tiny but well-formed DER
my $chain = "\x30\x03\x02\x01\x06";
my %x = (certificate => $der, serial => 7, signer => sub { "\x01\x02" });
my $x = AWS::Signature::V4::X509->new(key_type => 'ecdsa', %x);
is $x->algorithm, 'AWS4-X509-ECDSA-SHA256', 'algorithm, from the key type';
is $x->credential_id, 7, 'credential id is the serial';
ok !$x->can_sign_chunks, 'it cannot sign chunks';
is dies { $x->signing_key('20150830/us-east-1/iam/aws4_request') },
bad_request('credentials variant'), 'no derived key';
is $x->signature('any/scope', 'anything'), '0102', 'the signature is the hex of the signer output';
is [$x->extra_fields], ['X-Amz-X509' => encode_base64($der, '')], 'the certificate';
$x = AWS::Signature::V4::X509->new(key_type => 'RSA', %x, chain => [$chain, $chain]);
is [$x->extra_fields], [
'X-Amz-X509' => encode_base64($der, ''),
'X-Amz-X509-Chain' => join(',', (encode_base64($chain, '')) x 2),
], 'the certificate and the chain';
for my $case (
t/variants.t view on Meta::CPAN
is dies { AWS::Signature::V4::X509->new(%x, key_type => 'DSA') },
bad_request('RSA or ECDSA'), 'key type is checked';
is dies { AWS::Signature::V4::X509->new(key_type => 'RSA', signer => sub { 1 }) },
bad_request('certificate'), 'certificate is needed';
is dies { AWS::Signature::V4::X509->new(key_type => 'RSA', certificate => $der, serial => 1) },
bad_request('signer'), 'a way to sign is needed';
};
subtest 'the main class gives the same results as the variants' => sub {
my %cred = (access_key_id => 'AKID', secret_access_key => 'secret');
my $s = AWS::Signature::V4->new(service => 'iam', region => 'us-east-1', credentials => {%cred});
my $r = $s->sign(method => 'GET', url => 'https://iam.amazonaws.com/', time => 1440938160);
my $c = AWS::Signature::V4::Credentials->new(%cred);
is $r->{signature}, $c->signature($r->{scope}, $r->{string_to_sign}), 'credentials signature';
is $s->algorithm, $c->algorithm, 'credentials algorithm';
my $x = AWS::Signature::V4->new(
service => 'iam', region => 'us-east-1',
x509 => {key_type => 'RSA', certificate => "\x30\x03\x02\x01\x05", serial => 9,
signer => sub { 'sig' }});
$r = $x->sign(method => 'GET', url => 'https://iam.amazonaws.com/', time => 1440938160);
is $r->{signature}, unpack('H*', 'sig'), 'x509 signature';
like $r->{authorization}, qr{Credential=9/20150830/us-east-1/iam/aws4_request}, 'x509 credential';
ok exists $r->{headers}{'x-amz-x509'}, 'the certificate header';
is $x->algorithm, 'AWS4-X509-RSA-SHA256', 'x509 algorithm';
( run in 2.341 seconds using v1.01-cache-2.11-cpan-85d3896f969 )