Amazon-MWS
view release on metacpan or search on metacpan
Revision history for Amazon::MWS
0.150 Sat Sep 23 13:20:08 2017 CEST
[BUG FIXES]
* Fix error diagnostics for ack response (Marco Pessotto).
* Don't skip failed variants for forced uploads (Stefan Hornburg/Racke).
[ENHANCEMENTS]
* Add API methods (James Risner):
GetInboundGuidanceForASIN
GetInboundGuidanceForSKU
0.141 Mon Dec 5 13:30:14 2016 CET
[ENHANCEMENTS]
* Add GetPrepInstructionsForASIN and GetPrepInstructionsForSKU
API methods provided by James Risner.
0.131 Wed May 25 09:50:20 2016 CEST
[ENHANCEMENTS]
* Allow to limit the product upload steps (Marco Pessotto).
* Removed confirmed => 1 filters to get the orders waiting for shipping
(Marco Pessotto).
* Make _get_pending_jobs method of the uploader public
(Marco Pessotto).
* Allow additional parameters for resume method and keep the jobs
in predictable order (Marco Pessotto).
[DOCUMENTATION]
* Add sections about uploading products, deleting products
and resuming uploads to the main documentation (Stefan Hornburg/Racke).
* Add note about image links to Product class documentation
(Stefan Hornburg/Racke).
* Add comment about usage of Amazon::MWS::Client to the documentation.
(Stefan Hornburg/Racke).
0.130 Sat Apr 2 08:08:26 2016 CEST
[ENHANCEMENTS]
* Add method to retrieve skus with warnings (Marco Pessotto).
t/job-selection.t
t/jobs.txt
t/manifest.t
t/order-reports.t
t/orders.t
t/pod-coverage.t
t/pod.t
t/product-filtering.t
t/shipping-confirmation.t
t/SubmitFeed.t
t/uploader.t
META.yml Module YAML meta-data (added by MakeMaker)
META.json Module JSON meta-data (added by MakeMaker)
DESCRIPTION
See Amazon::MWS::Client for the implementation of the low level Amazon
MWS API.
For the high level methods provided by the Amazon::MWS::Uploader you
are going to need to supply database storage.
Uploading products
Basically you take the products which need to be uploaded to Amazon and
create Amazon::MWS::XML::Product objects from them:
use Amazon::MWS::XML::Product;
use Amazon::MWS::Uploader;
use Try::Tiny;
my @upload;
foreach my $product_object (@send_to_amazon) {
my %prod = ....
try {
my $product = Amazon::MWS::XML::Product->new(%prod);
push @upload, $product;
} catch {
warn "Failure with $prod{sku} : $_";
};
}
Now you create an Amazon::MWS::XML::Uploader object and do the actual
upload.
my $uploader = Amazon::MWS::XML::Uploader->new(...);
try {
$uploader->products(\@upload);
$uploader->upload;
} catch {
die "Uploader failure with: $_";
};
A product upload goes through the following steps:
product
Basic product data.
inventory
Product inventory count (required).
price
Product images (required).
Please note that only http:// links are allowed. If you pass https://
links, they will be rejected by Amazon.
variants
Product variants (optional).
You are not by any means finished with the upload, see "Resume
uploads".
Delete products
Deleting products is much more straightforward, but you have still to
care of tracking the upload process, see again "Resume uploads".
my $uploader = Amazon::MWS::Uploader->new(...);
$uploader->delete_skus(@skus);
Resume uploads
This is necessary to check whether the upload has been processed by
Amazon.
my $uploader = Amazon::MWS::XML::Uploader->new(...);
$uploader->resume;
The uploader also goes automatically through the steps for the product
uploads.
MWS in practice
Product price
Every product uploaded needs a price of 0.01 or higher, otherwise you
get the following error:
0.00 price (standard or sales) will not be accepted.
Please ensure that every SKU in your feed has a price at least equal to or greater than 0.01
Shipping costs
You need to configure the shipping costs in Amazon Seller Central, you
can't pass them through MWS:
https://sellercentral.amazon.com/gp/shipping/dispatch.html
Stuck uploads
There is no guarantee that Amazon finishes your uploads at all. We had
uploads stuck for at least a week.
Multiple marketplaces
You can use this module and the uploader for multiple Amazon
marketplaces. Please make sure that you disable Amazon's
synchronisation between marketplaces.
For marketplaces with a different currency you need to convert your
price first.
The list of marketplaces can be found at:
http://docs.developer.amazonservices.com/en_US/dev_guide/DG_Endpoints.html
Throttling and Quota
With Amazon MWS you have to deal with Amazon throttling your uploads
and imposing quotas.
Possible reasons:
Upload too often
Stuck uploads
Orders with orderlines
Throttle Reponse
<?xml version="1.0"?>
<ErrorResponse xmlns="http://mws.amazonaws.com/doc/2009-01-01/">
<Error>
<Type></Type>
<Code>RequestThrottled</Code>
error: Image file size: 13972730 bytes exceeds the the maximum allowed file size: 10485760 bytes.
There is no point in using such big image files for Amazon.
Error 99001: "brand_name"
Brand is a required field for products.
Uploader Module
Amazon::MWS::Uploader is an upload agent for Amazon::MWS.
XML Modules
Generic Feed
Amazon::MWS::XML::GenericFeed
Feed
Amazon::MWS::XML::Feed
examples/amazon_mws.sql view on Meta::CPAN
feed_name VARCHAR(255) NOT NULL,
feed_file VARCHAR(255) NOT NULL UNIQUE,
processing_complete BOOLEAN NOT NULL DEFAULT FALSE,
aborted BOOLEAN NOT NULL DEFAULT FALSE,
success BOOLEAN NOT NULL DEFAULT FALSE,
errors TEXT,
notes TEXT,
last_updated TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);
-- table to keep track of the uploaded items
CREATE TABLE amazon_mws_products (
-- don't enforce the sku format
sku VARCHAR(255) NOT NULL,
shop_id VARCHAR(64) NOT NULL,
-- given that we just test for equality, don't enforce a type.
-- So an epoch will do just fine, as it would be a random date,
-- as long as the script sends consistent data
timestamp_string VARCHAR(255) NOT NULL DEFAULT '0',
status VARCHAR(32),
lib/Amazon/MWS.pm view on Meta::CPAN
=head1 DESCRIPTION
See L<Amazon::MWS::Client> for the implementation of the low level
Amazon MWS API.
For the high level methods provided by the L<Amazon::MWS::Uploader>
you are going to need to supply database storage.
=head2 Uploading products
Basically you take the products which need to be uploaded to Amazon
and create L<Amazon::MWS::XML::Product> objects from them:
use Amazon::MWS::XML::Product;
use Amazon::MWS::Uploader;
use Try::Tiny;
my @upload;
foreach my $product_object (@send_to_amazon) {
my %prod = ....
try {
my $product = Amazon::MWS::XML::Product->new(%prod);
push @upload, $product;
} catch {
warn "Failure with $prod{sku} : $_";
};
}
Now you create an L<Amazon::MWS::XML::Uploader> object and do the
actual upload.
my $uploader = Amazon::MWS::XML::Uploader->new(...);
try {
$uploader->products(\@upload);
$uploader->upload;
} catch {
die "Uploader failure with: $_";
};
A product upload goes through the following steps:
=over 4
=item product
Basic product data.
=item inventory
Product inventory count (required).
lib/Amazon/MWS.pm view on Meta::CPAN
Please note that B<only http:// links> are allowed. If you pass https://
links, they will be rejected by Amazon.
=item variants
Product variants (optional).
=back
You are not by any means finished with the upload, see L</Resume uploads>.
=head2 Delete products
Deleting products is much more straightforward, but you have still to
care of tracking the upload process, see again L</Resume uploads>.
my $uploader = Amazon::MWS::Uploader->new(...);
$uploader->delete_skus(@skus);
=head2 Resume uploads
This is necessary to check whether the upload has been
processed by Amazon.
my $uploader = Amazon::MWS::XML::Uploader->new(...);
$uploader->resume;
The uploader also goes automatically through the steps for the
product uploads.
=head1 MWS in practice
=head2 Product price
Every product uploaded needs a price of 0.01 or higher, otherwise you
get the following error:
0.00 price (standard or sales) will not be accepted.
Please ensure that every SKU in your feed has a price at least equal to or greater than 0.01
=head2 Shipping costs
You need to configure the shipping costs in Amazon Seller Central, you can't pass them
through MWS:
L<https://sellercentral.amazon.com/gp/shipping/dispatch.html>
=head2 Stuck uploads
There is no guarantee that Amazon finishes your uploads at all. We had uploads
stuck for at least a week.
=head2 Multiple marketplaces
You can use this module and the uploader for multiple Amazon marketplaces.
Please make sure that you disable Amazon's synchronisation between marketplaces.
For marketplaces with a different currency you need to convert your price first.
The list of marketplaces can be found at:
L<http://docs.developer.amazonservices.com/en_US/dev_guide/DG_Endpoints.html>
=head2 Throttling and Quota
With Amazon MWS you have to deal with Amazon throttling your uploads and
imposing quotas.
Possible reasons:
=over 4
=item Upload too often
=item Stuck uploads
=item Orders with orderlines
=back
=head3 Throttle Reponse
<?xml version="1.0"?>
<ErrorResponse xmlns="http://mws.amazonaws.com/doc/2009-01-01/">
<Error>
lib/Amazon/MWS.pm view on Meta::CPAN
error: Image file size: 13972730 bytes exceeds the the maximum allowed file size: 10485760 bytes.
There is no point in using such big image files for Amazon.
=head3 Error 99001: "brand_name"
Brand is a required field for products.
=head1 Uploader Module
L<Amazon::MWS::Uploader> is an upload agent for Amazon::MWS.
=head1 XML Modules
=over 4
=item Generic Feed
L<Amazon::MWS::XML::GenericFeed>
=item Feed
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
our $VERSION = '0.18';
use constant {
AMW_ORDER_WILDCARD_ERROR => 999999,
DEBUG => $ENV{AMZ_UPLOADER_DEBUG},
};
=head1 NAME
Amazon::MWS::Uploader -- high level agent to upload products to AMWS
=head1 DESCRIPTION
This module provide an high level interface to the upload process. It
has to keep track of the state to resume the uploading, which could
get stuck on the Amazon's side processing, so database credentials
have to be provided (or the database handle itself).
The table structure needed is defined and commented in sql/amazon.sql
=head1 SYNOPSIS
my $agent = Amazon::MWS::Uploader->new(
db_dsn => 'DBI:mysql:database=XXX',
db_username => 'xxx',
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
access_key_id => 'xxx',
secret_key => 'xxx',
marketplace_id => 'xxx',
endpoint => 'xxx',
products => \@products,
);
# say once a day, retrieve the full batch and send it up
$agent->upload;
# every 10 minutes or so, continue the work started with ->upload, if any
$agent->resume;
=head1 UPGRADE NOTES
When migrating from 0.05 to 0.06 please execute this SQL statement
ALTER TABLE amazon_mws_products ADD COLUMN listed BOOLEAN;
UPDATE amazon_mws_products SET listed = 1 WHERE status = 'ok';
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
# forse raise error and auto-commit
$options->{RaiseError} = 1;
$options->{AutoCommit} = 1;
my $dbh = DBI->connect($dsn, $self->db_username, $self->db_password,
$options) or die "Couldn't connect to $dsn!";
return $dbh;
}
=item purge_missing_products
If true, the first time C<products_to_upload> is called, products not
passed to the C<products> constructor will be purged from the
C<amazon_mws_products> table. Default to false.
This setting is DEPRECATED because can have some unwanted
side-effects. You are recommended to delete the obsoleted products
yourself.
=cut
has purge_missing_products => (is => 'rw');
=item reset_all_errors
If set to a true value, don't skip previously failed items and
effectively reset all of them.
Also, when the accessor is set for send_shipping_confirmation, try to
upload again previously failed orders.
=cut
has reset_all_errors => (is => 'ro');
=item reset_errors
A string containing a comma separated list of error codes, optionally
prefixed with a "!" (to reverse its meaning).
Example:
"!6024,6023"
Meaning: reupload all the products whose error code is B<not> 6024 or
6023.
"6024,6023"
Meaning: reupload the products whose error code was 6024 or 6023
=cut
has reset_errors => (is => 'ro',
isa => sub {
my $string = $_[0];
# undef/0/'' is fine
if ($string) {
die "reset_errors must be a comma separated list of error code, optionally prefixed by a '!' to negate its meaning"
if $string !~ m/^\s*!?\s*(([0-9]+)(\s*,\s*)?)+/;
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
The directory where the xsd files for the feed building can be found.
=item feeder
A L<Amazon::MWS::XML::Feed> object. Lazy attribute, you shouldn't pass
this to the constructor, it is lazily built using C<products>,
C<merchant_id> and C<schema_dir>.
=item feed_dir
A working directory where to stash the uploaded feeds for inspection
if problems are detected.
=item schema
The L<XML::Compile::Schema> object, built lazily from C<feed_dir>
=item xml_writer
The xml writer, built lazily.
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
=item products
An arrayref of L<Amazon::MWS::XML::Product> objects, or anything that
(properly) responds to C<as_product_hash>, C<as_inventory_hash>,
C<as_price_hash>. See L<Amazon::MWS::XML::Product> for details.
B<This is set as read-write, so you can set the product after the
object construction, but if you change it afterward, you will get
unexpected results>.
This routine also check if the product needs upload and delete
disappeared products. If you are doing the check yourself, use
C<checked_products>.
=item checked_products
As C<products>, but no check is performed. This takes precedence.
=item sqla
Lazy attribute to hold the C<SQL::Abstract> object.
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
my $sth = $self->_exe_query($self->sqla->select(amazon_mws_products => [qw/sku
timestamp_string
status
listed
error_code
/],
{
status => { -not_in => [qw/deleted/] },
shop_id => $self->_unique_shop_id,
}));
my %uploaded;
while (my $row = $sth->fetchrow_hashref) {
$row->{timestamp_string} ||= 0;
$uploaded{$row->{sku}} = $row;
}
return \%uploaded;
}
has products_to_upload => (is => 'lazy');
has checked_products => (is => 'rw', isa => ArrayRef);
sub _build_products_to_upload {
my $self = shift;
if (my $checked = $self->checked_products) {
return $checked;
}
my $product_arrayref = $self->products;
return [] unless $product_arrayref && @$product_arrayref;
my @products = @$product_arrayref;
my $existing = $self->existing_products;
my @todo;
foreach my $product (@products) {
my $sku = $product->sku;
if (my $exists = $existing->{$sku}) {
# mark the item as visited
$exists->{_examined} = 1;
}
print "Checking $sku\n" if $self->debug;
next unless $self->product_needs_upload($product->sku, $product->timestamp_string);
print "Scheduling product " . $product->sku . " for upload\n";
if (my $limit = $self->limit_inventory) {
my $real = $product->inventory;
if ($real > $limit) {
print "Limiting the $sku inventory from $real to $limit\n" if $self->debug;
$product->inventory($limit);
}
}
if (my $children = $product->children) {
my @good_children;
foreach my $child (@$children) {
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
shop_size => qr{size$merchant_re},
amazon_size => qr{size$amazon_re},
);
return \%patterns;
}
=head1 MAIN METHODS
=head2 upload
If the products is set, begin the routine to upload them. Because of
the asynchronous way AMWS works, at some point it will bail out,
saving the state in the database. You should reinstantiate the object
and call C<resume> on it every 10 minutes or so.
The workflow is described here:
L<http://docs.developer.amazonservices.com/en_US/feeds/Feeds_Overview.html>
This has to be done for each feed: Product, Inventory, Price, Image,
Relationship (for variants).
This method first generate the feeds in the feed directory, and then
calls C<resume>, which is in charge for the actual uploading.
=head2 resume
Restore the state and resume where it was left.
This method accepts an optional list of parameters. Each parameter may be:
=over 4
=item a scalar
This is considered a job id.
=item a hashref
This will be merged in the query to retrieve the pending jobs. A
sample usage could be:
$upload->resume({ task => [qw/upload product_deletion/] });
to resume only those specific tasks.
=back
=head2 get_pending_jobs
Return the list of hashref with the pending jobs out of the database.
Accepts the same parameters as C<resume> (which actually calls this
method).
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
sub _slurp_file {
my ($self, $file) = @_;
open (my $fh, '<', $file) or die "Couldn't open $file $!";
local $/ = undef;
my $content = <$fh>;
close $fh;
return $content;
}
sub upload {
my $self = shift;
# create the feeds to be uploaded using the products
my @products = @{ $self->products_to_upload };
unless (@products) {
print "No products, can't upload anything\n";
return;
}
my $feeder = Amazon::MWS::XML::Feed->new(
products => \@products,
xml_writer => $self->xml_writer,
merchant_id => $self->merchant_id,
);
my @feeds;
foreach my $feed_name (qw/product
inventory
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
variants
/) {
my $method = $feed_name . "_feed";
if (my $content = $feeder->$method) {
push @feeds, {
name => $feed_name,
content => $content,
};
}
}
if (my $job_id = $self->prepare_feeds(upload => \@feeds)) {
$self->_mark_products_as_pending($job_id, @products);
return $job_id;
}
return;
}
sub _mark_products_as_pending {
my ($self, $job_id, @products) = @_;
die "Bad usage" unless $job_id;
# these skus were cleared up when asking for the products to upload
foreach my $p (@products) {
my %identifier = (
sku => $p->sku,
shop_id => $self->_unique_shop_id,
);
my %data = (
amws_job_id => $job_id,
status => 'pending',
warnings => '', # clear out
timestamp_string => $p->timestamp_string,
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
shop_id => $self->_unique_shop_id,
},
{ -asc => 'job_started_epoch'});
my $pending = $self->_exe_query($stmt, @bind);
my %jobs;
while (my $row = $pending->fetchrow_hashref) {
$jobs{$row->{task}} ||= [];
push @{$jobs{$row->{task}}}, $row;
}
my @out;
foreach my $task (qw/product_deletion upload shipping_confirmation order_ack/) {
if (my $list = delete $jobs{$task}) {
if ($task eq 'order_ack') {
for (1..2) {
push @out, pop @$list if @$list;
}
}
elsif ($task eq 'shipping_confirmation') {
while (@$list) {
push @out, pop @$list;
}
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
amws_job_id => $job_id,
shop_id => $self->_unique_shop_id,
}));
# and revert the products' status
my $status;
if ($task eq 'product_deletion') {
# let's pretend we were deleting good products
$status = 'ok';
}
elsif ($task eq 'upload') {
$status = 'redo';
}
if ($status) {
print "Updating product to $status for products with job id $job_id\n";
$self->_exe_query($self->sqla->update('amazon_mws_products',
{ status => $status },
{
amws_job_id => $job_id,
shop_id => $self->_unique_shop_id,
}));
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
=head2 process_feeds(\%job_row)
Given the hashref with the db row of the job, check at which point it
is and resume.
=cut
sub process_feeds {
my ($self, $row) = @_;
# print Dumper($row);
# upload the feeds one by one and stop if something is blocking
my $job_id = $row->{amws_job_id};
print "Processing job $job_id\n";
# query the feeds table for this job
my ($stmt, @bind) = $self->sqla->select(amazon_mws_feeds => '*',
{
amws_job_id => $job_id,
aborted => 0,
success => 0,
shop_id => $self->_unique_shop_id,
},
['amws_feed_pk']);
my $sth = $self->_exe_query($stmt, @bind);
my $unfinished;
while (my $feed = $sth->fetchrow_hashref) {
last unless $self->upload_feed($feed);
}
$sth->finish;
($stmt, @bind) = $self->sqla->select(amazon_mws_feeds => '*',
{
shop_id => $self->_unique_shop_id,
amws_job_id => $job_id,
});
$sth = $self->_exe_query($stmt, @bind);
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
$update = {
aborted => 1,
status => 'Feed error',
};
$self->_print_or_warn_error("Job $job_id aborted!\n");
}
elsif ($success == $total) {
$update = { success => 1 };
print "Job successful!\n";
# if we're here, all the products are fine, so mark them as
# such if it's an upload job
if ($row->{task} eq 'upload') {
$self->_exe_query($self->sqla->update('amazon_mws_products',
{ status => 'ok',
listed_date => DateTime->now,
listed => 1,
},
{
amws_job_id => $job_id,
shop_id => $self->_unique_shop_id,
}));
}
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
}
if ($update) {
$self->_exe_query($self->sqla->update(amazon_mws_jobs => $update,
{
amws_job_id => $job_id,
shop_id => $self->_unique_shop_id,
}));
}
}
=head2 upload_feed($type, $feed_id);
Routine to upload the feed. Return true if it's complete, false
otherwise.
=cut
sub upload_feed {
my ($self, $record) = @_;
my $job_id = $record->{amws_job_id};
my $type = $record->{feed_name};
my $feed_id = $record->{feed_id};
print "Checking $type feed for $job_id\n";
# http://docs.developer.amazonservices.com/en_US/feeds/Feeds_FeedType.html
my %names = (
product => '_POST_PRODUCT_DATA_',
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
warn "Order $existing->{amazon_order_id} uncompletely registered with id $existing->{shop_order_id}, please indagate why (skipping)\n" . Dumper($existing);
}
}
else {
push @orders_to_register, $ord;
}
}
return unless @orders_to_register;
my $feed_content = $self->acknowledge_feed(Success => @orders_to_register);
# here we have only one feed to upload and check
my $job_id = $self->prepare_feeds(order_ack => [{
name => 'order_ack',
content => $feed_content,
}]);
# store the pairing amazon order id / shop order id in our table
foreach my $order (@orders_to_register) {
my %order_pairs = (
shop_id => $self->_unique_shop_id,
amazon_order_id => $order->amazon_order_number,
# this will die if we try to insert an undef order_number
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
sku => $sku,
shop_id => $self->_unique_shop_id,
}));
print "Scheduling $sku for redoing\n";
}
}
}
=head2 skus_in_job($job_id)
Check the amazon_mws_product for the SKU which were uploaded by the
given job ID.
=cut
sub skus_in_job {
my ($self, $job_id) = @_;
my $sth = $self->_exe_query($self->sqla->select('amazon_mws_products',
[qw/sku/],
{
amws_job_id => $job_id,
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
OrderFulfillment => $order->as_shipping_confirmation_hashref,
};
}
return $feeder->create_feed(OrderFulfillment => \@messages);
}
=head2 send_shipping_confirmation($shipped_orders)
Schedule the shipped orders (an L<Amazon::MWS::XML::ShippedOrder>
object) for the uploading.
=head2 order_already_shipped($shipped_order)
Check if the shipped orders (an L<Amazon::MWS::XML::ShippedOrder> was
already notified as shipped looking into our table, returning the row
with the order.
To see the status, check shipping_confirmation_ok (already done),
shipping_confirmation_error (faulty), shipping_confirmation_job_id (pending).
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
}
else {
die "It looks like you are trying to send a shipping confirmation "
. " without prior order acknowlegdement. "
. "At least in the amazon_mws_orders there is no trace of "
. "$report->{amazon_order_id} $report->{shop_order_id}";
}
}
return unless @orders_to_notify;
my $feed_content = $self->shipping_confirmation_feed(@orders_to_notify);
# here we have only one feed to upload and check
my $job_id = $self->prepare_feeds(shipping_confirmation => [{
name => 'shipping_confirmation',
content => $feed_content,
}]);
# and store the job id in the table
foreach my $ord (@orders_to_notify) {
$self->_exe_query($self->sqla->update(amazon_mws_orders => {
shipping_confirmation_job_id => $job_id,
shipping_confirmation_error => undef,
},
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
# do not stop the unconfirmed to be considered
# confirmed => 1,
}));
my @out;
while (my $row = $sth->fetchrow_hashref) {
push @out, $row;
}
return @out;
}
=head2 product_needs_upload($sku, $timestamp)
Lookup the product $sku with timestamp $timestamp and return the sku
if the product needs to be uploaded or can be safely skipped. This
method is stateless and doesn't alter anything.
=cut
sub product_needs_upload {
my ($self, $sku, $timestamp) = @_;
my $debug = $self->debug;
return unless $sku;
my $forced = $self->_force_hashref;
# if it's forced, we have nothing to check, just pass it.
if ($forced->{$sku}) {
print "Forcing $sku as requested\n" if $debug;
return $sku;
}
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
return;
}
}
elsif ($status eq 'pending') {
print "Skipping pending item $sku\n" if $debug;
return;
}
die "I shouldn't have reached this point with status <$status>";
}
}
print "$sku wasn't uploaded so far, scheduling it\n" if $debug;
return $sku;
}
=head2 orders_in_shipping_job($job_id)
Lookup the C<amazon_mws_orders> table and return a list of
C<amazon_order_id> for the given shipping confirmation job. INTERNAL.
=cut
lib/Amazon/MWS/Uploader.pm view on Meta::CPAN
=cut
sub purge_old_jobs {
my ($self, $limit) = @_;
unless (defined $limit) {
$limit = 500;
}
my $range = time() - $self->order_ack_days_timeout * 60 * 60 * 24;
my @and = (
task => [qw/product_deletion
upload/],
job_started_epoch => { '<', $range },
[ -or => {
aborted => 1,
success => 1,
},
],
);
if (my $shop_id = $self->shop_id) {
push @and, shop_id => $shop_id;
}
lib/Amazon/MWS/XML/Feed.pm view on Meta::CPAN
=head2 price_feed_name
=cut
sub price_feed {
return shift->_create_feed('Price');
}
=head2 image_feed
The Image feed allows you to upload various images for a product.
Amazon can display several images for each product. It is in your best
interest to provide several high-resolution images for each of your
products so customers can make informed buying decisions.
=head3 Image Requirements
=over 4
=item Format - photographs, not drawings
t/error-parsing.t view on Meta::CPAN
secret_key => '123412341234',
marketplace_id => '123412341234',
endpoint => 'https://mws-eu.amazonservices.com',
feed_dir => 't/feeds',
schema_dir => 'schemas',
);
plan skip_all => "Missing schema and feed dirs"
unless (-d $constructor{schema_dir} && -d $constructor{feed_dir});
my $uploader = Amazon::MWS::Uploader->new(%constructor);
my $error_msg = q{upload-2016-03-14-19-07-09 8541 The SKU data provided conflicts with the Amazon catalog. The standard_product_id value(s) provided correspond to the ASIN XXXXXX, but some information contradicts with the Amazon catalog. The followi...
is_deeply($uploader->_parse_error_message_mismatches($error_msg),
{
asin => 'XXXXXX',
shop_part_number => 'MERCHANT_ID',
amazon_part_number => 'AMAZON_ID',
part_number => {
shop => 'MERCHANT_ID',
amazon => 'AMAZON_ID',
}
});
t/job-selection.t view on Meta::CPAN
endpoint => 'https://mws-eu.amazonservices.com',
feed_dir => $feed_dir,
schema_dir => 'schemas',
db_dsn => 'dbi:SQLite:dbname=t/test.db',
db_username => '',
db_password => '',
);
my $uploader = Amazon::MWS::Uploader->new(%constructor);
ok ($uploader->dbh);
my $create_table =<<'SQL';
CREATE TABLE amazon_mws_jobs (
amws_job_id VARCHAR(64) NOT NULL,
shop_id VARCHAR(64) NOT NULL,
task VARCHAR(64) NOT NULL,
-- if complete one or those has to be set.
aborted BOOLEAN NOT NULL DEFAULT FALSE,
success BOOLEAN NOT NULL DEFAULT FALSE,
last_updated TIMESTAMP,
job_started_epoch INTEGER,
t/job-selection.t view on Meta::CPAN
-- this can be null
amws_job_id VARCHAR(64) REFERENCES amazon_mws_jobs(amws_job_id),
error_code integer NOT NULL DEFAULT '0',
error_msg TEXT,
listed_date DATETIME,
-- our update
last_updated TIMESTAMP,
PRIMARY KEY (sku, shop_id)
);
SQL
$uploader->dbh->do('DROP TABLE IF EXISTS amazon_mws_jobs');
$uploader->dbh->do($create_table) or die;
$uploader->dbh->do('DROP TABLE IF EXISTS amazon_mws_products');
$uploader->dbh->do($create_p_table) or die;
my $dbh = $uploader->dbh;
my @populate = get_sample_records('t/jobs.txt');
my $pop_sth = $dbh->prepare("INSERT INTO amazon_mws_jobs (amws_job_id, shop_id, task, aborted, success, last_updated, job_started_epoch, status) values (?, ?, ?, 0, 0, ?, ?, NULL)");
$dbh->begin_work;
foreach my $sample (@populate) {
$pop_sth->execute(@$sample);
}
$dbh->commit;
t/job-selection.t view on Meta::CPAN
my @things;
while ($row =~ m/(?<=\|)\s*(.+?)\s*(?=\|)/g) {
my $v = $1;
push @things, $v;
}
push @records, \@things if @things;
}
close $fh;
return @records;
}
my @jobs = $uploader->get_pending_jobs;
ok (@jobs > 0, "Found " . scalar(@jobs) . " jobs\n");
# diag Dumper([$uploader->get_pending_jobs]);
is $jobs[0]->{task}, 'product_deletion', "First job is product_deletion";
is ((grep { $_->{task} ne 'product_deletion' } @jobs)[0]{task}, 'upload',
"next in line is upload");
is $jobs[$#jobs]{task}, 'order_ack', "Last is order_ack";
ok (scalar(grep { $_->{task} eq 'shipping_confirmation' } @jobs), "Found shipconfirms");
# db is bogus, will just remove them
{
my @named = $uploader->get_pending_jobs({ task => [qw/upload/] });
ok (scalar(@named), "Found jobs");
is (scalar(grep { $_->{task} ne 'upload' } @named), 0, "Only upload found");
diag "Resuming all uploads";
$uploader->resume({ task => 'upload' });
}
{
my @named = $uploader->get_pending_jobs('product_deletion-2016-04-20-22-00-08');
is (scalar(@named), 1, "Found a single job");
is $named[0]{amws_job_id}, 'product_deletion-2016-04-20-22-00-08';
diag "Resuming a single job";
$uploader->resume('product_deletion-2016-04-20-22-00-08');
}
diag "Doing ship confirm and deletion";
$uploader->resume({ task => [qw/shipping_confirmation product_deletion/] });
diag "Resuming everything";
$uploader->resume;
@jobs = grep { $_->{task} ne 'order_ack' } $uploader->get_pending_jobs;
ok (!@jobs, "No regular jobs expected now");
@jobs = $uploader->get_pending_jobs;
ok (@jobs, "But there are still order_ack jobs pending");
| product_deletion-2016-04-21-02-08-06 | shoppe_it | product_deletion | 2016-04-21 04:08:06 | 1461204486 |
| product_deletion-2016-04-21-02-13-06 | shoppe_es | product_deletion | 2016-04-21 04:13:06 | 1461204786 |
| product_deletion-2016-04-21-02-47-07 | shoppe_fr | product_deletion | 2016-04-21 04:47:07 | 1461206827 |
| product_deletion-2016-04-21-03-08-07 | shoppe_it | product_deletion | 2016-04-21 05:08:07 | 1461208087 |
| product_deletion-2016-04-21-03-14-07 | shoppe_es | product_deletion | 2016-04-21 05:14:07 | 1461208447 |
| product_deletion-2016-04-21-03-48-07 | shoppe_fr | product_deletion | 2016-04-21 05:48:07 | 1461210487 |
| product_deletion-2016-04-21-04-08-06 | shoppe_it | product_deletion | 2016-04-21 06:08:06 | 1461211686 |
| product_deletion-2016-04-21-04-12-07 | shoppe_uk | product_deletion | 2016-04-21 06:12:07 | 1461211927 |
| shipping_confirmation-2016-04-19-07-01-11 | shoppe | shipping_confirmation | 2016-04-19 09:01:11 | 1461049271 |
| shipping_confirmation-2016-04-19-09-01-08 | shoppe | shipping_confirmation | 2016-04-19 11:01:08 | 1461056468 |
| upload-2016-04-18-10-00-09 | shoppe | upload | 2016-04-18 12:00:09 | 1460973609 |
| upload-2016-04-18-10-30-08 | shoppe | upload | 2016-04-18 12:30:08 | 1460975408 |
| upload-2016-04-18-11-00-09 | shoppe | upload | 2016-04-18 13:00:09 | 1460977209 |
| upload-2016-04-18-11-30-09 | shoppe | upload | 2016-04-18 13:30:09 | 1460979009 |
t/order-reports.t view on Meta::CPAN
my %constructor = (
merchant_id => '__MERCHANT_ID__',
access_key_id => '12341234',
secret_key => '123412341234',
marketplace_id => '123412341234',
endpoint => 'https://mws-eu.amazonservices.com',
schema_dir => 'schemas',
feed_dir => File::Spec->catdir(qw/t feeds/),
);
my $uploader = Amazon::MWS::Uploader->new(%constructor);
ok($uploader);
my $xml = <<'AMAZONXML';
<AmazonEnvelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="amzn-envelope.xsd">
<Header>
<DocumentVersion>1.01</DocumentVersion>
<MerchantIdentifier>XXXXX_666666666</MerchantIdentifier>
</Header>
<MessageType>OrderReport</MessageType>
<Message>
<MessageID>1</MessageID>
t/order-reports.t view on Meta::CPAN
<Type>Shipping</Type>
<Amount currency="USD">0.00</Amount>
</Component>
</Promotion>
</Item>
</OrderReport>
</Message>
</AmazonEnvelope>
AMAZONXML
my @orders = ($uploader->_parse_order_reports_xml($xml), $uploader->_parse_order_reports_xml($xml_doc));
ok(@orders == 2, "Got the orders");
my $count = 0;
foreach my $order (@orders) {
$count++;
ok ($order, "object ok");
ok ($order->amazon_order_number, "Got order number") and diag $order->amazon_order_number;
ok ($order->struct, "struct ok");
my $order_date = $order->order_date;
t/product-filtering.t view on Meta::CPAN
secret_key => '123412341234',
marketplace_id => '123412341234',
endpoint => 'https://mws-eu.amazonservices.com',
feed_dir => 't/feeds',
schema_dir => 'schemas',
existing_products => $existing_products,
products => \@products,
debug => 1,
);
my $uploader = Amazon::MWS::Uploader->new(%constructor);
ok($uploader, "object created");
is_deeply ($uploader->existing_products, $existing_products,
"lazy attribute passed at constructor");
ok(scalar(@{ $uploader->products_to_upload }), "Found the product to upload");
ok(!$uploader->product_needs_upload(1234 => '2014-11-11'));
$existing_products->{1234}->{status} = 'failed';
$existing_products->{1234}->{error_code} = '20000';
$uploader = Amazon::MWS::Uploader->new(%constructor);
ok(!$uploader->product_needs_upload(1234 => '2014-11-11'));
is_deeply($uploader->products_to_upload, [], "No products to uploads (failed)") or diag Dumper($uploader);
$constructor{reset_errors} = '20000';
$uploader = Amazon::MWS::Uploader->new(%constructor);
ok($uploader->product_needs_upload(1234 => '2014-11-11'));
ok(scalar(@{ $uploader->products_to_upload }), "Found the product to upload");
eval { Amazon::MWS::XML::Product->new(sku => '1234',
price => '10',
ean => '4444123412343',
images => [ "http://test.org/hello there.jpg" ]) };
ok $@, "unescaped url found:" . $@;
eval { Amazon::MWS::XML::Product->new(sku => '1234',
price => '10',
ean => '4444123412343',
t/shipping-confirmation.t view on Meta::CPAN
},
],
},
"Structure appears ok");
exit unless $test_extended;
my $feed_dir = 't/feeds';
my $uploader = Amazon::MWS::Uploader->new(
merchant_id => 'My Store',
access_key_id => '12341234',
secret_key => '123412341234',
marketplace_id => '123412341234',
endpoint => 'https://mws-eu.amazonservices.com',
feed_dir => $feed_dir,
schema_dir => $schema_dir,
);
ok($uploader, "Uploader ok");
my $feed = $uploader->shipping_confirmation_feed($shipped_order);
ok($feed, "Can create the feed and validates against the schema"); # and diag $feed;
# test against the example provided in the documentation
my $expected = <<'XML';
<?xml version="1.0" encoding="UTF-8"?>
<AmazonEnvelope>
<Header>
<DocumentVersion>1.1</DocumentVersion>
t/shipping-confirmation.t view on Meta::CPAN
items => [
{
merchant_order_item_code => 1234567,
merchant_fulfillment_item_id => 1234567,
quantity => 2,
},
],
);
$shipped_order = Amazon::MWS::XML::ShippedOrder->new(%shipped);
is_deeply ([ split(/\n/, $uploader->shipping_confirmation_feed($shipped_order)) ],
[ split(/\n/, $expected) ],
"Feed looks ok");
t/uploader.t view on Meta::CPAN
my %constructor = (
merchant_id => '__MERCHANT_ID__',
access_key_id => '12341234',
secret_key => '123412341234',
marketplace_id => '123412341234',
endpoint => 'https://mws-eu.amazonservices.com',
feed_dir => $feed_dir,
schema_dir => 'schemas',
);
my $uploader = Amazon::MWS::Uploader->new(%constructor);
ok($uploader);
ok($uploader->client->can('agent'), "Client can call agent");
ok($uploader->client->agent->isa('LWP::UserAgent'));
ok($uploader->schema, "schema built");
ok($uploader->xml_reader, "Reader ok");
ok($uploader->xml_writer, "Writer ok");
ok($uploader->generic_feeder->xml_writer);
is($uploader->_unique_shop_id, $constructor{merchant_id});
$uploader = Amazon::MWS::Uploader->new(%constructor,
shop_id => 'shoppe');
is($uploader->_unique_shop_id, 'shoppe');
eval {
$uploader = Amazon::MWS::Uploader->new(%constructor,
reset_errors => '! 2341 , 1234 , 1234 ,'
);
};
ok (!$@, "No exception");
is_deeply($uploader->_reset_error_structure,
{
negate => 1,
codes => {
2341 => 1,
1234 => 1,
}
}, "reset error structure ok")
or diag Dumper($uploader->_reset_error_structure);
eval {
$uploader = Amazon::MWS::Uploader->new(%constructor,
reset_errors => '2341 , 1234 , 1234 ,'
);
};
ok (!$@, "No exception");
is_deeply($uploader->_reset_error_structure,
{
negate => 0,
codes => {
2341 => 1,
1234 => 1,
}
}, "reset error structure ok (no negate)")
or diag Dumper($uploader->_reset_error_structure);
eval {
$uploader = Amazon::MWS::Uploader->new(%constructor,
reset_errors => 'balklasdfl'
);
};
ok ($@, "Found exception") and diag $@;
eval {
$uploader = Amazon::MWS::Uploader->new(%constructor,
db_options => undef);
};
ok (!$@, "undef as db_options is fine") and diag $@;
$uploader = Amazon::MWS::Uploader->new(%constructor,
skus_warnings_modes => {
8002 => 'warn',
8003 => 'print',
8001 => 'invalid',
});
{
my @warned;
local $SIG{__WARN__} = sub {
my ($warn) = @_;
like $warn, qr/\(800\d\)/;
push @warned, $warn;
};
foreach my $code (qw/8001 8002 8003 8008/) {
$uploader->_error_logger(warning => $code => "$code ć warn");
}
is (scalar(@warned), 2) or diag Dumper(\@warned);
is_deeply(\@warned, [
"Invalid mode invalid for warning: 8001 ć warn (8001)\n",
"warning: 8002 ć warn (8002)\n",
]);
}
my $now = DateTime->now;
my $old = $now->clone->subtract(hours => 2);
ok (!$uploader->job_timed_out({
task => 'order_ack',
job_started_epoch => $old->epoch,
}),
"order_ack doesn't timeout in 2 hours since " . $old->ymd);
ok (!$uploader->job_timed_out({
task => 'upload',
job_started_epoch => $old->epoch,
}),
"upload doesn't timeout in 2 hours since " . $old->ymd);
$old = $now->clone->subtract(days => 4);
ok (!$uploader->job_timed_out({
task => 'order_ack',
job_started_epoch => $old->epoch,
}),
"order_ack doesn't timeout in 4 days since " . $old->ymd);
ok ($uploader->job_timed_out({
task => 'upload',
job_started_epoch => $old->epoch,
}),
"upload timeouts in 4 days since " . $old->ymd);
$old = $now->clone->subtract(days => 31);
ok ($uploader->job_timed_out({
task => 'order_ack',
job_started_epoch => $old->epoch,
}),
"order_ack doesn't timeout in 31 days since " . $old->ymd);
my @warns = $uploader->_print_or_warn_error("test me\n");
is $warns[0], 'warn';
$uploader = Amazon::MWS::Uploader->new(%constructor, quiet => 1);
@warns = $uploader->_print_or_warn_error("test me\n");
is $warns[0], 'print';
( run in 1.783 second using v1.01-cache-2.11-cpan-b16cb0d3907 )