AnyEvent-Net-Curl-Queued
view release on metacpan or search on metacpan
used to reproduce the same behaviour for the remaining agents, while
taskset <http://linux.die.net/man/1/taskset> avoids the skew on
multiprocessor systems.
The download target is a copy of the Apache documentation
<http://httpd.apache.org/docs/2.2/> on a local Apache server. The test
platform configuration:
* Intel® Core⢠i7-2600 CPU @ 3.40GHz with 8 GB RAM;
* Ubuntu 11.10 (64-bit);
* Perl v5.16.2 (installed via perlbrew);
* libcurl/7.28.0 (without AsynchDNS, which slows down
curl_easy_init()
<http://curl.haxx.se/libcurl/c/curl_easy_init.html>).
The script eg/benchmark.pl uses Benchmark::Forking and Class::Load to
keep UA modules isolated and loaded only once.
$ taskset 1 perl benchmark.pl --count 100 --parallel 8 --repeat 10
Request rate WWW::M LWP::UA L::P::N::C Mojo::UA HTTP::L HTTP::T lftp P::D AE::C::M YADA Furl curl wget LWP::C
WWW::Mechanize v1.72 534/s -- -32% -61% -63% -80% -82% -83% -84% -85% -86% -94% -95% -97% -97%
LWP::UserAgent v6.04 782/s 46% -- -42% -46% -71% -73% -75% -76% -77% -79% -92% -93% -95% -95%
LWP::Protocol::Net::Curl v0.011 1360/s 154% 74% -- -6% -50% -53% -57% -59% -61% -64% -86% -88% -91% -91%
Mojo::UserAgent v3.82 1450/s 171% 85% 7% -- -46% -50% -54% -56% -58% -62% -85% -87% -91% -91%
HTTP::Lite v2.4 2700/s 405% 245% 98% 86% -- -7% -14% -18% -22% -29% -71% -76% -82% -83%
HTTP::Tiny v0.025 2910/s 445% 272% 114% 101% 8% -- -7% -11% -16% -23% -69% -74% -81% -81%
lftp v4.3.1 3140/s 488% 302% 131% 117% 17% 8% -- -4% -9% -17% -67% -72% -80% -80%
Parallel::Downloader v0.121560 3280/s 514% 319% 141% 127% 22% 13% 4% -- -5% -13% -65% -70% -79% -79%
AnyEvent::Curl::Multi v1.1 3460/s 548% 342% 155% 139% 28% 19% 10% 5% -- -9% -63% -69% -77% -78%
YADA v0.038 3790/s 610% 385% 179% 162% 41% 30% 21% 16% 10% -- -60% -66% -75% -76%
Furl v2.01 9420/s 1663% 1104% 593% 550% 249% 223% 200% 187% 172% 148% -- -15% -39% -40%
curl v7.28.0 11100/s 1977% 1318% 716% 666% 311% 281% 253% 238% 221% 193% 18% -- -28% -29%
wget v1.12 15400/s 2777% 1864% 1031% 961% 470% 428% 389% 368% 344% 305% 63% 39% -- -1%
LWP::Curl v0.12 15600/s 2818% 1892% 1047% 976% 478% 435% 396% 375% 350% 311% 65% 40% 1% --
(output formatted to show module versions at row labels and keep column labels abbreviated)
ATTRIBUTES
allow_dups
Allow duplicate requests (default: false). By default, requests to the
same URL (more precisely, requests with the same signature are issued
only once. To seed POST parameters, you must extend the
AnyEvent::Net::Curl::Queued::Easy class. Setting allow_dups to true
value disables request checks.
common_opts
"opts" in AnyEvent::Net::Curl::Queued::Easy attribute common to all
workers initialized under the same queue. You may define User-Agent
string here.
http_response
Encapsulate the response with HTTP::Response (only when the scheme is
HTTP/HTTPS); a global version of "http_response" in
AnyEvent::Net::Curl::Queued::Easy. Default: disabled.
completed
Count completed requests.
cv
AnyEvent condition variable. Initialized automatically, unless you
specify your own. Also reset automatically after "wait", so keep your
own reference if you really need it!
max
Maximum number of parallel connections (default: 4; minimum value: 1).
multi
Net::Curl::Multi instance.
queue
ArrayRef to the queue. Has the following helper methods:
queue_push
Append item at the end of the queue.
queue_unshift
Prepend item at the top of the queue.
dequeue
Shift item from the top of the queue.
count
Number of items in queue.
share
Net::Curl::Share instance.
stats
AnyEvent::Net::Curl::Queued::Stats instance.
timeout
Timeout (default: 60 seconds).
unique
Signature cache.
watchdog
The last resort against the non-deterministic chaos of evil lurking
sockets.
( run in 0.836 second using v1.01-cache-2.11-cpan-39bf76dae61 )