Cache-Static
view release on metacpan or search on metacpan
problem:
we need to pass a filehandle back to inherit the lock.
or we could save it globally somewheres?
pseudo code:
get_if_same:
if(!defined(is_same(...))) {
if(my $FH = open_with_flock(LOCK_EX, block => 0)) {
$cache::static::_curr_locked_FH = $FH;
} else {
#wait until it's been set, then return it
open_with_flock(LOCK_SH, block => 1);
return get(...);
}
} else {
return get(...);
}
set:
my $FH = $cache::static::_curr_locked_FH || open(...);
...
close($FH);
an optimization idea stolen from the lighthttpd folks:
(http://www.lighttpd.net/documentation/performance.html)
stat() cache
A stat(2) can be expensive; caching it saves time and context switches.
granted, they are talking about the context of httpd serving static
pages, but it could still cut down on our number of stat calls by a
large factor in the common case...
next up:
in the case where you allow old results:
before writing, copy the cache file "$f" to "$f.bak"
any get_if_same() during that time reads "$f.bak"
when done, remove "$f.bak" (this results in race condition in reader:
1 - find the file locked, try the .bak file
2 - .bak file is removed, goto 1
OR have a crontab that runs e.g. hourly to clean up .bak files...
start the lock when generation starts, remove the lock when new value
has been set
situation:
refresh of key K takes 30 seconds.
every 5 seconds, we get a hit.
this means we regenerate 6 times, when we only needed to do it once.
doc/digests view on Meta::CPAN
available/installed.
MD5: 1.405/1.397/1.396
newhash: 0.923/0.920/0.925
#MD5 is much faster than SHA1 (at least in perl)
#MD4 is marginally faster than MD5 (but less available)
#Pearson, JHash, Elf, DJB, FNV all show promise but don't outperform newhash
#testing done on Athlon-XP downclocked to 1000MHz
#general notes:
#OO is faster
#using addfile instead of add is MUCH faster!
time perl -e 'use Digest::MD5 qw(md5 md5_hex md5_base64); for my $i
(0..100) { open(FH, "foobar"); my $digest = md5(join ("", <FH>));
close(FH); } '
MD5: 4.806/4.731/4.752
MD4: 4.625/4.647/4.620
( run in 1.092 second using v1.01-cache-2.11-cpan-49f99fa48dc )