App-BloomUtils

 view release on metacpan or  search on metacpan

README  view on Meta::CPAN

132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
Usage:
 
 gen_bloom_filter(%args) -> [status, msg, payload, meta]
 
Generate bloom filter.
 
Examples:
 
*   Create a bloom filter for 100k items and 0.1% maximum false-positive
    rate (actual bloom size and false-positive rate will be shown on
    stderr):
 
     gen_bloom_filter( false_positive_rate => "0.1%", num_items => 100000);
 
You supply lines of text from STDIN and it will output the bloom filter
bits on STDOUT. You can also customize "num_bits" ("m") and "num_hashes"
("k"), or, more easily, "num_items" and "fp_rate". Some rules of thumb
to remember:
 
*   One byte per item in the input set gives about a 2% false positive
    rate. So if you expect two have 1024 elements, create a 1KB bloom

lib/App/BloomUtils.pm  view on Meta::CPAN

88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
                p => {},
            },
        },
    },
    'cmdline.skip_format' => 1,
    args_rels => {
    },
    examples => [
        {
            summary => 'Create a bloom filter for 100k items and 0.1% maximum false-positive rate '.
                '(actual bloom size and false-positive rate will be shown on stderr)',
            argv => [qw/--num-items 100000 --fp-rate 0.1%/],
            'x.doc.show_result' => 0,
            test => 0,
        },
    ],
    links => [
        {url=>'prog:bloom-filter-calculator'},
    ],
};
sub gen_bloom_filter {

lib/App/BloomUtils.pm  view on Meta::CPAN

459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
Usage:
 
 gen_bloom_filter(%args) -> [status, msg, payload, meta]
 
Generate bloom filter.
 
Examples:
 
=over
 
=item * Create a bloom filter for 100k items and 0.1% maximum false-positive rate (actual bloom size and false-positive rate will be shown on stderr):
 
 gen_bloom_filter( false_positive_rate => "0.1%", num_items => 100000);
 
=back
 
You supply lines of text from STDIN and it will output the bloom filter bits on
STDOUT. You can also customize C<num_bits> (C<m>) and C<num_hashes> (C<k>), or, more
easily, C<num_items> and C<fp_rate>. Some rules of thumb to remember:
 
=over

script/bloomgen  view on Meta::CPAN

47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
 
Usage:
 
 % bloomgen [--debug] [--false-positive-rate=s] [--fp-rate=s] [-k=s]
     [--log-level=level] [-m=s] [-n=s] [--num-bits=s] [--num-hashes=s]
     [--num-items=s] [-p=s] [--page-result[=program]] [--quiet] [--trace]
     [--verbose]
 
Examples:
 
Create a bloom filter for 100k items and 0.1% maximum false-positive rate (actual bloom size and false-positive rate will be shown on stderr):
 
 % bloomgen --num-items 100000 --fp-rate 0.1%
 
=head1 DESCRIPTION
 
You supply lines of text from STDIN and it will output the bloom filter bits on
STDOUT. You can also customize C<num_bits> (C<m>) and C<num_hashes> (C<k>), or, more
easily, C<num_items> and C<fp_rate>. Some rules of thumb to remember:
 
=over

script/gen-bloom-filter  view on Meta::CPAN

47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
 
Usage:
 
 % gen-bloom-filter [--debug] [--false-positive-rate=s] [--fp-rate=s]
     [-k=s] [--log-level=level] [-m=s] [-n=s] [--num-bits=s]
     [--num-hashes=s] [--num-items=s] [-p=s] [--page-result[=program]]
     [--quiet] [--trace] [--verbose]
 
Examples:
 
Create a bloom filter for 100k items and 0.1% maximum false-positive rate (actual bloom size and false-positive rate will be shown on stderr):
 
 % gen-bloom-filter --num-items 100000 --fp-rate 0.1%
 
=head1 DESCRIPTION
 
You supply lines of text from STDIN and it will output the bloom filter bits on
STDOUT. You can also customize C<num_bits> (C<m>) and C<num_hashes> (C<k>), or, more
easily, C<num_items> and C<fp_rate>. Some rules of thumb to remember:
 
=over

t/00-compile.t  view on Meta::CPAN

30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
use File::Spec;
use IPC::Open3;
use IO::Handle;
 
open my $stdin, '<', File::Spec->devnull or die "can't open devnull: $!";
 
my @warnings;
for my $lib (@module_files)
{
    # see L<perlfaq8/How can I capture STDERR from an external command?>
    my $stderr = IO::Handle->new;
 
    diag('Running: ', join(', ', map { my $str = $_; $str =~ s/'/\\'/g; q{'} . $str . q{'} }
            $^X, @switches, '-e', "require q[$lib]"))
        if $ENV{PERL_COMPILE_TEST_DEBUG};
 
    my $pid = open3($stdin, '>&STDERR', $stderr, $^X, @switches, '-e', "require q[$lib]");
    binmode $stderr, ':crlf' if $^O eq 'MSWin32';
    my @_warnings = <$stderr>;
    waitpid($pid, 0);
    is($?, 0, "$lib loaded ok");
 
    shift @_warnings if @_warnings and $_warnings[0] =~ /^Using .*\bblib/
        and not eval { +require blib; blib->VERSION('1.01') };
 
    if (@_warnings)
    {
        warn @_warnings;
        push @warnings, @_warnings;

t/00-compile.t  view on Meta::CPAN

63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
{ SKIP: {
    open my $fh, '<', $file or warn("Unable to open $file: $!"), next;
    my $line = <$fh>;
 
    close $fh and skip("$file isn't perl", 1) unless $line =~ /^#!\s*(?:\S*perl\S*)((?:\s+-\w*)*)(?:\s*#.*)?$/;
    @switches = (@switches, split(' ', $1)) if $1;
 
    close $fh and skip("$file uses -T; not testable with PERL5LIB", 1)
        if grep { $_ eq '-T' } @switches and $ENV{PERL5LIB};
 
    my $stderr = IO::Handle->new;
 
    diag('Running: ', join(', ', map { my $str = $_; $str =~ s/'/\\'/g; q{'} . $str . q{'} }
            $^X, @switches, '-c', $file))
        if $ENV{PERL_COMPILE_TEST_DEBUG};
 
    my $pid = open3($stdin, '>&STDERR', $stderr, $^X, @switches, '-c', $file);
    binmode $stderr, ':crlf' if $^O eq 'MSWin32';
    my @_warnings = <$stderr>;
    waitpid($pid, 0);
    is($?, 0, "$file compiled ok");
 
    shift @_warnings if @_warnings and $_warnings[0] =~ /^Using .*\bblib/
        and not eval { +require blib; blib->VERSION('1.01') };
 
    # in older perls, -c output is simply the file portion of the path being tested
    if (@_warnings = grep { !/\bsyntax OK$/ }
        grep { chomp; $_ ne (File::Spec->splitpath($file))[2] } @_warnings)
    {