Net-IMP
view release on metacpan or search on metacpan
- Allow extended information with IMP_DENY too.
Update Filter and Cascade to use extended information in IMP_LOG
and IMP_DENY.
0.630 2015/02/20
- Documentation update API. Allow extended information with IMP_LOG.
0.629 2014/01/12
- fix smartmatch warnings with perl 5.19.7
- fix test failure str2cfg with perl 5.19.7 (stringification of regex
changed again)
0.628 2014/01/11
- fix case in ProtocolPinning, when for streaming data last rule for
direction was matched, match could have been longer and no max_unbound
was defined for dir. In this case it did not strip the matched data
from the buffer. This problem caused no harm but only a warning with
perl >= 5.16
0.627 2013/12/12
- bin/imp-pcap-filter.pl skipped first packet of udp connection
0.626 2013/10/09
- ProtocolPinning: support for IMP_PAUSE/IMP_CONTINUE if max_unbound undef
0.625 2013/10/07
- ProtocolPinning: fix bug for streaming mode if data call contained more data
then necessary to match last rule in dir and other dir had unfinshed match
on last rule
0.624 2013/10/05
- ProtocolPinning:
- fix bug for streaming mode if data call contained more data then necessary
to match last rule in dir (died with "buffer should be empty").
- Added some more test and optimizations
0.623 2013/09/23
- ProtocolPinning now has support for packet types.
Contrary to stream types a rule must match whole packet, no more and no less.
Configuration keys allow_dup and allow_reorder for unreliable packet
transports like UDP.
0.622 2013/09/16
- Bugfix Cascade
0.621 2013/07/16
based on adaptor if necessary for the data type
0.54 2013/01/07
- stream type now -1
- put factory into Net::IMP::Base, arguments for factory are not given directly
to analyzer anymore but need to be accessed using factory_args
- better documentation, clearer distinction what are global, factory or analyzer
methods
- support interfaces with dtype/rtype instead of global dtypes/rtypes.
decision which interface to use inside factory
0.53 2012/12/28
- allow other streaming data types, with number<0.
support these types inside Cascade
0.52 2012/12/20
- Cascade: allow and ignore (pre)pass for parts of data packets (e.g.
packetized, not streamed data), as long as it is not followed by a
replacement for other parts of packet
- update documentation regarding non-stream data
0.51 2012/12/11
- fix Cascade::USED_RTYPES to return dualvar constants for rtypes instead
of strings
0.50 2012/12/10
Net::IMP
IMP = Inspection and Modification Interface
This library provides a way to add inspection and modifications plugins
into existing data interceptors, e.g. proxies, intrusion detection
systems...
Contrary to ICAP (Internet Content Adaption Protocol, used mainly to add
content filtering to HTTP or mail) it is streaming aware, e.g. no need
to collect all data before inspection.
It is also performant because it has the ability to skip uninteresting
content or to pass a defined window of content before inspection, thus
keeping latencies small.
And it is callback driven, so no need to create a thread or process for
each connection. Thus it fits nicely to AnyEvent, POE...
Right now there are two sample users of the interface (see bin/ folder).
lib/Net/IMP.pm view on Meta::CPAN
IMP_LOG_CRIT
IMP_LOG_ALERT
IMP_LOG_EMERG
);
our @EXPORT_OK = (@log_levels, 'IMP_DATA','IMP_DATA_TYPES', 'IMP_PASS_IF_BUSY');
our %EXPORT_TAGS = ( log => \@log_levels );
# data types/protocols
# These two are the basic types, more application specific types might
# be defined somewhere else and be mapped to a number within supported_dtypes.
# The only important thing is, that streaming data should be <0, while
# packetized data (like HTTP header or UDP datagrams) should be > 0
# If no explicit type is given in sub data, it will assume IMP_DATA_STREAM.
use constant IMP_DATA_STREAM => dualvar(-1,'imp.data.stream');
use constant IMP_DATA_PACKET => dualvar(+1,'imp.data.packet');
# the numerical order of the constants describes priority when
# cascading modules, e.g. replacement has a higher value then
# pass and gets thus forwarded as the cause for the data
lib/Net/IMP.pm view on Meta::CPAN
}
######################################################################
# definition of new data type suites
######################################################################
package Net::IMP::HTTP;
use Net::IMP 'IMP_DATA';
use Exporter 'import';
our @EXPORT = IMP_DATA('http',
'header' => +1, # packet type
'body' => -2, # streaming type
...
);
=head1 DESCRIPTION
IMP is a protocol for inspection, modification and rejection of data between
two sides (client and server) using an analyzer implementing this interface.
=head2 Basics
lib/Net/IMP.pm view on Meta::CPAN
position value with the original position in the data stream.
In any other case it should be set to 0.
C<$type> is the type of the data.
There are two global data type definitions:
=over 4
=item IMP_DATA_STREAM (-1)
This is for generic streaming data, e.g. chunks from these datatypes can be
concatenated and analyzed together, parts can be replaced etc.
=item IMP_DATA_PACKET (+1)
This is for generic packetized data, where each chunk (e.g. call to C<data>)
contains a single packet, which should be analyzed as a separate entity.
This means no concatenating with previous or future chunks and no replacing of
only parts of the packet.
Also, any offsets given in calls to C<data> or in the results should be at
packet boundary (or IMP_MAX_OFFSET), at least for data modifications.
It will ignore (pre)pass which are not a packet boundary in the hope, that more
(pre)pass will follow.
A (pre)pass for some parts of a packet followed by a replacement is not allowed
and will probably cause an exception.
=back
All other data types are considered either subtypes of IMP_DATA_PACKET
(value >0) or of IMP_DATA_STREAM (value<0) and share their restrictions.
Also only streaming data of the same type can be concatenated and
analyzed together.
Results will be delivered through the callback or via C<poll_results>.
=item $analyzer->poll_results => @results
Returns outstanding results.
If a callback is attached, no results will be delivered this way.
=item $analyzer->busy($dir,0|1)
lib/Net/IMP.pm view on Meta::CPAN
=back
=head2 Helper Functions
The function C<IMP_DATA> is provided to simplify definition of new data types,
for example:
our @EXPORT = IMP_DATA('http',
'header' => +1, # packet type
'body' => -2, # streaming type
...
);
push @EXPORT = IMP_DATA('httprq[http+10]',
'header' => +1, # packet type
'content' => -2, # streaming type
...
);
This call of IMP_DATA is equivalent to the following perl declaration:
use Scalar::Util 'dualvar';
our @EXPORT = (
'IMP_DATA_HTTP', 'IMP_DATA_HTTP_HEADER','IMP_DATA_HTTP_BODY',...
'IMP_DATA_HTTPRQ', 'IMP_DATA_HTTPRQ_HEADER','IMP_DATA_HTTPRQ_BODY',...
);
lib/Net/IMP/Cascade.pm view on Meta::CPAN
if ( $part->{pass} == IMP_MAXOFFSET || $buf->{eof} ) {
# part done, skip it in the future
push @fwd,[$pi,$dir,undef]; # buf = undef is special
}
# insert dummy
@$ibuf = $new_buf->(
start => $buf->{end},
end => $buf->{end},
gstart => $buf->{gend},
gend => $buf->{gend},
# keep type for streaming data
$buf->{dtype} < 0 ? ( dtype => $buf->{dtype} ):(),
);
last;
}
} else {
# only part of buffer can be passed
# split buffer and re-enter loop, this will foreward the
# first part and keep the later part
$DEBUG && debug(
lib/Net/IMP/Filter.pm view on Meta::CPAN
my ($class,$imp,%args) = @_;
if (ref($class)) {
%args = (%$class, %args);
$imp ||= $class->{imp};
}
my $self = lock_ref_keys( bless {
%args,
imp => $imp, # analyzer object
buf => [
# list of buffered data [ offset,buf,type ] per dir
# buffers for same streaming type will be concatenated
[ [0,'',0] ],
[ [0,'',0] ],
],
pass => [0,0], # may pass up to this offset
prepass => [0,0], # may prepass up to this offset
skipped => [0,0], # flag if last data got not send to analyzer
# because of pass into future
eof => [0,0], # flag if eof received
dead => 0, # set if deny|fatal received
},(ref $class || $class) );
lib/Net/IMP/Filter.pm view on Meta::CPAN
$self->{skipped}[$dir] = 1;
}
}
# data left which need to be forwarded to analyzer
if ( ! $buf->[-1][2] ) {
# replace empty (untyped) buffer with new data
$buf->[-1][1] = $data;
$buf->[-1][2] = $type;
} elsif ( $type < 0 and $buf->[-1][2] == $type ) {
# streaming data of same type can be added to current buffer
$buf->[-1][1] .= $data;
} else {
# need new buffer
push @$buf,[
$buf->[-1][0] + length($buf->[-1][1]), # base = end of last
$data,
$type
];
}
lib/Net/IMP/Filter.pm view on Meta::CPAN
}
} elsif ( $offset < $buf0->[0] ) {
$DEBUG && debug("duplicate $rtype $offset ($buf0->[0])");
unshift @$buf,$buf0;
last;
} elsif ( $offset == $buf0->[0] ) {
# at border, e.g. forward 0 bytes
unshift @$buf,$buf0;
last;
} elsif ( $buf0->[2] < 0 ) {
# streaming type, can pass part of buf
$DEBUG && debug("pass part of buf");
push @fwd, [
$dir,
substr($buf0->[1],0,$offset - $end,''),
$buf0->[2],
];
# put back with adjusted offset
$buf0->[0] = $offset;
unshift @$buf, $buf0;
last;
lib/Net/IMP/Pattern.pm view on Meta::CPAN
$offset and die "cannot deal with gaps in data";
# if this is the wrong dir return, we already issued PASS
return if defined $self->{rxdir} and $dir != $self->{rxdir};
# accumulate results
my @rv;
my $buf;
if ( $type > 0 or $type != $self->{buftype}[$dir] ) {
# packet data or other streaming type
$buf = $data;
if ( $self->{buf}[$dir] ne '' ) {
# pass previous buffer and reset it
debug("reset buffer because type=$type, buftype=$self->{buftype}[$dir]");
$self->{offset}[$dir] += length($self->{buf}[$dir]);
$self->{buf}[$dir] = '';
push @rv, [ IMP_PASS,$dir,$self->{offset}[$dir] ];
} elsif ( ! $self->{buftype}[$dir] and not $type > 0 ) {
# initial streaming buf
$self->{buf}[$dir] = $buf;
}
$self->{buftype}[$dir] = $type;
} else {
# streaming data, match can span multiple chunks
$buf = ( $self->{buf}[$dir] .= $data );
}
$DEBUG && debug("got %d bytes $type on %d, bufsz=%d, rxlen=%d",
length($data),$dir,length($buf),$self->{rxlen});
# for packet types we accumulate datain newdata and set changed if newdata
# are different from old
my $changed = 0;
my $newdata = '';
lib/Net/IMP/ProtocolPinning.pm view on Meta::CPAN
goto PASS_AND_RETURN; # wait for more data
}
# check against current set
if ( $type>0 ) {
# packet data
if ( $self->{buf}[$dir] ne '' ) {
$self->run_callback([
IMP_DENY,
$dir,
"packet data after unmatched streaming data"
]);
}
for( my $i=0;$i<@$crs;$i++ ) {
if ( my ($len) = $match->($rules->[$crs->[$i]],\$data)) {
# match
$pass_until = $self->{off_passed}[$dir] =
$self->{off_buf}[$dir] += $len;
if ( $self->{matched} ) {
# preserve hash of matched packet so that duplicates are
# detected later
lib/Net/IMP/ProtocolPinning.pm view on Meta::CPAN
$DEBUG && debug("no matching rule for ${type}[$dir] - deny");
$self->{buf} = undef;
$self->run_callback([
IMP_DENY,
$dir,
"rule#@$crs did not match"
]);
return;
} else {
# streaming data
my $temp_fail;
my $final_match;
for( my $i=0;$i<@$crs;$i++ ) {
my ($len,$removed)
= $match->($rules->[$crs->[$i]],\$self->{buf}[$dir]);
if ( ! defined $len ) {
# will never match against rule
next;
} elsif ( ! $len ) {
# note that it might match if buf gets longer but check other
lib/Net/IMP/ProtocolPinning.pm view on Meta::CPAN
"full match rule $crs->[$i] - remove ruleset");
shift(@$rs);
# switch to other dir if this dir is done for now
if ( ! @$rs || ! $rs->[0] ) {
my $ors = $self->{ruleset}[$dir ? 0:1];
shift @$ors if @$ors && ! $ors->[0];
goto CHECK_DONE if ! @$ors && ! @$rs;
}
}
$final_match = 1;
# no allow_dup for streaming
}
# pass data
if ( $final_match and $self->{buf}[$dir] ne '' ) {
# try to match more
$data = $self->{buf}[$dir];
$self->{buf}[$dir] = '';
goto NEXT_RULE;
}
goto CHECK_DONE if ! @$rs;
t/09_pattern_stream_vs_packet.t view on Meta::CPAN
#!/usr/bin/perl
# Net::IMP::Pattern with mix of packet and streaming data
use strict;
use warnings;
use Net::IMP;
use Net::IMP::Pattern;
use Net::IMP::Debug;
use Test::More tests => 1;
$DEBUG=0; # enable for extensiv debugging
( run in 0.305 second using v1.01-cache-2.11-cpan-4d50c553e7e )