view release on metacpan or  search on metacpan

lib/Regexp/Log/BlueCoat.pm  view on Meta::CPAN

    '%y' =>
'(?#=time)(?#=hour)\\d\\d(?#!hour):(?#=minute)\\d\\d(?#!minute):(?#=second)\\d\\d(?#!second)(?#!time)',

    # %z    - [Not used.] -
    '%z' => '',

    # %A    cs (user-agent) User agent No
    '%A' => '(?#=user-agent).*(?#!user-agent)',

    # %B    cs-bytes The number of bytes received by the server Yes
    '%b' => '(?#=cs-bytes)\\d+(?#!cs-bytes)',

# %C    cs (cookie) Cookie data No
# %D    s-supplier-ip SUPPLIER IP - IP address of server/cache from which the object was received.  Yes
# %E    s-Policy-Message Policy enforcement message Yes
# %F    - [Not used.] -
    '%F' => '',

    # %G    - [Not used.] -
    '%G' => '',

# %H    s-hierarchy How and where the object was retrieved from the cache hierarchy (DIRECT from the server, PARENT_HIT = from the parent cache, and so on) No
    '%H' =>
'(?#=s-hierarchy)DIRECT|NONE|(?:PARENT|SIBLING)_HIT|FIRST_PARENT_MISS(?#!s-hierarchy)',

# %I    s-ip Server IP, the IP address of the server on which the log entry was generated Yes
# %J    - [Not used.] -
    '%J' => '',

    # %K    - [Not used.] -
    '%K' => '',

# %L    localtime Local date and time of the user request in format: [DD/MMM/YYYY:hh:mm:ss +nnnn] Yes
    '%L' =>
'\\[(?#=localtime)(?#=localday)\\d\\d(?#!localday)/(?#=localmonth)\\d\\d(?#!localmonth)/(?#=localyear)\\d\\d\\d\\d(?#!localyear):(?#=localhour)\\d\\d(?#!localhour):(?#=localminute)\\d\\d(?#!localminute):(?#=localsecond)\\d\\d(?#!localsecond) \\+\\d\\...

    # %M    - [Not used.] -
    '%M' => '',

# %N    s-computername Server name, the name of the server on which the log entry was generated Yes
    '%N' => "(?#=s-computername)$HOST(?#!s-computername)",

    # %O    - [Not used.] -
    '%O' => '',

    # %P    s-port Server port, the port number the client is connected to.  Yes
    '%P' => '(?#=s-port)\\d+(?#!s-port)',

# %Q    cs-uri-query The URI query portion of the URL No
# %R    cs (Referer) Request referrer No
# %S    s-sitename Internet service and instance number running on client computer Yes
# %T    duration Elapsed time, seconds Yes
    '%T' => '(?#=duration)\\d+(?#!duration)',

# %U    cs-uri-stem Object path from request URL Yes
# %V    cs-version The protocol (HTTP, FTP) version used by the client.  Yes
# %W    sc-filter-result UFS event (May differ between Websense or SmartFilter or others).  No
    # this is handled in _postprocess() and is unsupported yet
    '%W' => '',

# %X    cs (X-Forwarded-For) The IP address of the device which sent the HTTP request.  No
# %Y    - [Not used.] -
    '%Y' => '',

    # %Z    - [Not used.] -
    '%Z' => '',

    # UFS specific
    # Smartfilter

    # Login specific
    '%u-username' => '(?#=cs-username)[-.\\w]+(?#!cs-username)',
    '%u-ldap'     =>
      '(?#=cs-username)-|(?:[A-Za-z]+=[^,]*,)*[A-Za-z]=[^,]*?(?#!cs-username)',
);

=head1 METHODS

Regexp::Log::BlueCoat is a standard Regexp::Log object, and therefore
supports all the standard Regexp::Log methods.

Regexp::Log::BlueCoat's constructor accepts several BlueCoat specific
arguments:

    ufs    - URL Filtering Service
    login  - The type of username information

Note: Though BlueCoat supports SmartFilter, Websense and others,
Regexp::Log::BlueCoat only support I<SmartFilter> UFS in this version.

The appropriate accessors are defined for them (if used to set, they
return the new value for the attribute).

=over 4

=item ufs( [$ufs] )

Get/set the URL Filter System type (C<%f> and C<%W>).
Only C<smartfilter> is supported in this version.

=cut

sub ufs {
    my $self = shift;
    $self->{ufs} = shift if @_;
    return $self->{ufs};
}

=item ufs_category( category => string, [...] )

This method lets you override the default category names in your UFS.

For example, I<SmartFilter> allows to configure the name of the
categories; Regexp::Log::BlueCoat supports the default category names,
but lets you override them if needed.

The changes are applied on the objet current C<ufs>.

    $log->ufs('smartfilter');
    $log->ufs_category( hm => 'FunStuff' );    # change the Humor category

lib/Regexp/Log/BlueCoat.pm  view on Meta::CPAN

 %g    timestamp            UNIX type timestamp.
 %h    c-ip                 Client Hostname (uses IP to avoid reverse DNS)
                            - same as %a
 %i    cs-uri               The requested URI. Note: Web trends expects
                            this to be only cs-uri-stem + cs-uri-query
 %j    -                    [Not used.]
 %l    -                    Client Identification string.
                            (User Login name remote). - always '-'
 %m    cs-method            HTTP method. HTTP methods include GET, PUT,
                            POST, and so on.
 %n    -                    [Not used.]
 %o    -                    [Not used.]
 %p    r-port Port          fetched from on host - origin server port
 %q    -                    [Not used.]
 %r    cs-request-line      First line of the request
 %s    sc-status            The code returned by the cache to the client
                            (HTTP code). 
 %t    gmttime              GMT date and time of the user request, in
                            the format [DD/MM/YYYY:hh:mm:ss GMT]
 %u    cs-username          Authenticated user ID.
 %v    cs-host              Name of host sourcing the object.
 %w    s-action             What type of action did the CM take to process
                            this request. NOTE: 'cached' is used by ELFF
                            but has int value.
 %x    date                 Date in YYYY-MM-DD format
 %y    time                 GMT time in HH:MM:SS format
 %z    -                    [Not used.]
 %A    cs (user-agent)      User agent
 %B    cs-bytes             The number of bytes received by the server
 %C    cs (cookie)          Cookie data
 %D    s-supplier-ip        SUPPLIER IP - IP address of server/cache from
                            which the object was received.
 %E    s-Policy-Message     Policy enforcement message
 %F    -                    [Not used.]
 %G    -                    [Not used.]
 %H    s-hierarchy          How and where the object was retrieved from the
                            cache hierarchy (DIRECT from the server,
                            PARENT_HIT = from the parent cache, and so on) 
 %I    s-ip                 Server IP, the IP address of the server on which
                            the log entry was generated
 %J    -                    [Not used.]
 %K    -                    [Not used.]
 %L    localtime            Local date and time of the user request in
                            format: [DD/MMM/YYYY:hh:mm:ss +nnnn]
 %M    -                    [Not used.]
 %N    s-computername       Server name, the name of the server on which
                            the log entry was generated
 %O    -                    [Not used.]
 %P    s-port               Server port, the port number the client is
                            connected to.
 %Q    cs-uri-query         The URI query portion of the URL
 %R    cs (Referer)         Request referrer
 %S    s-sitename           Internet service and instance number running
                            on client computer
 %T    duration             Elapsed time, seconds
 %U    cs-uri-stem          Object path from request URL
 %V    cs-version           The protocol (HTTP, FTP) version used by
                            the client.
 %W    sc-filter-result     UFS event (May differ between Websense or
                            SmartFilter or others).
 %X    cs (X-Forwarded-For) The IP address of the device which sent
                            the HTTP request.
 %Y    -                    [Not used.]
 %Z    -                    [Not used.]

=head1 URL FILTERING SYSTEMS

The BlueCoat Systems Port 80 Security Appliance supports two URL Filtering
Systems (UFS): I<SmartFilter> and I<Websense>.

Since I only had access to log files generated with a BlueCoat + SmartFilter
combination, this version of Regexp::Log only supports I<SmartFilter> UFS.
Patches welcome!

=head2 SmartFilter

When C<ufs> is set to C<smartfilter>, the computed regular expression
matches the default SmartFilter category names. These can be changed 
in SmartFilter's configuration (furthermore  one can create one's own
categories, with user-defined names).

So we need to be able to modify the category names, either in an
object instance, or in class data (shared by all instances).

To compute a regular expression that matches your specific fields, there
are several possibilities:

=over 4

=item Make the changes in your object instance

The method ufs_category() lets you replace any standard category by
your own, and even add new "categories" (text that will be matched by
the C<%f> fields).

These changes are valid for the object only.

See ufs_category() for details.

=item Change the Regexp::Log::BlueCoat class itself

ufs_category() can be used as a class method.

One can also be adventurous and acces %Regexp::Log::BlueCoat::UFS directly,
but you'll need to read the source to understand the details.
Here's an example:

    $Regexp::Log::UFS{smartfilter} = { simple => '[-\\w]+' };

=back

I<SmartFilter> default categories are:

 Key  Default value                        Category
 ---  -------------                        --------
 sx   "sex"                                Sex
 dr   "drugs"                              Drugs
 hs   "hate speech"                        Hate Speech
 cs   "crim. skills"                       Criminal Skills
 nd   "nudity"                             Nudity
 os   "on-line sales"                      Online Sales