view release on metacpan or  search on metacpan

lib/Mail/SpamAssassin/Plugin/ExtractText.pm  view on Meta::CPAN

=head1 NAME

ExtractText - extracts text from documenmts.

=head1 SYNOPSIS

loadplugin Mail::SpamAssassin::Plugin::ExtractText

ifplugin Mail::SpamAssassin::Plugin::ExtractText

  extracttext_external  pdftotext  /usr/bin/pdftotext -nopgbrk -layout -enc UTF-8 {} -
  extracttext_use       pdftotext  .pdf application/pdf

  # http://docx2txt.sourceforge.net
  extracttext_external  docx2txt   /usr/bin/docx2txt {} -
  extracttext_use       docx2txt   .docx application/docx

  extracttext_external  antiword   /usr/bin/antiword -t -w 0 -m UTF-8.txt {}
  extracttext_use       antiword   .doc application/(?:vnd\.?)?ms-?word.*

  extracttext_external  unrtf      /usr/bin/unrtf --nopict {}
  extracttext_use       unrtf      .doc .rtf application/rtf text/rtf

  extracttext_external  odt2txt    /usr/bin/odt2txt --encoding=UTF-8 {}
  extracttext_use       odt2txt    .odt .ott application/.*?opendocument.*text
  extracttext_use       odt2txt    .sdw .stw application/(?:x-)?soffice application/(?:x-)?starwriter

  extracttext_external  tesseract  {OMP_THREAD_LIMIT=1} /usr/bin/tesseract -c page_separator= {} -
  extracttext_use       tesseract  .jpg .png .bmp .tif .tiff image/(?:jpeg|png|x-ms-bmp|tiff)

  # QR-code decoder
  extracttext_external  zbar       /usr/bin/zbarimg -q -D {}
  extracttext_use       zbar       .jpg .png .pdf image/(?:jpeg|png) application/pdf

  add_header   all          ExtractText-Flags _EXTRACTTEXTFLAGS_
  header       PDF_NO_TEXT  X-ExtractText-Flags =~ /\bpdftotext_NoText\b/
  describe     PDF_NO_TEXT  PDF without text
  score        PDF_NO_TEXT  0.001

  header       DOC_NO_TEXT  X-ExtractText-Flags =~ /\b(?:antiword|openxml|unrtf|odt2txt)_NoText\b/
  describe     DOC_NO_TEXT  Document without text
  score        DOC_NO_TEXT  0.001

  header       EXTRACTTEXT  exists:X-ExtractText-Flags
  describe     EXTRACTTEXT  Email processed by extracttext plugin
  score        EXTRACTTEXT  0.001

endif

=head1 DESCRIPTION

This module uses external tools to extract text from message parts,
and then sets the text as the rendered part. External tool must output
plain text, not HTML or other non-textual result.

How to extract text is completely configurable, and based on
MIME part type and file name.

=head1 CONFIGURATION

All configuration lines in user_prefs files will be ignored.

=over 4

=item extracttext_maxparts (default: 10)

Configure the maximum mime parts number to analyze, a value of 0 means all mime parts
will be analyzed

=item extracttext_timeout (default: 5 10)

Configure the timeout in seconds of external tool checks, per attachment.

Second argument speficies maximum total time for all checks.

=back

=head2 Tools

=over

=item extracttext_use

Specifies what tool to use for what message parts.

The general syntax is

extracttext_use  C<name>  C<specifiers>

=back

=over

=item name

the internal name of a tool.

=item specifiers

File extension and regular expressions for file names and MIME
types. The regular expressions are anchored to beginning and end.

=back

=head3 Examples

	extracttext_use  antiword  .doc application/(?:vnd\.?)?ms-?word.*
	extracttext_use  openxml   .docx .dotx .dotm application/(?:vnd\.?)openxml.*?word.*
	extracttext_use  openxml   .doc .dot application/(?:vnd\.?)?ms-?word.*
	extracttext_use  unrtf     .doc .rtf application/rtf text/rtf

=over

=item extracttext_external

Defines an external tool.  The tool must read a document on standard input
or from a file and write text to standard output.

The special keyword "{}" will be substituted at runtime with the temporary
filename to be scanned by the external tool.