view release on metacpan or  search on metacpan

README  view on Meta::CPAN

AI-Calibrate version 1.0
=========================

AI::Calibrate - Perl module for producing probabilities from classifier scores

In AI, classifiers usually return some sort of an instance score with their
classifications.  These scores can be used as probabilities in various
calculations, but first they need to be calibrated.  Naive Bayes, for example,
is a very useful classifier, but the scores it produces are usually "bunched"
around 0 and 1, making these scores poor probability estimates.  Support
vector machines have a similar problem.  Both classifier types should be
calibrated before their scores are used as probability estimates.  This module
calibrates a classifier using the Pool Adjacent Violators algorithm.

INSTALLATION

To install this module type the following:

   perl Makefile.PL
   make
   make test
   make install

DEPENDENCIES

No other modules are required.

lib/AI/Calibrate.pm  view on Meta::CPAN

  ... test classifier on $points ...
  $calibrated = calibrate($points);

=head1 DESCRIPTION

Classifiers usually return some sort of an instance score with their
classifications.  These scores can be used as probabilities in various
calculations, but first they need to be I<calibrated>.  Naive Bayes, for
example, is a very useful classifier, but the scores it produces are usually
"bunched" around 0 and 1, making these scores poor probability estimates.
Support vector machines have a similar problem.  Both classifier types should
be calibrated before their scores are used as probability estimates.

This module calibrates classifier scores using a method called the Pool
Adjacent Violators (PAV) algorithm.  After you train a classifier, you take a
(usually separate) set of test instances and run them through the classifier,
collecting the scores assigned to each.  You then supply this set of instances
to the calibrate function defined here, and it will return a set of ranges
mapping from a score range to a probability estimate.

For example, assume you have the following set of instance results from your