Language-FormulaEngine
view release on metacpan or search on metacpan
0.08 - 2023-04-25
- Fix perl < 5.16 compatibility.
0.07 - 2023-04-17
- New Formula object for tracking a parse tree along with its
original formula text and associated engine.
- New 'simplify' method for removing constant terms from formula
- New compiler output_api 'function_of_vars_no_default'
0.06 - 2021-01-26
- Compiled formulas now access vars via Namespace->get_value
the same as uncompiled evaluated formulas, though this may be
internally optimized to the $vars hashref like before if the
Namespace has not customized get_value.
- Compiler->output_api now determines whether compiled formula
take %vars or $namespace as arguments.
- Deprecated Compiler->variables_via_namespace
- Parser's scanner_rules have a 4th argument of the variables
to make available to the code (3rd argument)
- Fixed a bug in Parser->keyword_map where wrong values could
get cached between subclass and parent class.
- Document scanner_rules, keyword_map, etc.
- Remove dependency on Const::Fast
0.05 - 2020-04-07
lib/Language/FormulaEngine/Namespace.pm
lib/Language/FormulaEngine/Namespace/Default.pm
lib/Language/FormulaEngine/Parser.pm
lib/Language/FormulaEngine/Parser/ContextUtil.pm
t/00-syntax.t
t/10-scanner.t
t/20-parser.t
t/25-simplify.t
t/30-compiler.t
t/35-error.t
t/39-formula-object.t
t/40-default-ns-core.t
t/41-default-ns-functions.t
t/50-custom-namespace.t
t/author-pod-coverage.t
t/author-pod-syntax.t
lib/Language/FormulaEngine.pm view on Meta::CPAN
package Language::FormulaEngine;
use Moo;
use Carp;
use Try::Tiny;
use Module::Runtime 'require_module';
# ABSTRACT: Parser/Interpreter/Compiler for simple spreadsheet formula language
our $VERSION = '0.08'; # VERSION
has parser => (
is => 'lazy',
builder => sub {},
coerce => sub { _coerce_instance($_[0], 'parse', 'Language::FormulaEngine::Parser') }
);
has namespace => (
is => 'lazy',
lib/Language/FormulaEngine.pm view on Meta::CPAN
1;
__END__
=pod
=encoding UTF-8
=head1 NAME
Language::FormulaEngine - Parser/Interpreter/Compiler for simple spreadsheet formula language
=head1 VERSION
version 0.08
=head1 SYNOPSIS
my $vars= { foo => 1, bar => 3.14159265358979, baz => 42 };
my $engine= Language::FormulaEngine->new();
$engine->evaluate( 'if(foo, round(bar, 3), baz*100)', $vars );
# or for more speed on repeat evaluations
my $formula= $engine->compile( 'if(foo, round(bar, 3), baz*100)' );
print $formula->($vars);
package MyNamespace {
use Moo;
extends 'Language::FormulaEngine::Namespace::Default';
sub fn_customfunc { print "arguments are ".join(', ', @_)."\n"; }
};
my $engine= Language::FormulaEngine->new(namespace => MyNamespace->new);
my $formula= $engine->compile( 'CustomFunc(baz,2,3)' );
$formula->($vars); # prints "arguments are 42, 2, 3\n"
=head1 DESCRIPTION
This set of modules implement a parser, evaluator, and optional code generator for a simple
expression language similar to those used in spreadsheets.
The intent of this module is to help you add customizable behavior to your applications that an
"office power-user" can quickly learn and use, while also not opening up security holes in your
application.
In a typical business application, there will always be another few use cases that the customer
didn't know about or think to tell you about, and adding support for these use cases can result
in a never-ending expansion of options and chekboxes and dropdowns, and a lot of time spent
deciding the logical way for them to all interact.
One way to solve this is to provide some scripting support for the customer to use. However,
you want to make the language easy to learn, "nerfed" enough for them to use safely, and
prevent security vulnerabilities. The challenge is finding a language that they find familiar,
that is easy to write correct programs with, and that dosn't expose any peice of the system
that you didn't intend to expose. I chose "spreadsheet formula language" for a project back in
2012 and it worked out really well, so I decided to give it a makeover and publish it.
The default syntax is pure-functional, in that each operation has exactly one return value, and
cannot modify variables; in fact none of the default functions have any side-effects. There is
no assignment, looping, or nested data structures. The language does have a bit of a Perl twist
to it's semantics, like throwing exceptions rather than returning C<< #VALUE! >>, fluidly
interpreting values as strings or integers, and using L<DateTime> instead of days-since-1900
numbers for dates, but most users probably won't mind. And, all these decisions are fairly
easy to change with a subclass.
(but if you want big changes, you should L<review your options|/"SEE ALSO"> to make sure you're
lib/Language/FormulaEngine.pm view on Meta::CPAN
does most of the perl code generation.
Defaults to an instance of L<Language::FormulaEngine::Compiler>.
You can initialize this attribute with a class instance, a class name, or arguments for the
default compiler.
=head1 METHODS
=head2 parse
my $formula= $fe->parse( $formula_text, \$error );
Return a L<Language::FormulaEngine::Formula|Formula object> representing the expression.
Dies if it can't parse the expression, unless you supply C<$error> then the error is
stores in that scalarref and the methods returns C<undef>.
=head2 evaluate
my $value= $fe->evaluate( $formula_text, \%variables );
This method creates a new namespace from the default plus the supplied variables, parses the
formula, then evaluates it in a recursive interpreted manner, returning the result. Exceptions
may be thrown during parsing or execution.
=head2 compile
my $coderef= $fe->compile( $formula_text );
Parses and then compiles the C<$formula_text>, returning a coderef. Exceptions may be thrown
during parsing or execution.
=head1 CUSTOMIZING THE LANGUAGE
The module is called "FormulaEngine" in part because it is designed to be customized.
The functions are easy to extend, the variables are somewhat easy to extend, the compilation
can be extended after a little study of the API, and the grammar itself can be extended with
some effort.
If you are trying to addd I<lots> of functionality, you might be starting with the wrong module.
lib/Language/FormulaEngine/Compiler.pm view on Meta::CPAN
$self->code_body(undef);
$self;
}
sub generate_coderef_wrapper {
my ($self, $perl, $subname)= @_;
$self->error(undef);
my $wrapper_method= $self->can('_output_wrapper__'.$self->output_api)
or Carp::croak("Unsupported output_api='".$self->output_api."'");
my $code= join "\n", $self->$wrapper_method(qq{# line 0 "compiled formula"\n$perl});
my $ret;
{
local $@= undef;
if (defined ($ret= $self->_clean_eval($code))) {
set_subname $subname, $ret if defined $subname;
} else {
$self->error($@);
}
}
return $ret;
lib/Language/FormulaEngine/Compiler.pm view on Meta::CPAN
$value= $coderef->($namespace);
$value= $coderef->(\%namespace_attributes);
This either uses the supplied namespace *instead* of the default, or merges attributes with
the default namespace via L<Language::FormulaEngine::Namespace/clone_and_merge>.
=back
=head2 optimize_var_access
By default, when a formula accesses a variable it will call L<Language::FormulaEngine::Namespace/get_value>
but for higher performance, you can have the formula directly access the variables hashref,
bypassing C<get_value>.
If this attribute is not set, the compilation will default to using the optimization if the
L</namespace> is using the default implementation of C<get_value> (i.e. has not been overridden
by a subclass) and the coderefs are a function of variables.
=head2 error
After a failed call to C<compile>, this attribute holds the error message.
lib/Language/FormulaEngine/Compiler.pm view on Meta::CPAN
After compilation, this attribute holds the perl source code that was generated prior to being
wrapped with the coderef boilerplate.
=head1 METHODS
=head2 compile( $parse_tree, $subname )
Compile a parse tree, returning a coderef. Any references to functions will be immeditely
looked up within the L</namespace>. Any references to constants in the L</namespace> will be
inlined into the generated perl. Any other symbol is assumed to be a variable, and will be
looked up from the L</namespace> at the time the formula is invoked.
See attribute C<output_api> for the signature and behavior of this coderef.
Because the generated coderef contains a reference to the namespace, be sure never to store
one of the coderefs into that namespace object, else you get a memory leak.
The second argument C<$subname> is optional, but provided to help encourage use of
L<Sub::Util/set_subname> for generated code.
=head2 reset
lib/Language/FormulaEngine/Error.pm view on Meta::CPAN
for (@subclasses) {
my $pkg= __PACKAGE__.'::'.$_;
no strict 'refs';
*$_= sub { @_? $pkg->new(@_) : $pkg }
}
}
use Exporter 'import';
our @EXPORT_OK= ( @subclasses, qw( auto_wrap_error ) );
our %EXPORT_TAGS= ( all => \@EXPORT_OK );
# ABSTRACT: Exception objects for formula functions
our $VERSION = '0.08'; # VERSION
has message => ( is => 'rw', required => 1 );
sub mine {
return Language::FormulaEngine::ErrorMine->new($_[0]);
}
sub BUILDARGS {
lib/Language/FormulaEngine/Error.pm view on Meta::CPAN
1;
__END__
=pod
=encoding UTF-8
=head1 NAME
Language::FormulaEngine::Error - Exception objects for formula functions
=head1 VERSION
version 0.08
=head1 DESCRIPTION
In keeping with the theme of spreadsheet formulas, this module collection provides exception
objects that can be used for similar exception handling. These objects are intended to be
thrown using "die", but they can also be wrapped with a "trap" object so that they don't die
until used. For example, in a spreadsheet the error is I<returned> from a function, but
anything that uses that value needs to generate a similar error. That would require a lot of
argument checking and prevent using native Perl operations, but by returning an error wrapped
in a trap, any perl operation that attempts to use the trap will instead throw the exception
object.
=head1 ATTRIBUTES
lib/Language/FormulaEngine/Error.pm view on Meta::CPAN
C<$error> reference.
=head1 EXPORTABLE FUNCTIONS
Each of the sub-classes of error has a constructor function which you can export from this
module. You can also take a perl-generated exception and automatically wrap it with an
appropriate Error object using L</auto_wrap_error>.
=head2 ErrInval
The formula was given invalid inputs
=head2 ErrNA
The function encountered a condition where no value could be returned. i.e. the function is
not defined for the supplied parameters, such as accessing elements beyond the end of an array.
=head2 ErrREF
The formula referenced a non-existent or nonsensical variable.
=head2 ErrNUM
The function expected a number (or specific range of number, like positive, integer, etc) but
was given something it couldn't convert.
=head2 ErrNAME
The formula uses an unknown function name. This is thrown during compilation, or during
evaluation if the compile step is omitted.
=head2 auto_wrap_error
my $err_obj= auto_wrap_error( $perl_error_text );
Look at the perl error to see if it is a known type of error, and wrap it with the appropriate
type of error object.
=head1 AUTHOR
lib/Language/FormulaEngine/Formula.pm view on Meta::CPAN
=head1 NAME
Language::FormulaEngine::Formula
=head1 VERSION
version 0.08
=head1 SYNOPSIS
$formula= $engine->parse($text_expression);
$value= $formula->evaluate(x => 1);
$formula2= $formula->simplify(y => 2);
$coderef= $formula2->compile;
=head1 DESCRIPTION
This is a convenient way to carry around the details of a parsed formula and later
evaluate it, simplify it, or compile it. It's simply a wrapper around the engine
that created it + the parse tree.
=head1 ATTRIBUTES
=head2 engine
Reference to a L<Language::FormulaEngine> instance.
=head2 orig_text
Original string of text that was parsed into this formula. This may be
C<undef> if the formula was generated. In that case, see L</deparse>
or L</to_string>.
=head2 parse_tree
Reference to the output of L<Language::FormulaEngine::Parser/parse>
=head2 functions
A set of { $name => 1 } for each named function used in this formula.
=head2 symbols
A set of { $name => 1 } for each named variable used in this formula.
=head1 CONSTRUCTOR
Standard Moo constructor accepts any of the attributes above.
=head1 METHODS
=head2 evaluate
$raw_value= $formula->evaluate;
$raw_value= $formula->evaluate(\%alt_vars);
$raw_value= $formula->evaluate($alt_namespace);
Evaluate the formula, optionally specifying variables or a different namespace in which
to evaluate it.
=head2 simplify
$formula2= $formula1->simplify;
$formula2= $formula1->simplify(\%alt_vars);
$formula2= $formula1->simplify($alt_namespace);
Simplify the formula by substituting known variable values and evaluating pure functions.
You can optionally specify variables or a different namespace which should be used.
=head2 compile
my $sub= $formula->compile;
my $sub= $formula->compile($subname);
Return an optimized perl coderef for the formula. The signature of the coderef
depends on the settings of the C<< $formula->engine->compiler >>. Throws an
exception if the compile fails.
=head2 deparse
Re-stringify the formula, using C<< $self->engine->parser >>.
=head2 to_string
Return either C<orig_text>, or C<deparse>. This is used when stringifying the object.
=head1 AUTHOR
Michael Conrad <mconrad@intellitree.com>
=head1 COPYRIGHT AND LICENSE
lib/Language/FormulaEngine/Namespace.pm view on Meta::CPAN
the prefix "fn_" or "eval_", and provides them case-insensitive. Named values are provided
from hashrefs of L</constants> and L</variables>, also case-insensitive.
You can subclass this (or just write a class with the same interface) to provide more advanced
lookup for the functions or values.
=head1 ATTRIBUTES
=head2 variables
A hashref of C<< name => value >> which formulas may reference. The keys should be lowercase,
and incoming variable requests will be converted to lowercase before checking this hash.
Variables will not be "compiled" into perl coderefs, and will be looked up from the namespace
every time a formula is evaluated.
=head2 constants
Same as L</variables>, but these may be compiled into coderefs.
=head2 die_on_unknown_value
Controls behavior of L</get_value>. If false (the default) unknown symbol names will resolve
as perl C<undef> values. If true, unknown symbol names will throw an
L<ErrREF exception|Language::FormulaEngine::Error/ErrREF>.
lib/Language/FormulaEngine/Namespace.pm view on Meta::CPAN
my $value= $namespace->evaluate_call( $Call_parse_node );
Evaluate a function call, passing it either to a specialized evaluator or performing a more
generic evaluation of the arguments followed by calling a native perl function.
=head2 simplify_call
$new_tree= $namespace->simplify_call( $parse_tree );
Create a simplified formula by reducing variables and evaluating
functions down to constants. If all variables required by the
formula are defined, and true functions without side effects, this
will return a single parse node which is a constant the same as
evaluate() would return.
=head2 simplify_symref
$parse_node= $namespace->simplify_symref( $parse_node );
This is a helper for the "simplify" mechanism that returns a parse
node holding the constant value of C<< $self->get_value($name) >>
if the value is defined, else passes-through the same parse node.
lib/Language/FormulaEngine/Parser.pm view on Meta::CPAN
=head2 symbols
A set (hashref) of all non-function symbols encountered. (variables, constnts, etc.)
=head2 reset
Clear the results of the previous parse, to re-use the object. Returns C<$self> for chaining.
=head2 deparse
my $formula_text= $parser->deparse($tree);
Return a canonical formula text for the parse tree, or a parse tree that you supply.
=head1 EXTENSIBLE API
These methods and attributes are documented for purposes of subclassing the parser.
=head2 input
The input string being scanned.
Code within the parser should access this as C<< $self->{input} >> for efficiency.
t/39-formula-object.t view on Meta::CPAN
[ 'average( 3, abs(x) + sin(zero) )',
{ abs => 1, average => 1, sin => 1, sum => 1 }, { x => 1, zero => 1 },
'average( 3, abs( x ) )', { abs => 1, average => 1 }, { x => 1 },
2
],
);
for (@tests) {
my ($expr_text, $fn_set, $var_set, $simplified_text, $s_fn_set, $s_var_set, $value)= @$_;
subtest '"'.escape_str($expr_text).'"' => sub {
ok( my $formula= $engine->parse($expr_text), 'parse' );
is( "$formula", $expr_text, 'to_string' );
is( $formula->functions, $fn_set, 'functions' );
is( $formula->symbols, $var_set, 'symbols' );
ok( my $simplified= $formula->simplify, 'simplify' );
is( $simplified->deparse, $simplified_text, 'simplified deparse' );
is( $simplified->functions, $s_fn_set, 'simplified functions' );
is( $simplified->symbols, $s_var_set, 'simplified symbols' );
is( $formula->evaluate(x => 1), $value, 'evaluate' );
ok( my $sub= $formula->compile, 'compile' );
is( $sub->(x => 1), $value, 'coderef-exec' );
done_testing;
};
}
done_testing;
}
test_parser();
t/50-custom-namespace.t view on Meta::CPAN
#-----------------------------------------------------------------------------
# Example of compiling with a custom namespace
{
package MyContext;
use Moo;
extends 'Language::FormulaEngine::Namespace';
sub fn_customfunc { return "arguments are ".join(', ', @_)."\n"; }
};
subtest custom_namespace_function => sub {
my $engine= Language::FormulaEngine->new(namespace => MyContext->new);
my $formula= $engine->compile( 'CustomFunc(baz,2,3)' );
is( $formula->({ baz => 42 }), "arguments are 42, 2, 3\n", 'correct result' );
};
#-----------------------------------------------------------------------------
# Example of compiling with overridden get_value
#
{
package MyContext2;
use Moo;
extends 'Language::FormulaEngine::Namespace';
sub fn_customfunc { return "arguments are ".join(', ', @_)."\n"; }
sub get_value { return 1 + shift->next::method(@_); }
};
subtest custom_namespace_get_var => sub {
my $engine= Language::FormulaEngine->new(
namespace => { CLASS => 'MyContext2' },
);
my $formula= $engine->compile( 'CustomFunc(baz,2,3)' );
is( $formula->({ baz => 42 }), "arguments are 43, 2, 3\n", 'correct result' )
or diag 'code_body = '.$engine->compiler->code_body;
};
#-----------------------------------------------------------------------------
# Example of compiling with overridden get_value using deprecated
# variables_via_namespace compile option.
#
subtest deprecated_variables_via_namespace => sub {
my $engine= do {
local $SIG{__WARN__}= sub {}; # suppress the warning about variables_via_namespace being depricated
Language::FormulaEngine->new(
namespace => { CLASS => 'MyContext2' },
compiler => { variables_via_namespace => 1 }
);
};
my $formula= $engine->compile( 'CustomFunc(baz,2,3)' );
is( $formula->(variables => { baz => 42 }), "arguments are 43, 2, 3\n", 'correct result' )
or diag 'code_body = '.$engine->compiler->code_body;
};
#-----------------------------------------------------------------------------
# Check whether any of the above tests left behind any FormulaEngine objects
#
subtest leak_check => sub {
skip_all "Devel::Gladiator is not available"
unless eval { require Devel::Gladiator; };
my $current_arena= Devel::Gladiator::walk_arena();
my @leaked_objects= grep ref($_) =~ /^Language::FormulaEngine/, @$current_arena;
# Note: checking for leftover FormulaEngine objects also effectively checks for leftover
# compiled formulas, because a compiled formula holds a reference to the Namespace
ok( 0 == @leaked_objects, 'all formula engine objects cleaned up' )
or diag @leaked_objects;
@$current_arena= ();
};
done_testing;
( run in 0.334 second using v1.01-cache-2.11-cpan-3cd7ad12f66 )