App-Test-Generator
view release on metacpan or search on metacpan
via eval "require $package" (string eval); the package name is now
validated against Perl's package-name grammar and the module is
loaded by converting it to a file path and calling require on that,
so it is never compiled as arbitrary Perl source.
- Fix test-generator-mutate --changed_only interpolating --base_sha
into a backtick-executed git diff command; --base_sha is now
validated against a restrictive character class and git diff is
invoked via list-form open() instead of the shell.
- Fix test-generator-index _mutant_file_report() computing a
slash-sanitised $filename that was never used, leaving the report
path built from the raw, unsanitised $file; it now rejects any '..'
path segment in $file and verifies the resolved output path stays
under the report directory before writing.
- Fix extract-schemas _load_target_module() losing the drive letter
when reconstructing the lib-dir candidate with File::Spec->catdir(),
so the @INC-walking loop never found a 'lib' directory on Windows
when the input file and the cwd were on different drives; now uses
catpath() to keep the volume attached.
- Fix SchemaExtractor _infer_type_from_expression() and
_parse_modern_signature() using hand-rolled brace/bracket depth
counters to split comma-separated lists, which mishandled nested
brackets (e.g. a signature default value containing a hashref
literal could be split on the comma inside the hashref); both now
use Text::Balanced::extract_bracketed() to skip over nested
structures correctly.
- Fix Mutator.pm's _dedup_mutants(), _is_redundant_mutation(), and
apply_mutant() reaching into Mutant objects via direct hash access
(e.g. $m->{line}), bypassing the class's own accessor methods; all
six call sites now use the Mutant accessors ($m->line, $m->original,
$m->transform, etc).
- Fix Devel::App::Test::Generator::LCSAJ::Runtime's DB::DB debugger
hook calling abs_path() (a filesystem stat) on every single
statement executed under the debugger; resolved paths are now
memoised per raw $file in %NORM_CACHE.
- Fix lib/App/Test/Generator/Sample/Module.pm declaring
package Test::App::Generator::Sample::Module (reversed word order)
instead of App::Test::Generator::Sample::Module, so the package
name did not match its location under
lib/App/Test/Generator/Sample/; renamed throughout the file.
- Fix Generator.pm SYNOPSIS extract-schemas example passing two
positional arguments (bin/extract-schemas lib/Sample/Module.pm)
instead of one, referencing a nonexistent lib/Sample/Module.pm path,
and naming the generated schema file schemas/greet.yaml when
extract-schemas actually writes schemas/greet.yml; corrected all
three.
- Fix lib/App/Test/Generator/Exporter/YAML.pm having no package
statement (so it loaded into main::), no input validation before
YAML::XS::DumpFile(), and no test coverage; added the package
statement, Params::Validate::Strict validation of plan/file
(rejecting non-hashref plans and missing/empty file paths), and
t/Exporter-YAML.t.
- Fix generate()'s _assert_identifier() check on the function name
rejecting fully-qualified sub names such as DB::DB, breaking
t/app.t's self-test sweep against
Devel::App::Test::Generator::LCSAJ::Runtime.pm (DB::DB is a Perl
debugger hook, always installed into the DB:: package regardless
of its source package); the function-name check now allows "::"
separators, same as the existing module-name check.
- Fix SchemaExtractor's heuristic numeric-type inference missing
parameters that are only ever validated via an explicit
looks_like_number($param) call rather than direct arithmetic
adjacency (e.g. a parameter used as looks_like_number($b) in a
guard clause and only later folded into arithmetic via
$a + ($b // 0), where $b never sits next to an operator itself);
such parameters fell back to type 'string', so fuzz-generated
non-numeric inputs caused generated tests to fail against code
that legitimately dies on non-numeric input. An explicit
looks_like_number($param) call in the source is now treated as a
direct numeric-type assertion, checked before the existing
arithmetic-operator and comparison heuristics.
- Fix _compile_signature_isolated()'s "fast path" Safe compartment,
tried before falling back to the subprocess unconditionally;
Type::Params/Types::Common pull in XS modules and Safe cannot host
XS/dynamic loading, so the compartment never succeeded for any real
signature_for() declaration and was dead code giving a false
impression of sandboxing. Removed; the subprocess path (gated on
allow_signature_exec => 1) is now the only path.
- Fix Generator.pm splicing module/function/transform/field names
unescaped into generated test source; added _assert_identifier()
and applied it before every such splice point so a name that is
not identifier-shaped now croaks instead of producing a test file
with injected code.
- Fix extract-schemas2 calling Planner->new without the required
package argument, crashing on every invocation; package is now
passed through from the extracted schema.
- Fix Template.pm _dedup_cases() always being a no-op: a return
inside the eval{} block returned from the eval, not from
_dedup_cases, so the deduplicated result was discarded and the
unduplicated $cases was always returned to the caller.
- Fix SchemaExtractor _detect_dependencies() relationship detection
losing pos() state on a shared $code string across nested global
matches; affected loops now operate on independent copies/resets.
- Fix SchemaExtractor numeric boundary-value hints double-counting
values already present from a previous hint pass; boundary values
are now added through a seen-value set, so re-running hint
generation on the same schema no longer duplicates values.
- Fix SchemaExtractor _detect_external_object_dependency() not
resetting pos($method_body) before its global match, so a prior
match elsewhere on the same string could cause this detector to
start scanning mid-string and miss leading dependencies; pos() is
now explicitly reset to 0 first.
- Fix CoverageGuidedFuzzer calling target_sub with no time bound, so
a target that hangs (e.g. on an unexpected infinite-loop input)
hung the whole fuzz run; calls are now wrapped in alarm()-based
timeout (default 5s, configurable via timeout => N, 0 disables).
- Fix LCSAJ.pm's CFG builder connecting every branch point only to a
newly-created true-block successor, leaving the false-arm block
disconnected from the frontier and silently dropped as an empty
leaf with no lines or edges; both true and false successors are
now wired into the frontier so subsequent statements are recorded
against both arms.
- Fix Mutator generate_mutants() never calling each mutation
strategy's applies_to() pre-filter before mutate(), and never
populating the context/line_content fields on the Mutant objects
it constructs; both are now wired through.
- Fix Analyzer::SideEffect.pm and Analyzer::Complexity.pm counting
IO/exec keywords, branch/logic/exception keywords, and the
literal ? character when they merely appeared inside a string
literal or comment (e.g. "Are you sure?" or a # comment containing
"die"); source is now stripped of string and comment content
before these keyword/operator counts are taken.
- Fix Analyzer::Return.pm's \$self(?!->) returns_self pattern
matching the \$self prefix of any longer variable name (e.g.
\$self_backup, \$selfish); added a \b word boundary. Also fixed
all three return-pattern matches running as a single non-/g match,
so a method with several qualifying return statements only ever
contributed one evidence entry instead of one per occurrence.
- Fix Mutation::BooleanNegation and Mutation::ReturnUndef mutating
which previously carried only an internal TODO-style comment
despite being a public method.
- Add Notes sections to Mutation::BooleanNegation, ::ConditionalInversion,
::NumericBoundary, and ::ReturnUndef documenting the context and
line_content fields each mutant carries for Mutator's fast-mode
dedup, and to CoverageGuidedFuzzer.pm documenting that a
target_sub die is only recorded as a bug when the triggering input
is schema-valid.
0.38 Mon May 18 21:19:40 EDT 2026
[Bug fixes]
- Fix missing TER1/TER2/TER3 column in mutation dashboard
- remove duplicate _lcsaj_coverage_for_file calls
- move skipped-annotation note inside summary div
- Fix MUTANT_SKIP_BEGIN/END regex to match only lines where the annotation is the entire content,
preventing false positives in comments and POD
0.37 Mon May 18 14:58:48 EDT 2026
[Enhancements]
- Add MUTANT_SKIP_BEGIN / MUTANT_SKIP_END source annotations to exclude
lines from mutation testing; mismatched markers are fatal. Skipped
line counts appear in the per-file mutation report and summary table.
[Bug fixes]
- Fix fuzz schema generation looking in xt/conf instead of t/conf.
- Fix mutate.yml for external repos: add commit step so mutation.json
is persisted after each run, enabling the dashboard workflow_run
trigger to pick up fresh results.
- Fix BSD::Resource dependency failing on Windows; guard with OS check.
0.36 Sat May 9 18:54:04 EDT 2026
[Bug fixes]
- Fix uninitialized $version warning in CPAN Testers "no failures"
message when the CPAN API returns a 404 (no release data available).
- Fix "Can't open lib/App/Test/Generator.pm" error when test-generator-index
is run from a repository other than ATG itself; ATG version is now
found by searching @INC with fallback to the configured module_file path.
0.35 Sat May 9 09:40:08 EDT 2026
[Enhancements]
- Dashboard footer now shows the App::Test::Generator version used
to generate it, linking to the MetaCPAN distribution page.
- Executive Summary wording now varies by mutation score: positive
framing for high scores, an actionable hint for low scores, and a
clear message when no mutation data is available yet. Thresholds
use med_threshold and low_threshold from %config rather than
magic numbers.
- Coverage dashboard per-file HTML links now work correctly; Devel::Cover
instrumentation restored to use cover -test for accurate per-file
HTML generation.
- Dashboard file links now correctly target blib-lib-* filenames as
generated by Devel::Cover, fixing persistent 404 errors on all
per-file coverage pages.
- generate-test-dashboard now derives the Devel::Cover -select pattern
dynamically from GITHUB_REPOSITORY, making the script portable across
CPAN distributions without hardcoded module paths.
- Fuzz schema generation now correctly looks in t/conf rather than
xt/conf for existing schemas to augment.
- Redundant exclusion of mutant_*.t from prove invocation removed;
mutant stubs are in xt/ and were never matched by the t/ find anyway.
[Bug fixes]
- Fix https://www.cpantesters.org/cpan/report/04c7279a-476f-11f1-bf55-cb595875c975
Make t/type_params.t an extended test
0.34 Sun May 3 10:30:24 EDT 2026
[Bug fixes]
- Emitter::Perl: _emit_void_test() generated ok(!defined $result || 1, ...)
which is a tautology and always passes regardless of return value.
Fixed to ok(!defined $result, '$method returns nothing (void)').
- Generator: schema files named after Perl builtins (e.g. abs.yml) caused
generated tests to emit "use abs" which fails to compile. Added
_is_perl_builtin() guard to prevent builtin names being used as module
names in use_ok() calls.
- bin/test-generator-index: mutant stub generator incorrectly called
new_ok() for class methods such as App::Test::Generator::generate().
Added _is_class_method() helper to detect $class vs $self and emit
the correct calling convention in generated stubs.
- bin/test-generator-index: CPAN Testers failure table showed all
expected NA rows for Perl < 5.036 as noise. Rows below the detected
Perl version cliff are now omitted with a count notice, leaving only
unexpected failures visible.
- CoverageGuidedFuzzer: _load_json_module() used require $mod where $mod
contained '::' which Perl does not convert to path separators for
variable require. Fixed with s{::}{/}g conversion before require.
[Enhancements]
- Comprehensive test suite added across all modules:
- Function tests (white-box) for all private helpers
- Black-box unit tests for every public method against POD API spec
- Integration tests covering 10 end-to-end pipelines
- Edge case / destructive / boundary-condition tests
- Extended tests targeting surviving mutants and coverage gaps
- mutate.yml and dashboard.yml are now portable to any CPAN module.
Copying both files to another module's .github/workflows/ directory
will produce a working mutation test run and coverage dashboard
without modification. ATG-specific behaviour is gated on
github.repository = nigelhorne/App-Test-Generator.
- dashboard.yml: added missing local/bin PATH setup so installed
ATG scripts are found correctly on non-ATG modules.
- mutate.yml: Sample/Module.pm excluded from mutation scoring to
remove 30 artificial survivors from the overall score.
- Emitter::Perl mutation score improved from 37.5% to 100% through
targeted direct assertions on all _emit_*() helper return values.
- t/test-generator-index.t: 33 new tests for detect_perl_version_cliff(),
parse_version(), extract_perl_versions(), make_key(),
confidence_score(), and perldelta_url().
0.33 Tue Apr 21 16:09:24 EDT 2026
- Fixed NumericBoundary mutator incorrectly identifying the '<' in
open() mode strings and readline operators (<$fh>) as numeric
comparison operators, producing invalid mutants that caused
compilation errors in the mutated code. Two guards added to
mutate(): skip any operator whose next sibling is a PPI symbol
token (catching <$fh>), and skip operators not parented by a
( run in 1.207 second using v1.01-cache-2.11-cpan-6aa56a78535 )