App-Test-Generator
view release on metacpan or search on metacpan
0.38 Mon May 18 21:19:40 EDT 2026
[Bug fixes]
- Fix missing TER1/TER2/TER3 column in mutation dashboard
- remove duplicate _lcsaj_coverage_for_file calls
- move skipped-annotation note inside summary div
- Fix MUTANT_SKIP_BEGIN/END regex to match only lines where the annotation is the entire content,
preventing false positives in comments and POD
0.37 Mon May 18 14:58:48 EDT 2026
[Enhancements]
- Add MUTANT_SKIP_BEGIN / MUTANT_SKIP_END source annotations to exclude
lines from mutation testing; mismatched markers are fatal. Skipped
line counts appear in the per-file mutation report and summary table.
[Bug fixes]
- Fix fuzz schema generation looking in xt/conf instead of t/conf.
- Fix mutate.yml for external repos: add commit step so mutation.json
is persisted after each run, enabling the dashboard workflow_run
trigger to pick up fresh results.
- Fix BSD::Resource dependency failing on Windows; guard with OS check.
0.36 Sat May 9 18:54:04 EDT 2026
[Bug fixes]
- Fix uninitialized $version warning in CPAN Testers "no failures"
message when the CPAN API returns a 404 (no release data available).
- Fix "Can't open lib/App/Test/Generator.pm" error when test-generator-index
is run from a repository other than ATG itself; ATG version is now
found by searching @INC with fallback to the configured module_file path.
0.35 Sat May 9 09:40:08 EDT 2026
[Enhancements]
- Dashboard footer now shows the App::Test::Generator version used
to generate it, linking to the MetaCPAN distribution page.
- Executive Summary wording now varies by mutation score: positive
framing for high scores, an actionable hint for low scores, and a
clear message when no mutation data is available yet. Thresholds
use med_threshold and low_threshold from %config rather than
magic numbers.
- Coverage dashboard per-file HTML links now work correctly; Devel::Cover
instrumentation restored to use cover -test for accurate per-file
HTML generation.
- Dashboard file links now correctly target blib-lib-* filenames as
generated by Devel::Cover, fixing persistent 404 errors on all
per-file coverage pages.
- generate-test-dashboard now derives the Devel::Cover -select pattern
dynamically from GITHUB_REPOSITORY, making the script portable across
CPAN distributions without hardcoded module paths.
- Fuzz schema generation now correctly looks in t/conf rather than
xt/conf for existing schemas to augment.
- Redundant exclusion of mutant_*.t from prove invocation removed;
mutant stubs are in xt/ and were never matched by the t/ find anyway.
[Bug fixes]
- Fix https://www.cpantesters.org/cpan/report/04c7279a-476f-11f1-bf55-cb595875c975
Make t/type_params.t an extended test
0.34 Sun May 3 10:30:24 EDT 2026
[Bug fixes]
- Emitter::Perl: _emit_void_test() generated ok(!defined $result || 1, ...)
which is a tautology and always passes regardless of return value.
Fixed to ok(!defined $result, '$method returns nothing (void)').
- Generator: schema files named after Perl builtins (e.g. abs.yml) caused
generated tests to emit "use abs" which fails to compile. Added
_is_perl_builtin() guard to prevent builtin names being used as module
names in use_ok() calls.
- bin/test-generator-index: mutant stub generator incorrectly called
new_ok() for class methods such as App::Test::Generator::generate().
Added _is_class_method() helper to detect $class vs $self and emit
the correct calling convention in generated stubs.
- bin/test-generator-index: CPAN Testers failure table showed all
expected NA rows for Perl < 5.036 as noise. Rows below the detected
Perl version cliff are now omitted with a count notice, leaving only
unexpected failures visible.
- CoverageGuidedFuzzer: _load_json_module() used require $mod where $mod
contained '::' which Perl does not convert to path separators for
variable require. Fixed with s{::}{/}g conversion before require.
[Enhancements]
- Comprehensive test suite added across all modules:
- Function tests (white-box) for all private helpers
- Black-box unit tests for every public method against POD API spec
- Integration tests covering 10 end-to-end pipelines
- Edge case / destructive / boundary-condition tests
- Extended tests targeting surviving mutants and coverage gaps
- mutate.yml and dashboard.yml are now portable to any CPAN module.
Copying both files to another module's .github/workflows/ directory
will produce a working mutation test run and coverage dashboard
without modification. ATG-specific behaviour is gated on
github.repository = nigelhorne/App-Test-Generator.
- dashboard.yml: added missing local/bin PATH setup so installed
ATG scripts are found correctly on non-ATG modules.
- mutate.yml: Sample/Module.pm excluded from mutation scoring to
remove 30 artificial survivors from the overall score.
- Emitter::Perl mutation score improved from 37.5% to 100% through
targeted direct assertions on all _emit_*() helper return values.
- t/test-generator-index.t: 33 new tests for detect_perl_version_cliff(),
parse_version(), extract_perl_versions(), make_key(),
confidence_score(), and perldelta_url().
0.33 Tue Apr 21 16:09:24 EDT 2026
- Fixed NumericBoundary mutator incorrectly identifying the '<' in
open() mode strings and readline operators (<$fh>) as numeric
comparison operators, producing invalid mutants that caused
compilation errors in the mutated code. Two guards added to
mutate(): skip any operator whose next sibling is a PPI symbol
token (catching <$fh>), and skip operators not parented by a
PPI::Statement, PPI::Structure::Condition, or
PPI::Structure::Block (catching quoted mode strings in open()
argument lists). The same readline guard is also applied inside
the transform closure to prevent mutation at apply time if PPI
tokenisation differs between the analysis and mutation documents.
- Rationalised program names so that they can be used in other modules
without the need to have copies
- Added detect_prereq_version_cliffs() to cross-reference declared
prerequisites from the MetaCPAN API against installed module
Added mutation levels.
Setting to fast will reduce the number of mutants by deduping and removing unnecessary mutants.
See App::Test::Generator::Mutator::_is_redundant_level for a list of those optimised out
Added basic LSCAJ data to the test dashboard.
Added simple security string testing.
Don't push too hard on builtins as they don't have good parameter validation.
Ensure adding only the byte order marker honours $min
More use of _DESCRIPTION
0.29 Thu Feb 26 12:57:59 EST 2026
Added mutation testing
Added guided testing to extract-schemas
Some routines were incorrectly labelled as getter routines
Added more edge cases
Added fallback to extract parameters from classic Perl body styles
Added Type::Param support (https://github.com/nigelhorne/App-Test-Generator/issues/4)
Getter routines take no arguments
Fixed string testing when both min and max are given
Don't give $class or $self as parameters
No input/output no longer croaks, because there are now a few tests that can be run
Add a basic hashref to mandatory args
Use UUID::Tiny and Readonly::Values::Boolean
0.28 Mon Feb 9 19:54:59 EST 2026
Latest test dashboard
Some getter/setter routines were missed
Test getset routines
Added 'isa' test. Tests code dies when passed the wrong type of object
0.27 Thu Jan 29 07:54:01 EST 2026
Fewer false positives for "optional"
Sometimes "new:" was missing
If a return value is 'length($foo)' it's an integer with a minimum value of 0
Some random strings were mistakenly sent as tests that were not in the memberof set
0.26 Mon Jan 12 07:57:54 EST 2026
Added strict_pod mode to SchemaExtractor, which croaks if it finds discrepancies between the POD and code
Improved detection of optional/required arguments
Improved detection of integer output types
Improved POD parser of parameters
Added --dry-run
Improved config validation
Better call of object methods in transforms
Test the sanity of Unicode outputs
Idempotent tests
Prevent silent duplicate method overwrites
Implement confidence_threshold
Refactor _detect_accessor_methods
Only generate one of call_code and position_code
Check for global side effects
Sanity test on DIES
Added timeout tests to ensure the test doesn't hang
0.25 Thu Jan 1 15:41:06 EST 2026
Ternary ? 1 : 0 return is taken to be an indicator of boolean returns
Now parses this pod to know that it's supposed to return a string:
Returns the sanitized HTML string
Use :: to call methods in non-oo modules, rather than ->
Flag when type is set to object by can is not set
0.24 Sun Dec 28 15:10:09 EST 2025
Return chances of false positive file path semantics
If the type of an variable can't be determined, guess at string, but lower the confidence level
Error Message Extraction:
Capture error messages from die/croak/confess
Use them to understand what makes input invalid die "Age must be positive" if $age <= 0;
Store that a negative/zero age is invalid
Use test of scalar(@_) to check for number of optional/mandatory args
Throw a float at a routine that only takes integers - it should error
Added example extraction
Parse POD for example calls and extract valid parameter combinations =head2 SYNOPSIS my $obj = Module->new(foo => 'bar', count => 5);
Store these as known-good test values
Corpus bool testing now uses ok rather than is
Ensure new doesn't refer to other packages
Set the isa property in the new function
0.23 Wed Dec 24 10:48:03 EST 2025
Fix https://www.cpantesters.org/cpan/report/079dba46-d92f-11f0-9642-dde56d8775ea
Don't give false negative that memberof isn't known
SchemaExtractor output files now end in .yml not .yaml
Ensure rand_str isn't called when memberof is set
When test_empty is set, do the right thing for memberof
0.22 Sun Dec 21 20:59:58 EST 2025
Added better API to generate
Added parameter relationship detection (mutually exclusive, required groups, conditional requirements, dependencies, value constraints)
Improved Object Detection
Detect factory methods that return instances
Recognize singleton patterns
Identify when constructor needs specific parameters
Handle inheritance (when parent class new() is needed)
Better Default Value Extraction
$param = $param || 'default_value'
$param //= 'default_value'
$param = defined $param ? $param : 'default'
Expanded _detect_list_context to
Better distinguish scalar vs list returns
Detect void context methods (those that modify state)
Recognize chaining patterns more reliably Identify error return conventions (undef on failure, etc.)
Add confidence scoring transparency, explain WHY confidence is what it is:
_analysis: confidence_factors:
- "POD documentation present (+30)"
- "Type validation in code (+20)"
- "No constraint information (-10)"
Add support for:
- Signatures (sub foo($x, $y = 5) { })
- Postfix dereferencing ($array->@*)
- Subroutine attributes (sub foo :Returns(Int) { })
- Field declarations (Perl 5.38+)
Added a new section to generated YAML:
yamltest_hints: boundary_values: [0, 1, 100, 255]
detected from code invalid_inputs: ['', undef, -1]
from validation checks equivalence_classes: []
0.21 Sun Dec 14 08:07:09 EST 2025
Schemaextractor: don't put the package name as the argument
Validate config settings better
Fix max string testing with non-ASCII characters
Changed rand_str to be a unified generator that randomly produces codepoint strings,
grapheme clusters, ZWJ emoji sequences, or aggressive Unicode fuzz strings
Schemaextractor: Added advanced type detection for DateTime objects, file handles, coderefs, and enum validation patterns
Added enum as a synonym of memberof
Added tests for unix_timestamp semantic type
0.20 Fri Dec 5 07:53:43 EST 2025
Added the --version flag to fuzz-harness-generator
Ensure the max value of string is honoured better
Fix array context detection to only match return statements
Improve chances of detecting a boolean output
Make the list context detection more specific
0.19 Wed Nov 26 07:50:22 EST 2025
Fixed the loop iterating over mandatory args
Ensure mandatory_args honours matches
0.18 Tue Nov 25 11:58:12 EST 2025
Removed one place sending '' if test_empty was disabled
Don't send long strings if matches is set
Fixed handling of position code with more than argument
Renamed the sample module
0.17 Tue Nov 25 10:24:36 EST 2025
Improved TEST_VERBOSE output for Corpus tests
Make the strings more random
Added test_non_ascii (default 1) to the configuration matrix
Removed all legacy conf code
Re-ordered the POD
Don't give 42 if max < 42
Added App::Test::Generator::SchemaExtractor
0.16 Thu Nov 20 08:38:17 EST 2025
Use Data::Random::String to generate the string
Use Data::Random::Structure to generate references to hashes and arrays
Remove legacy support for Perl config files
Handle the case when all inputs should cause a die(),
allows App::Test::Generator to sanity test itself
If something is expected to die, don't look at its return code
More boolean edge cases
Don't pass _NAME to validate_strict
When a routine dies, it shouldn't return anything
Begin to use semantic types
Use Test::LectroTest to create more tests
Added custom properties
Separated the Template into its own class to ease maintenance
Use reusable functions to generate cases
Rand_set returns and array, so look at the first element
0.15 Tue Nov 11 19:30:18 EST 2025
Documented how to Fuzz test a CPAN module
populate_positions - don't attempt to deref a scalar
carp when undef is a corpus test case, but test_undefs is not set
Improved handling of cases array for WARNS and DIES
Improved handling of validating position settings in config files
Croak if the schema_file can't be opened
Close stdin and use /dev/null
Fixed logic in testing short strings when min >= 2
0.14 Fri Nov 7 20:26:32 EST 2025
Fixed logic when neither memberof nor notmemberof were defined,
which could cause an empty notmemberof array to be created
Transform: better number of items for arrayrefs
At least one of module and function must be defined
Don't send wrong data types in transform testing built ins
Document the edge_cases_array and ensure all tests in that array are run
Better handing of "input: undef" in the config file
Fix handling of matches in output schemas
0.13 Tue Nov 4 11:52:49 EST 2025
Added better documentation for transforms
transform: filled in more types for the foundation set of data
Added "value" to the transform keywords
Refactored and enabled the code to validate that $module exists and is installed
Added deprecation notice about loading configs from perl code
Allow yes/no as booleans in schema config settings
Fix sprintf issue when the text contains a % sign
0.12 Sun Nov 2 19:38:10 EST 2025
undef can now be used to indicate no input or output
Sending with no args if now configured using "test_undef" rather than "test_empty"
Use Data::Random::String::Matches to create random string that matches a regex
Flag keywords that are yet to be handled
$module can now be set to "builtin"
( run in 1.891 second using v1.01-cache-2.11-cpan-600a1bdf6e4 )