view release on metacpan or  search on metacpan

bin/test-generator-index  view on Meta::CPAN

C<test-generator-index> generates an HTML test coverage dashboard for
publication on GitHub Pages, combining four sources of test quality
data into a single report:

=over 4

=item * B<Statement and branch coverage> from L<Devel::Cover>, showing
which lines and branches were exercised by the test suite.

=item * B<LCSAJ path coverage> (TER3) from the LCSAJ runtime debugger,
showing which control-flow paths were executed. Displayed as blue
(covered) or red (uncovered) dots on per-file mutation pages.

=item * B<Mutation testing results> from C<bin/test-generator-mutate>,
showing which injected faults the test suite detected (killed) and
which it missed (survived).

=item * B<CPAN Testers failure analysis>, showing which Perl versions
and operating systems are failing, with automatic root cause detection
including Perl version cliffs, locale sensitivity, and dependency
version cliffs.

=back

In addition to the dashboard, the script drives the mutation-guided
test generation pipeline described in
L<App::Test::Generator/MUTATION-GUIDED TEST GENERATION>. Surviving
mutants are used to automatically generate new test stubs and fuzz
schemas that target the exact boundary conditions the test suite
missed:

=over 4

=item * C<--generate_mutant_tests=DIR> produces TODO stub files in
C<DIR/> for all surviving mutants, grouped by source file and
deduplicated by line.

=item * C<--generate_test=mutant> additionally attempts to produce
runnable YAML schemas in C<DIR/conf/> for NUM_BOUNDARY survivors
using L<App::Test::Generator::SchemaExtractor>.

=item * C<--generate_fuzz> augments existing schemas in C<DIR/conf/>
with boundary values from surviving mutants, writing timestamped
copies that are picked up automatically by C<t/fuzz.t>.

=back

The script is designed to be shared across projects. Copy it into the
C<scripts/> directory of each project that uses it:

    cp ../App-Test-Generator/scripts/test-generator-index scripts/

It is invoked automatically by C<scripts/generate_test_dashboard> on
each CI push via C<.github/workflows/dashboard.yml>.

=head1 SYNOPSIS

  https://$github_user.github.io/$github_repo/coverage/

=head1 INPUTS

  cover_html/cover.json     - Devel::Cover JSON report (statement/branch/condition)
  mutation.json             - Mutation testing results from test-generator-mutate
  cover_html/lcsaj_hits.json - LCSAJ path hit data from the LCSAJ runtime debugger
  cover_html/mutation_html/lib/ - Per-file LCSAJ path definitions (.lcsaj.json)
  coverage_history/*.json   - Historical coverage snapshots for the trend chart

=head1 OUTPUTS

  cover_html/index.html     - Main dashboard (coverage table, trend chart,
                              CPAN Testers failures, mutation report)
  cover_html/mutation_html/ - Per-file mutation heatmap pages

=head1 OPTIONS

  --generate_mutant_tests=DIR
      Generate a timestamped test stub file in DIR (typically 'xt/') for
      surviving mutants. The file is named mutant_YYYYMMDD_HHMMSS.t and
      contains:
        - TODO test stubs for High/Medium difficulty survivors, with
          boundary value suggestions, environment variable hints, and
          the enclosing subroutine name for navigation context
        - Comment-only hints for Low difficulty survivors
      Multiple mutations on the same source line are deduplicated into
      one stub - one good test kills all variants on that line.
      File is skipped entirely if there are no survivors to report.
      If not given, no test stubs are generated.

  --generate_test=CLASS
      When combined with --generate_mutant_tests=DIR, attempts to produce
      runnable test artefacts for surviving mutants rather than TODO stubs.

      Currently supported classes:

        mutant  For NUM_BOUNDARY survivors, calls
                App::Test::Generator::SchemaExtractor to extract the schema
                for the enclosing subroutine and augments it with the
                boundary value from the mutant (plus one value either side).
                The resulting YAML schema is written to DIR/conf/ and is
                picked up automatically by t/fuzz.t on the next test run.
                Falls back to a TODO stub if SchemaExtractor fails, the
                enclosing sub cannot be determined, or the extracted schema
                confidence is too low (very_low or none).

      This option is designed to accept additional classes in future, for
      example corpus-driven or property-based test generation.
      If not given, only TODO stubs are produced.

   --generate_fuzz
      Scans t/conf/ for existing YAML schema files and augments copies
      of them with boundary values extracted from surviving NUM_BOUNDARY
      mutants whose enclosing subroutine matches the schema's function
      field. The original schema is never modified. Augmented copies are
      written to xt/conf/mutant_fuzz_YYYYMMDD_HHMMSS_FUNCTION.yml and
      are picked up automatically by t/fuzz.t on the next test run.

      Schemas whose filename already starts with mutant_fuzz_ are skipped
      to avoid augmenting previously augmented schemas. Schemas with no
      matching survivors are skipped (with a note if --verbose is active).
      New boundary values are merged into whichever edge key already
      exists in the schema (edge_case_array or edge_cases), with
      deduplication against existing values.

      This flag is independent of --generate_test and can be used alone.

=head1 DASHBOARD SECTIONS

  Coverage Table    - Per-file statement/branch/condition/subroutine

bin/test-generator-index  view on Meta::CPAN

	return 'Boolean logic mutation survived. Add tests covering both true and false paths.' if $type =~ /BOOL|NEGATION/;

	return 'Comparison mutation survived. Verify equality and inequality cases explicitly.' if $type =~ /COMPARE|EQUAL/;

	return 'Mutation survived. Add targeted tests to validate this branch.'
}

# --------------------------------------------------
# _suggest_test
#
# Generate a short Perl code snippet
#     suggesting a test that would kill a
#     surviving mutant, based on its mutation
#     type. Displayed in the expandable mutant
#     details panel on per-file heatmap pages.
#
# Entry:      $m - mutant hashref containing at least
#             an 'id' field. The 'type' field is used
#             if present; otherwise type is inferred
#             from the ID prefix.
#
# Exit:       Returns a string of Perl test code, or
#             undef if no suggestion is available for
#             this mutation type.
#
# Side effects: None.
# --------------------------------------------------
sub _suggest_test {
	my $m = $_[0];

	# ---------------------------------------------------------
	# Determine mutation type
	# ---------------------------------------------------------

	# Prefer explicit type if present
	my $type = $m->{type};

	# Fallback: infer from ID prefix
	if((!$type || (length($type) == 0)) && $m->{id}) {
		($type) = $m->{id} =~ /^([A-Z_]+)/;
	}

	# ---------------------------------------------------------
	# Boundary condition mutation
	# ---------------------------------------------------------

	if($type && $type =~ /NUM|BOUNDARY/) {
		return <<"TEST";
# Boundary test suggestion
is( func(VALUE_AT_BOUNDARY), EXPECTED, 'Test boundary behaviour' );
TEST
	}

	# ---------------------------------------------------------
	# Boolean mutation
	# ---------------------------------------------------------

	if($type && $type =~ /BOOL|NEGATION/) {
		return <<"TEST";
# Boolean branch test suggestion
ok( !func(INPUT), 'Verify boolean branch behaviour' );
TEST
	}

	# ---------------------------------------------------------
	# Return value mutation
	# ---------------------------------------------------------

	if($type && $type =~ /RETURN/) {
		return <<"TEST";
# Return value assertion
is( func(INPUT), EXPECTED, 'Verify correct return value' );
TEST
	}

	return;
}

# --------------------------------------------------
# _survivor_class
#
# Purpose:    Map a survivor count to the appropriate
#             CSS class name for colour-coding a
#             source line in the mutation heatmap.
#             Higher counts map to darker red classes.
#
# Entry:      $count - number of surviving mutants on
#             a single source line.
#
# Exit:       Returns a CSS class name string:
#             'survived-1', 'survived-2', or
#             'survived-3' (for 3 or more).
#             Returns 'survived' as a fallback for
#             zero (should not occur in practice).
#
# Side effects: None.
# --------------------------------------------------
sub _survivor_class {
	my $count = $_[0];

	return 'survived-1' if $count == 1;
	return 'survived-2' if $count == 2;
	return 'survived-3' if $count >= 3;

	return 'survived';
}

# --------------------------------------------------
# _relative_link
#
# Purpose:    Compute a relative HTML href from one
#             per-file mutation page to another, so
#             Previous/Next navigation links work
#             correctly regardless of directory depth.
#
# Entry:      $from - source file path (relative to
#                     project root, e.g. lib/Foo.pm)
#             $to   - target file path (same form)
#
# Exit:       Returns a relative URL string suitable
#             for use in an href attribute.
#
# Side effects: None.
#
# Notes:      Both paths are converted to .html
#             filenames before computing the relative
#             path, mirroring how _mutant_file_report
#             names its output files.
# --------------------------------------------------
sub _relative_link {
	my ($from, $to) = @_;