view release on metacpan or  search on metacpan

README.md  view on Meta::CPAN

              NONINTERACTIVE_TESTING: 1

## Fuzz Testing your CPAN Module

Running fuzz tests when you run `make test` in your CPAN module.

Create a directory <t/conf> which contains the schemas.

Then create this file as <t/fuzz.t>:

    #!/usr/bin/env perl

    use strict;
    use warnings;

    use FindBin qw($Bin);
    use IPC::Run3;
    use IPC::System::Simple qw(system);
    use Test::Needs 'App::Test::Generator';
    use Test::Most;

    my $dirname = "$Bin/conf";

    if((-d $dirname) && opendir(my $dh, $dirname)) {
          while (my $filename = readdir($dh)) {
                  # Skip '.' and '..' entries and vi temporary files
                  next if ($filename eq '.' || $filename eq '..') || ($filename =~ /\.swp$/);

                  my $filepath = "$dirname/$filename";

                  if(-f $filepath) {      # Check if it's a regular file
                          my ($stdout, $stderr);
                          run3 ['fuzz-harness-generator', '-r', $filepath], undef, \$stdout, \$stderr;

                          ok($? == 0, 'Generated test script exits successfully');

                          if($? == 0) {
                                  ok($stdout =~ /^Result: PASS/ms);
                                  if($stdout =~ /Files=1, Tests=(\d+)/ms) {
                                          diag("$1 tests run");
                                  }
                          } else {
                                  diag("$filepath: STDOUT:\n$stdout");
                                  diag($stderr) if(length($stderr));
                                  diag("$filepath Failed");
                                  last;
                          }
                          diag($stderr) if(length($stderr));
                  }
          }
          closedir($dh);
    }

    done_testing();

## Property-Based Testing with Transforms

The generator can create property-based tests using [Test::LectroTest](https://metacpan.org/pod/Test%3A%3ALectroTest) when the
`properties` configuration option is enabled.
This provides more comprehensive
testing by automatically generating thousands of test cases and verifying that
mathematical properties hold across all inputs.

### Basic Property-Based Transform Example

Here's a complete example testing the `abs` builtin function:

**t/conf/abs.yml**:

    ---
    module: builtin
    function: abs

    config:
      test_undef: no
      test_empty: no
      test_nuls: no
      properties:
        enable: true
        trials: 1000

    input:
      number:
        type: number
        position: 0

    output:
      type: number
      min: 0

    transforms:
      positive:
        input:
          number:
            type: number
            min: 0
        output:
          type: number
          min: 0

      negative:
        input:
          number:
            type: number
            max: 0
        output:
          type: number
          min: 0

This configuration:

- Enables property-based testing with 1000 trials per property
- Defines two transforms: one for positive numbers, one for negative
- Automatically generates properties that verify `abs()` always returns non-negative numbers

Generate the test:

    fuzz-harness-generator t/conf/abs.yml > t/abs_property.t

The generated test will include:

- Traditional edge-case tests for boundary conditions
- Random fuzzing with 30 iterations (or as configured)
- Property-based tests that verify the transforms with 1000 trials each

### What Properties Are Tested?

The generator automatically detects and tests these properties based on your transform specifications:

- **Range constraints** - If output has `min` or `max`, verifies results stay within bounds
- **Type preservation** - Ensures numeric inputs produce numeric outputs
- **Definedness** - Verifies the function doesn't return `undef` unexpectedly
- **Specific values** - If output specifies a `value`, checks exact equality

For the `abs` example above, the generated properties verify:

    # For the "positive" transform:
    - Given a positive number, abs() returns >= 0
    - The result is a valid number
    - The result is defined

    # For the "negative" transform:
    - Given a negative number, abs() returns >= 0
    - The result is a valid number
    - The result is defined

### Advanced Example: String Normalization

Here's a more complex example testing a string normalization function:

**t/conf/normalize.yml**:

    ---
    module: Text::Processor
    function: normalize_whitespace

    config:
      properties:
        enable: true
        trials: 500

    input:
      text:
        type: string
        min: 0
        max: 1000
        position: 0

    output:
      type: string
      min: 0
      max: 1000

    transforms:
      empty_preserved:
        input:
          text:
            type: string
            value: ""
        output:
          type: string
          value: ""

      single_space:
        input:
          text:
            type: string
            min: 1
            matches: '^\S+(\s+\S+)*$'
        output:
          type: string
          matches: '^\S+( \S+)*$'

      length_bounded:
        input: