ForgeOps-Tracker

 view release on metacpan or  search on metacpan

CHANGELOG.md  view on Meta::CPAN


## 0.10.0 (2026-09-25)

- New `ForgeOps::Tracker::record_change(kind => ..., title => ..., details => {...}, environment => ..., service => ..., actor => ..., url => ..., id => ..., occurred_at => ...)` records something that changed in your system (a feature flag, a config...
- Changes between deploys are now detected automatically. Once per process, `init()` queues a snapshot of the Perl version for the background thread, and ForgeOps records whatever changed since the previous boot. Module versions are left out, since P...
- New `track_env_var_names` option (default 0) adds environment variable names, never values, to that snapshot. Host-specific names (`HOSTNAME`, `PATH`, `PORT`, `LC_*`, Kubernetes service variables, and others) and the client's own `FORGE_OPS_*` sett...

## 0.9.0

- Distributed tracing across services, using the W3C Trace Context standard (`traceparent`). Every request that passes through `PSGIPerformance` or `Dancer2Performance` now gets a trace id; one that arrives with a valid `traceparent` header continues...
- Every error reported during a request now says where it happened: `trace_id`, plus, under Dancer2, `transaction_name` (the same `GET /users/:id` name performance samples and traces already use) and `endpoint` (the HTTP method plus the route pattern...
- `ForgeOps::Tracker::http_span($method, $url, sub { my ($headers) = @_; ... }, data => {})` records an outgoing HTTP call as an `http` span named after the method and host, and hands your code the `traceparent` header to add to it; its parent id is ...
- An errored request's trace is always sent, however fast it was: an error reported while the request ran, or one escaping it. A Dancer2 route that dies now has its trace finished and sent (Dancer2 never runs `after_request` for it, so before this it...
- An error escaping a request keeps its trace id, transaction name, endpoint, user and breadcrumbs when it's reported after the performance integration has already finished the request: `Integrations::PSGI` enabled outside `PSGIPerformance` (the docu...
- With `track_tracing => 0`, requests still get a trace id (attached to errors and handed out by `http_span`, since that is also what links errors across services); only span reporting stops.
- Trace and span ids were already W3C shaped (32 and 16 lowercase hex characters); they are now also guaranteed never to be all zeros, the one value the standard reserves as invalid.

This client has earlier CPAN releases (see `$ForgeOps::Tracker::VERSION`'s history in source control), but no `CHANGELOG.md` existed for it before this entry; like `sdks/java`'s, it starts here without backfilling earlier versions.

README.md  view on Meta::CPAN

```

### Where an error happened

An error reported during a request (from `Integrations::PSGI`/`Integrations::Dancer2`, or your own
`report` call from inside a route) carries up to three extra fields:

- `trace_id`: the request's W3C trace id, which ForgeOps uses to link this error to errors that
  other services reported for the same trace.
- `transaction_name`: the same name performance monitoring and traces use, `"GET /users/:id"`.
- `endpoint`: the HTTP method and the route as declared, `"GET /users/:id"`. Never the literal
  path, so an id or a token in the URL never ends up here.

`Dancer2Performance` fills in all three as soon as Dancer2 has matched the route, before the route
itself runs. `PSGIPerformance` fills in only `trace_id`, since plain PSGI has no route pattern (a
raw path is neither a transaction name nor an endpoint); if your PSGI app has its own router, name
the request yourself with `ForgeOps::Tracker::set_request_route('GET /users/:id', 'GET /users/:id')`.
They need one of the performance integrations enabled, are left out entirely outside a request, and
are never PII-scrubbed: they're structured fields, not free text. An error that escapes the request
keeps them (and the affected user and breadcrumb trail) even though `Integrations::PSGI` sits outside
`PSGIPerformance` and reports it after the request's own state has been cleared.

## Custom metrics and infrastructure monitoring

Two explicit calls (nothing is automatic, so there is no `track_metrics` option): a business event you
name yourself, and a reading from one of your own hosts.

lib/ForgeOps/Tracker.pm  view on Meta::CPAN

our @current_breadcrumbs;

# The trace span() and record_span() below record into, if one is open: a plain package variable,
# same reasoning as @current_breadcrumbs above. The PSGI/Dancer2 performance integrations start
# and finish it around every request; finish_trace() always clears it, so nothing leaks into the
# next request a prefork worker serves. Request-handling code under a threaded or event-loop-based
# PSGI server should `local`-ize it instead.
our $current_trace;

# The current request's context, if one is running: { trace_id, parent_span_id, transaction_name,
# endpoint, errored }. Set by start_trace() whether or not span tracing is on (the trace id is also
# what links an error to errors in other services), cleared by finish_trace(), and a plain package
# variable for the same reason as $current_trace above (`local`-ize it the same way under a
# threaded or event-loop PSGI server).
our $current_request;

# Request context remembered for an error that escaped a request (see snapshot_onto), for when it
# is only reported after finish_trace() has already cleared $current_request: the PSGI error
# middleware sits outside the performance one, and Dancer2 runs its on_route_exception hooks in
# whatever order the plugins were loaded. A short list rather than something attached to the
# error itself, since a Perl error is as often a plain string as an object; newest first.

lib/ForgeOps/Tracker.pm  view on Meta::CPAN

#
#   eval { risky_operation() };
#   if ($@) {
#       ForgeOps::Tracker::report($@, { order_id => $order->id });
#   }
#
# \%user defaults to whatever set_user() last set (undef if nothing did); pass one explicitly to
# override that for this one report.
#
# During a request (between start_trace and finish_trace), the event also carries the request's
# trace_id, transaction_name and endpoint, and the request is marked errored so its trace is sent
# however fast it was. An error one of the integrations saw escape a request falls back to what
# snapshot_onto remembered for it (along with the user and breadcrumb trail), for when it's
# reported only after the request's own state is gone.
sub report {
    my ($error, $context, $user) = @_;
    my $snapshot = $current_request ? undef : _snapshot_for($error);
    $current_request->{errored} = 1 if $current_request;

    my $source = $current_request || $snapshot || {};
    $user = $current_user unless defined $user;
    $user = $snapshot->{user} if !defined $user && $snapshot;
    my @breadcrumbs = @current_breadcrumbs;
    @breadcrumbs = @{ $snapshot->{breadcrumbs} } if !@breadcrumbs && $snapshot;

    _reporter()->report($error, $context, $user, \@breadcrumbs, {
        map { $_ => $source->{$_} } qw(transaction_name endpoint trace_id)
    });
    return;
}

# current_trace_id(): the current request's W3C trace id (32 lowercase hex characters), or undef
# outside a request. Handy for your own logs: it's the id ForgeOps links errors across services
# with.
sub current_trace_id {
    return $current_request ? $current_request->{trace_id} : undef;
}

# set_request_route($transaction_name, $endpoint): names the current request once its route is
# known. $transaction_name is the same name its performance sample and root span use, $endpoint
# the HTTP method plus the route pattern ("GET /users/:id"), never the literal path. The Dancer2
# performance plugin calls this as soon as Dancer2 has matched the route, so an error reported
# from inside the route already carries both; plain PSGI has no route pattern of its own, so a
# PSGI app that has one (from its own router) can call this itself. Does nothing outside a request.
sub set_request_route {
    my ($transaction_name, $endpoint) = @_;
    return unless $current_request;
    $current_request->{transaction_name} = $transaction_name;
    $current_request->{endpoint} = $endpoint;
    return;
}

# snapshot_onto($error): remembers the current request's trace id, transaction name and endpoint,
# plus the affected user and breadcrumb trail, for $error, and marks the request errored. Called by
# the PSGI/Dancer2 performance integrations when an error escapes a request, just before they
# finish its trace, so a report made afterward (see report) still has all of it. The first snapshot
# for the same error wins: the innermost one saw the request closest to where it failed. Never
# dies. Not something app code normally calls directly.
sub snapshot_onto {
    my ($error) = @_;
    eval {
        return unless $current_request && defined $error;
        $current_request->{errored} = 1;
        return if _snapshot_for($error);

        my $entry = {
            (ref $error ? (address => Scalar::Util::refaddr($error), error => $error) : (text => "$error")),
            user        => $current_user,
            breadcrumbs => [ map { { %$_ } } @current_breadcrumbs ],
            map { $_ => $current_request->{$_} } qw(transaction_name endpoint trace_id),
        };
        Scalar::Util::weaken($entry->{error}) if ref $error;
        unshift @escaped, $entry;
        splice @escaped, $MAX_ESCAPED if @escaped > $MAX_ESCAPED;
        1;
    };
    return;
}

sub _snapshot_for {

lib/ForgeOps/Tracker/Client.pm  view on Meta::CPAN

sub deliver_metrics {
    my ($self, $entries) = @_;
    return $self->_post($self->{configuration}->custom_metrics_uri, { metrics => $entries }, 'metrics delivery');
}

sub deliver_infrastructure_metrics {
    my ($self, $entries) = @_;
    return $self->_post($self->{configuration}->infrastructure_metrics_uri, { metrics => $entries }, 'infrastructure metrics delivery');
}

# Delivers one finished trace, { trace_id => ..., spans => [...] }, to the spans endpoint.
sub deliver_spans {
    my ($self, $trace) = @_;
    return $self->_post($self->{configuration}->spans_uri, $trace, 'span delivery');
}

# Delivers one record_change() call. A plan without change tracking answers 403, which is just a
# false return here like any other non-2xx (with a last_status of REJECTED).
sub deliver_change {
    my ($self, $change) = @_;
    return $self->_post($self->{configuration}->changes_uri, $change, 'change delivery');

lib/ForgeOps/Tracker/Configuration.pm  view on Meta::CPAN

# The ingestion URL with credentials stripped out (they travel as the Authorization header
# instead, never embedded in the request URI).
sub ingestion_uri {
    my ($self) = @_;
    my $parsed = $self->_parsed_dsn;
    return undef unless $parsed;
    return $parsed->{ingestion_uri};
}

# Same derivation as ingestion_uri, with the trailing "/events" swapped for
# "/performance_samples": one DSN, two more endpoints alongside deliver's own, matching the Ruby
# gem's own Configuration#performance_samples_uri.
sub performance_samples_uri {
    my ($self) = @_;
    my $uri = $self->ingestion_uri;
    return undef unless defined $uri;

    (my $swapped = $uri) =~ s{/events\z}{/performance_samples};
    return $swapped;
}

lib/ForgeOps/Tracker/EventBuilder.pm  view on Meta::CPAN

#     optionally, ->trace returning a Devel::StackTrace-compatible object (frames() ->
#     filename/line/subroutine): e.g. Throwable::Error, Moo::Exception-based classes.
#   - a plain scalar, typically $@ after `die "..."` or `Carp::confess "..."`: Perl appends
#     " at FILE line N." to any die message that doesn't already end in "\n", and Carp::confess
#     appends a full "\tPACKAGE::sub(...) called at FILE line N" chain on top of that; both are
#     parsed below into real backtrace frames rather than left as one opaque string, verified
#     directly against real confess()/die output before relying on the format, not assumed from
#     documentation alone.
#
# \%request describes the request the error happened in: transaction_name (the same
# "GET /users/:id" name the request's root span and performance sample use), endpoint (the HTTP
# method plus the route pattern), and trace_id (the request's 32-character lowercase hex W3C trace
# id, which is also what links this error to errors other services reported for the same trace).
# Each is left out of the payload when undef, and all three are undef outside a request.
sub build {
    my ($self, $error, $context, $user, $breadcrumbs, $request) = @_;
    $context ||= {};
    my $config = $self->{configuration};

    my ($exception_class, $message, $backtrace) = $self->_analyze($error);

lib/ForgeOps/Tracker/EventBuilder.pm  view on Meta::CPAN

        environment     => $config->{environment},
        release         => $config->{release},
        server_name     => $config->{server_name},
        context         => { %$context },
        tags            => {},
        sdk_name        => $SDK_NAME,
    );
    $payload{user} = { %$user } if $user && %$user;
    # Omitted entirely (never sent as an empty array) when there's nothing to report.
    $payload{breadcrumbs} = [ map { { %$_ } } @$breadcrumbs ] if $breadcrumbs && @$breadcrumbs;
    for my $key (qw(transaction_name endpoint trace_id)) {
        $payload{$key} = $request->{$key} if $request && defined $request->{$key};
    }
    $self->_attach_sql(\%payload, $error);

    return $config->{scrub_pii} ? $self->_scrub_payload(\%payload) : \%payload;
}

# See SqlStatement for what's read off the error and how it's masked. The statement itself only
# goes out when capture_sql_statement is on; the extracted names go out on their own
# (capture_sql_objects) so an issue can still name the procedure or view involved.

lib/ForgeOps/Tracker/EventBuilder.pm  view on Meta::CPAN

    return \@lines;
}

sub _truncate_line {
    my ($line) = @_;
    return $line if length($line) <= $MAX_CONTEXT_LINE_LENGTH;
    return substr($line, 0, $MAX_CONTEXT_LINE_LENGTH) . '...';
}

# exception_class/occurred_at/environment/release/server_name/sdk_name/user/transaction_name/
# endpoint/trace_id are left alone: structured fields this client or the host app sets
# deliberately, not free text an exception or its context could accidentally spill sensitive data
# into (endpoint in particular is a declared route pattern, never the literal path). user specifically is a deliberate
# exemption, not an oversight: scrub_string's own email pattern would otherwise redact the exact
# thing this field exists to carry (the %scrubbed copy below never touches it either way).
sub _scrub_payload {
    my ($self, $payload) = @_;
    my %scrubbed = %$payload;
    $scrubbed{message} = scrub_string($payload->{message});
    $scrubbed{sql_statement} = scrub_string($payload->{sql_statement}) if defined $payload->{sql_statement};
    $scrubbed{backtrace} = [
        map {
            my %frame = %$_;

lib/ForgeOps/Tracker/Integrations/Dancer2Performance.pm  view on Meta::CPAN

#
# Also starts a trace per request in before_request and finishes it in after_request (root span
# named like the transaction), so a slow request's own breakdown reaches /spans; there is no
# automatic database or HTTP span, add those by hand with ForgeOps::Tracker::span (and
# ForgeOps::Tracker::http_span for an outgoing HTTP call, which also hands it the traceparent
# header to send).
#
# The trace continues the caller's when the request arrived with a usable W3C `traceparent` header
# (see ForgeOps::Tracker::TraceParent), and its trace id exists even with track_tracing off, since
# every error reported during the request carries it. Dancer2 has already matched the route when
# before_request fires, so the request is named there (transaction name and endpoint, both
# "GET /users/:id"), before the route runs: an error reported from inside the route carries both.
#
# A route that dies never reaches after_request (see below), so an on_route_exception hook
# finishes its trace instead, marked errored so it's sent however fast it was. It remembers the
# request's context for that error first (ForgeOps::Tracker::snapshot_onto): Dancer2 runs the two
# plugins' on_route_exception hooks in load order, and Integrations::Dancer2's report has to find
# the trace id either way.
#
# The transaction name is the matched route's own spec_route (e.g. "/users/:id", the pattern as
# originally declared): Dancer2 sets this on the request during route matching, before the route

lib/ForgeOps/Tracker/Integrations/PSGIPerformance.pm  view on Meta::CPAN

# every error reported during the request carries it. An error that escapes the app is remembered
# with the request's context (ForgeOps::Tracker::snapshot_onto) before the trace is finished, so
# Integrations::PSGI, enabled outside this middleware, still reports it with the trace id.
#
# The transaction name is "<HTTP method> <path>", the raw PATH_INFO, not a matched route pattern:
# plain PSGI has no route-matching concept of its own (a bare $env hash) to read one from
# generically, the same reasoning Go's own net/http integration documents for itself. A Dancer2
# app should use Integrations::Dancer2Performance instead, which reads the matched route's own
# pattern and so avoids a distinct id exploding into its own separate transaction. For the same
# reason an error reported during a plain PSGI request carries the trace id but no transaction
# name or endpoint (a raw path is neither), unless the app names the request itself with
# ForgeOps::Tracker::set_request_route.
sub call {
    my ($self, $env) = @_;

    ForgeOps::Tracker::clear_breadcrumbs();
    ForgeOps::Tracker::start_trace($env->{HTTP_TRACEPARENT});

    my $started_at = Time::HiRes::time;
    my @response = eval { @{ $self->app->($env) } };
    my $error = $@;

t/changes.t  view on Meta::CPAN

}

sub configuration {
    my $config = ForgeOps::Tracker::Configuration->new;
    $config->{environment} = 'production';
    return $config;
}

# record_change

subtest 'record_change posts the full payload to the changes endpoint' => sub {
    ForgeOps::Tracker::_reset_for_testing();
    init_enabled();

    my $queued = ForgeOps::Tracker::record_change(
        kind        => 'feature_flag',
        title       => 'Enabled new_checkout for 10%',
        details     => { flag => 'new_checkout', rollout => 10 },
        service     => 'web',
        actor       => 'luke',
        url         => 'https://flags.example.com/new_checkout',

t/changes.t  view on Meta::CPAN

    my @names = qw(
        HOSTNAME HOST HOME PATH PWD OLDPWD SHLVL _ TERM USER LOGNAME SHELL LANG LC_ALL LC_CTYPE TMPDIR TZ
        PORT DYNO INVOCATION_ID JOURNAL_STREAM SYSTEMD_EXEC_PID MEMORY_PRESSURE_WATCH KUBERNETES_SERVICE_HOST
        REDIS_SERVICE_HOST REDIS_SERVICE_PORT REDIS_SERVICE_PORT_HTTP REDIS_PORT_6379_TCP REDIS_PORT_6379_TCP_ADDR
        FORGE_OPS_DSN FORGE_OPS_RELEASE SECRET_KEY PLACK_ENV DATABASE_URL
    );

    is_deeply(ForgeOps::Tracker::Changes::env_var_names({ map { $_ => 'value' } @names }), [qw(DATABASE_URL PLACK_ENV SECRET_KEY)]);
};

subtest 'init sends one snapshot to the change_snapshots endpoint, once per process' => sub {
    ForgeOps::Tracker::_reset_for_testing(change_snapshot => 1);
    my $before = @{ requests_to('/api/v1/change_snapshots') };

    init_enabled();
    init_enabled();

    ok(wait_until(sub { @{ requests_to('/api/v1/change_snapshots') } > $before }));
    my $request = requests_to('/api/v1/change_snapshots')->[-1];
    is($request->{headers}{AUTHORIZATION}, 'Bearer secret-key');
    my $body = decode_json($request->{body});

t/metric_buffer.t  view on Meta::CPAN

    is($accepted, ForgeOps::Tracker::MetricBuffer::MAX_ENTRIES);
    $buffer->_clear;
};

subtest 'the metric uris swap the trailing /events segment' => sub {
    my $config = new_configuration();
    is($config->custom_metrics_uri, 'https://tracker.example.com/api/v1/custom_metrics');
    is($config->infrastructure_metrics_uri, 'https://tracker.example.com/api/v1/infrastructure_metrics');
};

subtest 'capture_metric and capture_infrastructure_metric deliver to their own endpoints end to end' => sub {
    my $server = t::lib::EchoServer->start;
    ForgeOps::Tracker::_reset_for_testing();
    ForgeOps::Tracker::init(
        dsn                  => 'http://key@127.0.0.1:' . $server->{port} . '/api/v1/events',
        environment          => 'production',
        release              => 'a1b2c3d',
        enabled_environments => { production => 1 },
        server_name          => 'web-1',
    );
    ForgeOps::Tracker::capture_metric('signup');

t/trace_context.t  view on Meta::CPAN

    ok(!$config->should_propagate_trace('badexample.com'));
    ok(!$config->should_propagate_trace('example.com.evil.test'));
    ok($config->should_propagate_trace('10.0.3.4'));
    ok(!$config->should_propagate_trace('110.0.3.4'));
    ok(!$config->should_propagate_trace(undef));

    $config->{trace_propagation_targets} = [];
    ok(!$config->should_propagate_trace('example.com'), 'an empty list propagates nowhere');
};

subtest 'errors reported during a request carry its trace id, name and endpoint, unscrubbed' => sub {
    init_tracker();
    ForgeOps::Tracker::start_trace($HEADER);
    # An email-shaped route segment would be scrubbed anywhere else in the payload.
    ForgeOps::Tracker::set_request_route('GET /users/:a@b.co', 'GET /users/:a@b.co');
    ForgeOps::Tracker::report("boom\n");
    ForgeOps::Tracker::finish_trace('GET /users/:a@b.co', Time::HiRes::time, 5);

    is(reported()->[0]{trace_id}, $TRACE_ID);
    is(reported()->[0]{transaction_name}, 'GET /users/:a@b.co');
    is(reported()->[0]{endpoint}, 'GET /users/:a@b.co');
};

subtest 'errors outside a request are unchanged' => sub {
    init_tracker();
    ForgeOps::Tracker::report("before\n");
    ForgeOps::Tracker::start_trace();
    ForgeOps::Tracker::finish_trace('GET /x', Time::HiRes::time, 5);
    ForgeOps::Tracker::report("after\n");

    for my $event (@{ reported() }) {
        ok(!exists $event->{$_}, "no $_") for qw(trace_id transaction_name endpoint);
    }
    is(ForgeOps::Tracker::current_trace_id(), undef);
};

subtest 'a trace id exists even with tracing off, and no trace is sent' => sub {
    init_tracker(track_tracing => 0);
    ForgeOps::Tracker::start_trace($HEADER);
    is(ForgeOps::Tracker::current_trace_id(), $TRACE_ID);
    ForgeOps::Tracker::report("boom\n");
    ForgeOps::Tracker::finish_trace('GET /x', Time::HiRes::time, 5000);

    is(reported()->[0]{trace_id}, $TRACE_ID);
    ok(!exists reported()->[0]{endpoint}, 'an endpoint nobody set is left out');
    is_deeply(sent_traces(), []);
};

subtest 'nothing starts when the client is not enabled' => sub {
    init_tracker(environment => 'development');
    ForgeOps::Tracker::start_trace($HEADER);
    is(ForgeOps::Tracker::current_trace_id(), undef);
};

subtest "a continued trace's root span points at the caller's span; a fresh one has no parent" => sub {

t/trace_context.t  view on Meta::CPAN

    ForgeOps::Tracker::finish_trace('POST /checkout', Time::HiRes::time, 1);
    ForgeOps::Tracker::set_user();
    ForgeOps::Tracker::clear_breadcrumbs();

    ForgeOps::Tracker::report("string error\n");
    ForgeOps::Tracker::report($object);
    ForgeOps::Tracker::report("unrelated\n");

    for my $event (@{ reported() }[0, 1]) {
        is($event->{trace_id}, $TRACE_ID);
        is($event->{endpoint}, 'POST /checkout');
        is_deeply($event->{user}, { id => 7 });
        is($event->{breadcrumbs}[0]{message}, 'clicked pay');
    }
    ok(!exists reported()->[2]{trace_id}, 'an unrelated error gets nothing');
    is(scalar(@{ sent_traces() }), 1, 'snapshotting marked the fast request errored');
};

subtest 'http_span hands over a traceparent naming its own recorded span' => sub {
    init_tracker();
    ForgeOps::Tracker::start_trace($HEADER);

t/trace_context.t  view on Meta::CPAN

    };

    test_psgi $app, sub {
        my $cb = shift;
        $cb->(GET '/handled', traceparent => $HEADER);
        $cb->(GET '/boom');
    };

    my ($handled, $escaped) = @{ reported() };
    is($handled->{trace_id}, $TRACE_ID);
    ok(!exists $handled->{endpoint}, 'plain PSGI has no route pattern');
    like($escaped->{trace_id}, qr/\A[0-9a-f]{32}\z/);
    isnt($escaped->{trace_id}, $TRACE_ID, 'a request without the header starts its own trace');
    is(scalar(@{ sent_traces() }), 2, 'both fast requests errored, so both traces were sent');
    is(sent_traces()->[0]{spans}[0]{parent_span_id}, $PARENT_ID);
    is(sent_traces()->[1]{trace_id}, $escaped->{trace_id});
    is(ForgeOps::Tracker::current_trace_id(), undef, 'nothing leaks past the request');
};

# Dancer2 is an optional integration, not something `cpanm ForgeOps::Tracker` installs to run these
# tests, so its apps are compiled only when it is there and its subtests skip otherwise. A string

t/trace_context.t  view on Meta::CPAN


SKIP: {
    skip 'Dancer2 is not installed (only the optional Dancer2 integration needs it)', 3 unless $have_dancer2;

    subtest 'Dancer2: an error reported inside a route carries the continued trace and the route pattern' => sub {
        init_tracker();
        test_psgi(ErrorsFirstApp->to_app, sub { $_[0]->(GET '/orders/42', traceparent => $HEADER) });

        is(reported()->[0]{trace_id}, $TRACE_ID);
        is(reported()->[0]{transaction_name}, 'GET /orders/:id');
        is(reported()->[0]{endpoint}, 'GET /orders/:id');
        is(sent_traces()->[0]{spans}[0]{parent_span_id}, $PARENT_ID, 'fast, but errored: sent');
    };

    for my $app_class (qw(ErrorsFirstApp PerformanceFirstApp)) {
        subtest "Dancer2 ($app_class): a route that dies is reported with its context and its trace is sent" => sub {
            init_tracker();
            my $res;
            test_psgi($app_class->to_app, sub { $res = $_[0]->(POST '/checkout/9', traceparent => $HEADER) });

            is($res->code, 500);
            is(scalar(@{ reported() }), 1);
            is(reported()->[0]{trace_id}, $TRACE_ID);
            is(reported()->[0]{endpoint}, 'POST /checkout/:cart');
            is(scalar(@{ sent_traces() }), 1);
            is(sent_traces()->[0]{spans}[0]{name}, 'POST /checkout/:cart');
            is(ForgeOps::Tracker::current_trace_id(), undef, 'nothing leaks past the request');
        };
    }
}

done_testing;



( run in 1.055 second using v1.01-cache-2.11-cpan-16aabcbfbcb )