view release on metacpan or  search on metacpan

lib/App/Greple/xlate.pm  view on Meta::CPAN

ignored with a warning.  With the default C<--xlate-cache=auto>, specifying a seed also
implies creating the new document's cache file.

=item B<--xlate-anonymize>=I<file>

Anonymize sensitive strings before they are sent to the translation
API, and restore them in the output.  The dictionary file gives one
entry per item: in JSON (canonical, machine-generatable)

    [ { "category": "person",  "text": "山田太郎" },
      { "category": "company", "regex": "アクメ(株式会社)?" } ]

or in a simple line format (C<category pattern>, C</.../> for regex).
Each item is replaced by a category tag such as C<< <person id=1 /> >>;
the same string always gets the same tag, so the model can keep track
of who is who.  Unknown JSON fields are ignored, so generators (e.g. a
local LLM extracting entities) may add their own annotations.
Category C<lit> is reserved.  Local cache files still store restored
plain text: the concealment target is API transmission only.

A dictionary can be generated by an external tool -- for example a
local model extracting sensitive entities:

    llm -m <local-model> \
        -s 'Extract sensitive entities as a JSON array of objects
            with "category" and "text" fields.' \
        < report.md > report.anon.json
    greple -Mxlate --xlate-anonymize=report.anon.json ...

A UTF-8 BOM in the file is tolerated.  Values in the front matter
line format may carry a trailing comment only on their own line, not
after the value.

=item B<--xlate-anonymize-mark>[=I<regex>]

Collect anonymization entries from inline marks in the document
itself.  Mark the first occurrence like C<{{ person("山田太郎") }}>
and every occurrence of the string document-wide is anonymized.  The
mark itself stays in the source and in the translation, so a document
can also be processed by a Jinja2-style macro processor (define the
C<person> macro to print or redact the name).  A custom I<regex> must
contain C<< (?<category>...) >> and C<< (?<text>...) >> named captures.

Note that with an optional-value option like this, a following
file argument would be taken as the value: write
C<--xlate-anonymize-mark=> (with a trailing C<=>) when using the
default notation.

Alternative notations can be configured, for example
C<< --xlate-anonymize-mark='@@(?<category>[a-z][a-z0-9_]*):(?<text>[^\n]+?)@@' >>
for C<@@person:NAME@@>-style marks, or an HTML-comment form that stays
invisible in rendered Markdown.  Mark rules are collected per
document: a string marked in one input file is not concealed in
another file of the same run (unlike front matter values, which
accumulate across files).

=item B<--xlate-template>[=I<regex>]

Treat template expressions (default: Jinja2 C<{{ ... }}>,
C<{% ... %}>, C<{# ... #}>) as opaque placeholders: instruct the
model to copy them unchanged and verify, per block, that the response
contains exactly the same expressions, each the same number of times.
Their order may change, since translation legitimately reorders them
to follow the target language word order.  A broken expression
aborts the run; the cache is checkpointed and frozen, so nothing paid
for is lost.

Note that with an optional-value option like this, a following
file argument would be taken as the value: write
C<--xlate-template=> (with a trailing C<=>) when using the
default notation.

=item B<--xlate-frontmatter>

Treat a leading C<---> ... C<---> block as YAML front matter: exclude
it from translation and from the phase-2 context slices, and add its
flat C<key: value> values to the anonymization rules (category
C<var>) as a safety net.  With multiple input files the collected
values accumulate (erring on the side of concealment).

Always leave a blank line after the closing C<--->.  With a
paragraph-style match pattern, front matter that runs directly into
the body text forms one straddling block that the exclusion cannot
suppress (a warning is printed in that case); the values are still
anonymized, but the front matter itself would be sent for
translation.

=item B<--xlate-glossary>=I<glossary>

Specify a glossary ID to be used for translation.  This option is only
available when using the DeepL engine.  The glossary ID should be obtained
from your DeepL account and ensures consistent translation of specific terms.

=item B<--xlate-dryrun>

Do not call the translation API; instead show, through the progress
display, each payload exactly as it would be transmitted (after
anonymization and masking).  Useful for checking what leaves the
machine and for estimating the cost of a run.

=item B<-->[B<no->]B<xlate-progress> (Default: True)

See the translation result in real time in the STDERR output.  The
C<From> payload is shown as transmitted, after anonymization and
masking.

=item B<--xlate-stripe>

Use L<App::Greple::stripe> module to show the matched part by zebra
striping fashion.  This is useful when the matched parts are connected
back-to-back.

The color palette is switched according to the background color of the
terminal.  If you want to specify explicitly, you can use
B<--xlate-stripe-light> or B<--xlate-stripe-dark>.

=item B<--xlate-mask>

Perform masking function and display the converted text as is without
restoration.