App-Greple-xlate
view release on metacpan or search on metacpan
lib/App/Greple/xlate.pm view on Meta::CPAN
=item B<--xlate-color>
=item B<--xlate-fold>
=item B<--xlate-fold-width>=I<n> (Default: 70)
Invoke the translation process for each matched area.
Without this option, B<greple> behaves as a normal search command. So
you can check which part of the file will be subject of the
translation before invoking actual work.
Command result goes to standard out, so redirect to file if necessary,
or consider to use L<App::Greple::update> module.
Option B<--xlate> calls B<--xlate-color> option with B<--color=never>
option.
With B<--xlate-fold> option, converted text is folded by the specified
width. Default width is 70 and can be set by B<--xlate-fold-width>
option. Four columns are reserved for run-in operation, so each line
could hold 74 characters at most.
=item B<--xlate-engine>=I<engine>
Specifies the translation engine to be used.
At this time, the following engines are available
=over 2
=item * B<gpt5>: gpt-5.6-terra (via the C<llm> command)
=item * B<deepl>: DeepL API (via the C<deepl> command)
=item * B<gpt3>: gpt-3.5-turbo (legacy, via the C<gpty> command)
=item * B<gpt4o>: gpt-4o-mini (legacy, via the C<gpty> command)
=back
Engine modules are searched in backend namespaces first (C<llm>, then
C<gpty>), then directly under C<App::Greple::xlate>. So C<gpt5> loads
C<App::Greple::xlate::llm::gpt5> which calls the C<llm> command, while
C<gpt4o> falls back to C<App::Greple::xlate::gpty::gpt4o>. Use
C<--xlate-setopt backend=gpty> to force a specific backend.
=item B<--xlate-labor>
=item B<--xlabor>
Instead of calling translation engine, you are expected to work for.
After preparing text to be translated, they are copied to the
clipboard. You are expected to paste them to the form, copy the
result to the clipboard, and hit return.
=item B<--xlate-to> (Default: C<EN-US>)
Specify the target language. LLM engines accept any language name
or code the model understands; it is interpolated into the
translation prompt. You can get available languages by C<deepl
languages> command when using B<DeepL> engine.
=item B<--xlate-from> (Default: C<ORIGINAL>)
Label used for the original text in C<conflict>, C<colon> and
C<ifdef> output formats. With the B<DeepL> engine a non-default
value is also passed as the source language.
=item B<--xlate-format>=I<format> (Default: C<conflict>)
Specify the output format for original and translated text.
The following formats other than C<xtxt> assume that the part to be
translated is a collection of lines. In fact, it is possible to
translate only a portion of a line, but specifying a format other than
C<xtxt> will not produce meaningful results.
=over 4
=item B<conflict>, B<cm>
Original and converted text are printed in L<git(1)> conflict marker
format.
<<<<<<< ORIGINAL
original text
=======
translated Japanese text
>>>>>>> JA
You can recover the original file by next L<sed(1)> command.
sed -e '/^<<<<<<< /d' -e '/^=======$/,/^>>>>>>> /d'
=item B<colon>, I<:::::::>
The original and translated text are output in a markdown's custom
container style.
::::::: ORIGINAL
original text
:::::::
::::::: JA
translated Japanese text
:::::::
Above text will be translated to the following in HTML.
<div class="ORIGINAL">
original text
</div>
<div class="JA">
translated Japanese text
</div>
Number of colon is 7 by default. If you specify colon sequence like
C<:::::>, it is used instead of 7 colons.
=item B<ifdef>
Original and converted text are printed in L<cpp(1)> C<#ifdef>
format.
#ifdef ORIGINAL
original text
#endif
#ifdef JA
translated Japanese text
#endif
You can retrieve only Japanese text by the B<unifdef> command:
unifdef -UORIGINAL -DJA foo.ja.pm
=item B<space>
=item B<space+>
Original and converted text are printed separated by single blank
line. For C<space+>, it also outputs a newline after the converted
text.
=item B<xtxt>
If the format is C<xtxt> (translated text) or unkown, only translated
text is printed.
=back
=item B<--xlate-maxlen>=I<chars> (Default: 0)
Specify the maximum length of text to be sent to the API at once.
The default value 0 means the engine's own limit: for the free DeepL
account service that is 128K for the API (B<--xlate>) and 5000 for
the clipboard interface (B<--xlate-labor>). You may be able to
change these value if you are using Pro service.
=item B<--xlate-maxline>=I<n> (Default: 0)
Specify the maximum lines of text to be sent to the API at once.
Set this value to 1 if you want to translate one line at a time. This
option takes precedence over the C<--xlate-maxlen> option.
=item B<--xlate-prompt>=I<text>
Specify a custom prompt to be sent to the translation engine. This
option is available for the LLM engines (C<gpt3>, C<gpt4o>, C<gpt5>)
but not for DeepL. You can customize the translation behavior by
providing specific instructions to the AI model. If the prompt
contains C<%s>, it will be replaced with the target language name.
For the llm-backed C<gpt5> engine, the document is supplied separately
as a JSON request whose C<input> member is the array to translate and
whose optional C<context> member contains reference data. A fixed
instruction that treats those members as document data, not commands,
is appended even when a custom prompt is used.
=item B<--xlate-context>=I<text>
Specify additional context information to be sent to the translation
engine. This option can be used multiple times to provide multiple
context strings. The context information helps the translation engine
understand the background and produce more accurate translations.
=item B<--xlate-context-window>=I<n>
(Context-aware engines only, e.g. C<gpt5> on the llm backend)
Number of surrounding translated blocks passed as reference context
when re-translating changed blocks (default 2). The context also
includes the raw source text around the changed region (headings,
list structure, captions) and, when available, the previous version
of the changed text recovered from the cache, so that unchanged
wording is preserved. Set to 0 to disable context-aware translation
entirely.
Note that each changed region is translated in its own API call and
the context can add up to about 8000 characters to the JSON user
request, so context-aware translation trades some extra cost for
consistency. Document-derived context is kept out of the system
prompt.
=item B<--xlate-cache-seed>=I<file>
Initialize a new document's cache from another document's cache
file. Useful for periodic reports: seed the new issue's cache with
the previous issue's, so unchanged paragraphs are not re-translated
and edited paragraphs keep the previous issue's wording. The seed
is used only when the target cache is empty; otherwise it is
ignored with a warning. With the default C<--xlate-cache=auto>, specifying a seed also
implies creating the new document's cache file.
=item B<--xlate-anonymize>=I<file>
Anonymize sensitive strings before they are sent to the translation
API, and restore them in the output. The dictionary file gives one
entry per item: in JSON (canonical, machine-generatable)
[ { "category": "person", "text": "山田太郎" },
{ "category": "company", "regex": "アクメ(æ ªå¼ä¼šç¤¾)?" } ]
or in a simple line format (C<category pattern>, C</.../> for regex).
Each item is replaced by a category tag such as C<< <person id="1" /> >>;
the same string always gets the same tag, so the model can keep track
of who is who. Unknown JSON fields are ignored, so generators (e.g. a
local LLM extracting entities) may add their own annotations.
Category C<lit> is reserved. Local cache files still store restored
plain text: the concealment target is API transmission only.
A dictionary can be generated by an external tool -- for example a
local model extracting sensitive entities:
llm -m <local-model> \
-s 'Extract sensitive entities as a JSON array of objects
with "category" and "text" fields.' \
< report.md > report.anon.json
greple -Mxlate --xlate-anonymize=report.anon.json ...
A UTF-8 BOM in the file is tolerated. Values in the front matter
line format may carry a trailing comment only on their own line, not
after the value.
=item B<--xlate-anonymize-mark>[=I<regex>]
Collect anonymization entries from inline marks in the document
itself. Mark the first occurrence like C<{{ person("山田太郎") }}>
and every occurrence of the string document-wide is anonymized. The
mark itself stays in the source and in the translation, so a document
can also be processed by a Jinja2-style macro processor (define the
C<person> macro to print or redact the name). A custom I<regex> must
contain C<< (?<category>...) >> and C<< (?<text>...) >> named captures.
Note that with an optional-value option like this, a following
file argument would be taken as the value: write
C<--xlate-anonymize-mark=> (with a trailing C<=>) when using the
default notation.
Alternative notations can be configured, for example
C<< --xlate-anonymize-mark='@@(?<category>[a-z][a-z0-9_]*):(?<text>[^\n]+?)@@' >>
for C<@@person:NAME@@>-style marks, or an HTML-comment form that stays
invisible in rendered Markdown. Mark rules are collected per
lib/App/Greple/xlate.pm view on Meta::CPAN
=item * L<https://qiita.com/kaz-utashiro/items/1c1a51a4591922e18250>
Greple module to translate and replace only the necessary parts with DeepL API (in Japanese)
=item * L<https://qiita.com/kaz-utashiro/items/a5e19736416ca183ecf6>
Generating documents in 15 languages with DeepL API module (in Japanese)
=item * L<https://qiita.com/kaz-utashiro/items/1b9e155d6ae0620ab4dd>
Automatic translation Docker environment with DeepL API (in Japanese)
=back
=head1 AUTHOR
Kazumasa Utashiro
=head1 LICENSE
Copyright © 2023-2026 Kazumasa Utashiro.
This library is free software; you can redistribute it and/or modify
it under the same terms as Perl itself.
=cut
use v5.26;
use warnings;
use utf8;
use Data::Dumper;
use Text::ANSI::Fold ':constants';
use Command::Run;
use Hash::Util qw(lock_keys);
use Unicode::EastAsianWidth;
use List::Util qw(max);
use Exporter 'import';
our @EXPORT_OK = qw($VERSION &opt %opt);
our @EXPORT_TAGS = ( all => [ qw($VERSION) ] );
our %opt = (
debug => \(our $debug = 0),
engine => \(our $xlate_engine),
progress => \(our $show_progress = 1),
format => \(our $output_format = 'conflict'),
collapse => \(our $collapse_spaces = 1),
from => \(our $lang_from = 'ORIGINAL'),
to => \(our $lang_to = 'EN-US'),
fold => \(our $fold_line = 0),
width => \(our $fold_width = 70),
auth_key => \(our $auth_key),
method => \(our $cache_method //= $ENV{GREPLE_XLATE_CACHE} || 'auto'),
update => \(our $force_update = 0),
dryrun => \(our $dryrun = 0),
maxlen => \(our $max_length = 0),
maxline => \(our $max_line = 0),
prompt => \(our $prompt),
mask => \(our $mask),
maskfile => \(our $maskfile),
glossary => \(our $glossary),
backend => \(our $engine_backend = ''),
cache_seed => \(our $cache_seed),
context_window => \(our $context_window = 2),
anonymize => \(our $anonymize_file),
anonymize_mark => \(our $anonymize_mark),
template => \(our $template_option),
frontmatter => \(our $use_frontmatter = 0),
review => \(our $show_review = 0),
contexts => (\our @contexts),
);
lock_keys %opt;
sub opt :lvalue { ${$opt{+shift}} }
our $current_file;
my $current_text; # whole document, set in begin()
my $frontmatter_len = 0; # body starts after this offset
our $call_context; # per-call context for context-aware engines
our $engine_supports_context; # engine declares $XLATE_CONTEXT
my $colon_count = 7;
our %formatter = (
xtxt => undef,
none => undef,
conflict => sub {
join '',
"<<<<<<< $lang_from\n",
$_[0],
"=======\n",
$_[1],
">>>>>>> $lang_to\n";
},
cm => 'conflict',
colon => sub {
my $colon = ':' x $colon_count;
join '',
"$colon $lang_from\n",
$_[0],
"$colon\n",
"$colon $lang_to\n",
$_[1],
"$colon\n";
},
ifdef => sub {
join '',
"#ifdef $lang_from\n",
$_[0],
"#endif\n",
"#ifdef $lang_to\n",
$_[1],
"#endif\n";
},
space => sub { join("\n", @_) },
'space+' => sub { join("\n", @_) . "\n" },
discard => sub { '' },
);
# aliases
lib/App/Greple/xlate.pm view on Meta::CPAN
die "Select translation engine.\n";
}
if ($output_format =~ /^(:+)$/) {
$colon_count = length($1);
$output_format = 'colon';
}
if (my $file = cache_file) {
my @opt;
if ($cache_method =~ /create|clear/i) {
push @opt, clear => 1;
}
if ($cache_method =~ /accumulate/i) {
push @opt, accumulate => 1;
}
if ($force_update) {
push @opt, force_update => 1;
}
if (defined $cache_seed) {
push @opt, seed => $cache_seed;
}
if ($dryrun) {
push @opt, readonly => 1;
}
require App::Greple::xlate::Cache;
tie %cache, 'App::Greple::xlate::Cache', $file, @opt;
die "skip $current_file" if $cache_method eq 'create';
}
}
sub end {
# if (my $obj = tied %cache) {
# $obj->update;
# }
}
sub set {
while (my($key, $val) = splice @_, 0, 2) {
next if $key eq &::FILELABEL;
die "$key: Invalid option.\n" if not exists $opt{$key};
opt($key) = $val;
}
}
1;
__DATA__
builtin xlate-debug! $debug
builtin xlate-progress! $show_progress
builtin xlate-format=s $output_format
builtin xlate-fold-line! $fold_line
builtin xlate-fold-width=i $fold_width
builtin xlate-from=s $lang_from
builtin xlate-to=s $lang_to
builtin xlate-cache:s $cache_method
builtin xlate-update! $force_update
builtin xlate-engine=s $xlate_engine
builtin xlate-dryrun $dryrun
builtin xlate-maxlen=i $max_length
builtin xlate-maxline=i $max_line
builtin xlate-prompt=s $prompt
builtin xlate-glossary=s $glossary
builtin xlate-context=s @contexts
builtin xlate-context-window=i $context_window
builtin xlate-anonymize=s $anonymize_file
builtin xlate-anonymize-mark:s $anonymize_mark
builtin xlate-template:s $template_option
builtin xlate-review! $show_review
builtin xlate-cache-seed=s $cache_seed
builtin deepl-auth-key=s $App::Greple::xlate::deepl::auth_key
builtin deepl-method=s $App::Greple::xlate::deepl::method
option default --need=1 --no-regioncolor --cm=/544E,/454E,/533E,/353E
option --xlate-setopt --prologue &__PACKAGE__::set($<shift>)
option --xlate-color \
--postgrep &__PACKAGE__::postgrep \
--callback &__PACKAGE__::callback \
--begin &__PACKAGE__::begin \
--end &__PACKAGE__::end
option --xlate --xlate-color --color=never
option --xlate-fold --xlate --xlate-fold-line
option --xlate-labor --xlate --deepl-method=clipboard
option --xlabor --xlate-labor
option --xlate-mask \
--begin &__PACKAGE__::begin \
--callback &__PACKAGE__::mask_string
option --cache-clear --xlate-cache=clear
option --xlate-frontmatter \
--xlate-setopt frontmatter=1 \
--exclude '\A---\n(?s:.*?)^---\n'
option --match-all --re '\A(?s).+\z'
option --match-entire --match-all
option --match-paragraph --re '^(.+\n)+'
option --match-podtext -Mperl --pod --re '^(\w.*\n)(\S.*\n)*'
option --ifdef-color --re '^#ifdef(?s:.*?)^#endif.*\n'
option --xlate-stripe --xlate-stripe-auto
option --xlate-stripe-light -Mstripe
option --xlate-stripe-dark -Mstripe::config=darkmode
option --xlate-stripe-auto \
-Mtermcolor::bg(light=-Mstripe,dark=-Mstripe::config=darkmode)
option --lineify-cm \
-Mxlate::Filter --of &lineify_cm
option --lineify-colon \
-Mxlate::Filter --of &lineify_colon
# LocalWords: deepl ifdef unifdef Greple greple perl DeepL ChatGPT
# LocalWords: gpt html img src xlabor
( run in 1.160 second using v1.01-cache-2.11-cpan-b16cb0d3907 )