view release on metacpan or  search on metacpan

lib/AI/Ollama/RequestOptions.pm  view on Meta::CPAN

has 'num_batch' => (
    is       => 'ro',
    isa      => Int,
);

=head2 C<< num_ctx >>

Sets the size of the context window used to generate the next token.

=cut

has 'num_ctx' => (
    is       => 'ro',
    isa      => Int,
);

=head2 C<< num_gpu >>

The number of layers to send to the GPU(s). On macOS it defaults to 1 to enable metal support, 0 to disable.

=cut

has 'num_gpu' => (
    is       => 'ro',
    isa      => Int,
);

=head2 C<< num_gqa >>

The number of GQA groups in the transformer layer. Required for some models, for example it is 8 for `llama2:70b`.

=cut

has 'num_gqa' => (
    is       => 'ro',
    isa      => Int,
);

=head2 C<< num_keep >>

Number of tokens to keep from the prompt.

=cut

has 'num_keep' => (
    is       => 'ro',
    isa      => Int,
);

=head2 C<< num_predict >>

Maximum number of tokens to predict when generating text. (Default: 128, -1 = infinite generation, -2 = fill context)

=cut

has 'num_predict' => (
    is       => 'ro',
    isa      => Int,
);

=head2 C<< num_thread >>

Sets the number of threads to use during computation. By default, Ollama will detect this for optimal performance. It is recommended to set this value to the number of physical CPU cores your system has (as opposed to the logical number of cores).

=cut

has 'num_thread' => (
    is       => 'ro',
    isa      => Int,
);

=head2 C<< numa >>

Enable NUMA support. (Default: false)

=cut

has 'numa' => (
    is       => 'ro',
);

=head2 C<< penalize_newline >>

Penalize newlines in the output. (Default: false)

=cut

has 'penalize_newline' => (
    is       => 'ro',
);

=head2 C<< presence_penalty >>

Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.

=cut

has 'presence_penalty' => (
    is       => 'ro',
    isa      => Num,
);

=head2 C<< repeat_last_n >>

Sets how far back for the model to look back to prevent repetition. (Default: 64, 0 = disabled, -1 = num_ctx)

=cut

has 'repeat_last_n' => (
    is       => 'ro',
    isa      => Int,
);

=head2 C<< repeat_penalty >>

Sets how strongly to penalize repetitions. A higher value (e.g., 1.5) will penalize repetitions more strongly, while a lower value (e.g., 0.9) will be more lenient. (Default: 1.1)

=cut

has 'repeat_penalty' => (
    is       => 'ro',
    isa      => Num,
);

=head2 C<< rope_frequency_base >>

The base of the rope frequency scale. (Default: 1.0)