view release on metacpan or  search on metacpan

lib/AI/Ollama/Client/Impl.pm  view on Meta::CPAN

    return $tx
}


sub createBlob( $self, %options ) {
    my $tx = $self->build_createBlob_request(%options);


    my $res = Future::Mojo->new();

    my $r1 = Future::Mojo->new();
    $r1->then( sub( $tx ) {
        my $resp = $tx->res;
        $self->emit(response => $resp);
        # Should we validate using OpenAPI::Modern here?!
        if( $resp->code == 201 ) {
            # Blob was successfully created
            $res->done($resp);
        } else {
            # An unknown/unhandled response, likely an error
            $res->fail( sprintf( "unknown_unhandled code %d: %s", $resp->code, $resp->body ), $resp);
        }
    })->retain;

    # Start our transaction
    $self->emit(request => $tx);
    $tx = $self->ua->start_p($tx)->then(sub($tx) {
        $r1->resolve( $tx );
        undef $r1;
    })->catch(sub($err) {
        $self->emit(response => $tx, $err);
        $r1->fail( $err => $tx );
        undef $r1;
    });

    return $res
}

=head2 C<< build_generateChatCompletion_request >>

Build an HTTP request as L<Mojo::Request> object. For the parameters see below.

=head2 C<< generateChatCompletion >>

  use Future::Utils 'repeat';
  my $response = $client->generateChatCompletion();
  my $streamed = $response->get();
  repeat {
      my ($res) = $streamed->shift;
      if( $res ) {
          my $str = $res->get;
          say $str;
      }

      Future::Mojo->done( defined $res );
  } until => sub($done) { $done->get };

Generate the next message in a chat with a provided model.

This is a streaming endpoint, so there will be a series of responses. The final response object will include statistics and additional data from the request.


=head3 Options

=over 4

=item C<< format >>

The format to return a response in. Currently the only accepted value is json.

Enable JSON mode by setting the format parameter to json. This will structure the response as valid JSON.

Note: it's important to instruct the model to use JSON in the prompt. Otherwise, the model may generate large amounts whitespace.

=item C<< keep_alive >>

How long (in minutes) to keep the model loaded in memory.

=over

=item -

If set to a positive duration (e.g. 20), the model will stay loaded for the provided duration.


=item -

If set to a negative duration (e.g. -1), the model will stay loaded indefinitely.


=item -

If set to 0, the model will be unloaded immediately once finished.


=item -

If not set, the model will stay loaded for 5 minutes by default


=back

=item C<< messages >>

The messages of the chat, this can be used to keep a chat memory

=item C<< model >>

The model name.

Model names follow a C<model:tag> format. Some examples are C<orca-mini:3b-q4_1> and C<llama2:70b>. The tag is optional and, if not provided, will default to C<latest>. The tag is used to identify a specific version.

=item C<< options >>

Additional model parameters listed in the documentation for the Modelfile such as C<temperature>.

=item C<< stream >>

If C<false> the response will be returned as a single response object, otherwise the response will be streamed as a series of objects.