AI-Ollama-Client
view release on metacpan or search on metacpan
0.05 2025-03-01
* Add File::ShareDir::Install to CONFIGURE_REQUIRES
This fixes GH #3
0.04 2025-02-28
* Regenerate with better OpenAPI converter
* Distribute our OpenAPI spec
* Request builders are now public
* HTTP events are emitted for easier tracing
- request
- response
0.03 2024-04-08
* Fix prerequisites
0.02 2024-04-07
* Regenerated documentation
* Fixed code for uploading a model
0.01 2024-04-05
* Released on an unsuspecting world
make of the Package, you should contact the Copyright Holder and seek
a different licensing arrangement.
Definitions
"Copyright Holder" means the individual(s) or organization(s)
named in the copyright notice for the entire Package.
"Contributor" means any party that has contributed code or other
material to the Package, in accordance with the Copyright Holder's
procedures.
"You" and "your" means any person who would like to copy,
distribute, or modify the Package.
"Package" means the collection of files distributed by the
Copyright Holder, and derivatives of that collection and/or of
those files. A given Package may consist of either the Standard
Version, or a Modified Version.
"Distribute" means providing a copy of the Package or making it
Holder.
"Original License" means this Artistic License as Distributed with
the Standard Version of the Package, in its current version or as
it may be modified by The Perl Foundation in the future.
"Source" form means the source code, documentation source, and
configuration files for the Package.
"Compiled" form means the compiled bytecode, object code, binary,
or any other form resulting from mechanical transformation or
translation of the Source form.
Permission for Use and Modification Without Distribution
(1) You are permitted to use the Standard Version and create and use
Modified Versions for any purpose without restriction, provided that
you do not Distribute the Modified Version.
Permissions for Redistribution of the Standard Version
(2) You may Distribute verbatim copies of the Source form of the
Standard Version of this Package in any medium without restriction,
either gratis or for a Distributor Fee, provided that you duplicate
all of the original copyright notices and associated disclaimers. At
your discretion, such verbatim copies may or may not include a
Compiled form of the Package.
(3) You may apply any bug fixes, portability changes, and other
modifications made available from the Copyright Holder. The resulting
Package will still be considered the Standard Version, and as such
will be subject to the Original License.
Distribution of Modified Versions of the Package as Source
(4) You may Distribute your Modified Version as Source (either gratis
or for a Distributor Fee, and with or without a Compiled form of the
Modified Version) provided that you clearly document how it differs
from the Standard Version, including, but not limited to, documenting
any non-standard features, executables, or modules, and provided that
you do at least ONE of the following:
(a) make the Modified Version available to the Copyright Holder
of the Standard Version, under the Original License, so that the
Copyright Holder may include your modifications in the Standard
Version.
(b) ensure that installation of your Modified Version does not
prevent the user installing or running the Standard Version. In
addition, the Modified Version must bear a name that is different
(c) allow anyone who receives a copy of the Modified Version to
make the Source form of the Modified Version available to others
under
(i) the Original License or
(ii) a license that permits the licensee to freely copy,
modify and redistribute the Modified Version using the same
licensing terms that apply to the copy that the licensee
received, and requires that the Source form of the Modified
Version, and of any works derived from it, be made freely
available in that license fees are prohibited but Distributor
Fees are allowed.
Distribution of Compiled Forms of the Standard Version
or Modified Versions without the Source
(5) You may Distribute Compiled forms of the Standard Version without
the Source, provided that you include complete instructions on how to
get the Source of the Standard Version. Such instructions must be
valid at the time of your distribution. If these instructions, at any
time while you are carrying out such distribution, become invalid, you
must provide new instructions on demand or cease further distribution.
If you provide valid instructions or cease distribution within thirty
days after you become aware that the instructions are invalid, then
you do not forfeit any of your rights under this license.
(6) You may Distribute a Modified Version in Compiled form without
the Source, provided that you comply with Section 4 with respect to
the Source of the Modified Version.
Aggregating or Linking the Package
(7) You may aggregate the Package (either the Standard Version or
Modified Version) with other packages and Distribute the resulting
aggregation provided that you do not charge a licensing fee for the
Package. Distributor Fees are permitted, and licensing fees for other
components in the aggregation are permitted. The terms of this license
apply to the use and Distribution of the Standard or Modified Versions
as included in the aggregation.
(8) You are permitted to link Modified and Standard Versions with
other works, to embed the Package in a larger work of your own, or to
build stand-alone binary or bytecode versions of applications that
include the Package, and Distribute the result without restriction,
provided the result does not expose a direct interface to the Package.
Items That are Not Considered Part of a Modified Version
(9) Works (including, but not limited to, modules and scripts) that
merely extend or make use of the Package, do not, by themselves, cause
the Package to be a Modified Version. In addition, such works are not
considered parts of the Package itself, and are not subject to the
terms of this license.
(11) If your Modified Version has been derived from a Modified
Version made by someone other than you, you are nevertheless required
to ensure that your Modified Version complies with the requirements of
this license.
(12) This license does not grant you the right to use any trademark,
service mark, tradename, or logo of the Copyright Holder.
(13) This license includes the non-exclusive, worldwide,
free-of-charge patent license to make, have made, use, offer to sell,
sell, import and otherwise transfer the Package with respect to any
patent claims licensable by the Copyright Holder that are necessarily
infringed by the Package. If you institute patent litigation
(including a cross-claim or counterclaim) against any party alleging
that the Package constitutes direct or contributory patent
infringement, then this Artistic License to you shall terminate on the
date that such litigation is filed.
(14) Disclaimer of Warranty:
THE PACKAGE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS "AS
IS' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES. THE IMPLIED
},
"name" : "AI-Ollama-Client",
"no_index" : {
"directory" : [
"t",
"inc"
]
},
"prereqs" : {
"build" : {
"requires" : {
"strict" : "0"
}
},
"configure" : {
"requires" : {
"ExtUtils::MakeMaker" : "0",
"File::ShareDir::Install" : "0"
}
},
"runtime" : {
"requires" : {
"Carp" : "0",
"File::ShareDir" : "0",
"Future" : "0",
"Future::Mojo" : "0",
"Future::Queue" : "0",
"Future::Utils" : "0",
"Mojo::JSON" : "0",
"Mojo::URL" : "0",
"Mojo::UserAgent" : "0",
"Mojolicious" : "9",
"PerlX::Maybe" : "0",
"Role::EventEmitter" : "0",
"URI::Template" : "0",
"YAML::PP" : "0",
"experimental" : "0.031",
"perl" : "5.020",
"stable" : "0.031"
}
},
"test" : {
"requires" : {
"Test2::V0" : "0"
}
}
},
"release_status" : "stable",
"resources" : {
"bugtracker" : {
"web" : "https://github.com/Corion/AI-Ollama-Client/issues"
},
"license" : [
"https://dev.perl.org/licenses/"
],
"repository" : {
"type" : "git",
"url" : "git://github.com/Corion/AI-Ollama-Client.git",
"web" : "https://github.com/Corion/AI-Ollama-Client"
---
abstract: 'Client for AI::Ollama'
author:
- 'Max Maischein <corion@cpan.org>'
build_requires:
Test2::V0: '0'
strict: '0'
configure_requires:
ExtUtils::MakeMaker: '0'
File::ShareDir::Install: '0'
dynamic_config: 0
generated_by: 'ExtUtils::MakeMaker version 7.64, CPAN::Meta::Converter version 2.150010'
license: artistic_2
meta-spec:
url: http://module-build.sourceforge.net/META-spec-v1.4.html
version: '1.4'
name: AI-Ollama-Client
no_index:
directory:
- t
- inc
requires:
Carp: '0'
File::ShareDir: '0'
Future: '0'
Future::Mojo: '0'
Future::Queue: '0'
Future::Utils: '0'
Mojo::JSON: '0'
Mojo::URL: '0'
Mojo::UserAgent: '0'
Mojolicious: '9'
Moo: '2'
OpenAPI::Modern: '0'
PerlX::Maybe: '0'
Role::EventEmitter: '0'
URI::Template: '0'
YAML::PP: '0'
experimental: '0.031'
perl: '5.020'
stable: '0.031'
resources:
bugtracker: https://github.com/Corion/AI-Ollama-Client/issues
license: https://dev.perl.org/licenses/
repository: git://github.com/Corion/AI-Ollama-Client.git
version: '0.05'
x_serialization_backend: 'CPAN::Meta::YAML version 0.018'
x_static_install: 1
Makefile.PL view on Meta::CPAN
my @tests = map { glob $_ } 't/*.t', 't/*/*.t';
my %module = (
NAME => $module,
AUTHOR => q{Max Maischein <corion@cpan.org>},
VERSION_FROM => $main_file,
ABSTRACT_FROM => $main_file,
META_MERGE => {
"meta-spec" => { version => 2 },
resources => {
repository => {
web => "https://github.com/Corion/$distlink",
url => "git://github.com/Corion/$distlink.git",
type => 'git',
},
bugtracker => {
web => "https://github.com/Corion/$distbase/issues",
# mailto => 'meta-bugs@example.com',
},
license => "https://dev.perl.org/licenses/",
},
dynamic_config => 0, # we promise to keep META.* up-to-date
x_static_install => 1, # we are pure Perl and don't do anything fancy
},
MIN_PERL_VERSION => '5.020', # I use signatures
'LICENSE'=> 'artistic_2',
PL_FILES => {},
CONFIGURE_REQUIRES => {
'ExtUtils::MakeMaker' => 0,
'File::ShareDir::Install' => 0,
},
README.mkdn view on Meta::CPAN
AI::Ollama::Client - Client for AI::Ollama
# SYNOPSIS
use 5.020;
use AI::Ollama::Client;
my $client = AI::Ollama::Client->new(
server => 'https://example.com/',
);
my $res = $client->someMethod()->get;
say $res;
# METHODS
## `checkBlob`
my $res = $client->checkBlob()->get;
Check to see if a blob exists on the Ollama server which is useful when creating models.
## `createBlob`
my $res = $client->createBlob()->get;
Create a blob from a file. Returns the server file path.
## `generateChatCompletion`
my $res = $client->generateChatCompletion()->get;
Generate the next message in a chat with a provided model.
Returns a [AI::Ollama::GenerateChatCompletionResponse](https://metacpan.org/pod/AI%3A%3AOllama%3A%3AGenerateChatCompletionResponse).
## `copyModel`
my $res = $client->copyModel()->get;
Creates a model with another name from an existing model.
## `createModel`
my $res = $client->createModel()->get;
Create a model from a Modelfile.
Returns a [AI::Ollama::CreateModelResponse](https://metacpan.org/pod/AI%3A%3AOllama%3A%3ACreateModelResponse).
## `deleteModel`
my $res = $client->deleteModel()->get;
Delete a model and its data.
## `generateEmbedding`
my $res = $client->generateEmbedding()->get;
Generate embeddings from a model.
Returns a [AI::Ollama::GenerateEmbeddingResponse](https://metacpan.org/pod/AI%3A%3AOllama%3A%3AGenerateEmbeddingResponse).
## `generateCompletion`
use Future::Utils 'repeat';
my $responses = $client->generateCompletion();
repeat {
my ($res) = $responses->shift;
if( $res ) {
my $str = $res->get;
say $str;
}
Future::Mojo->done( defined $res );
} until => sub($done) { $done->get };
Generate a response for a given prompt with a provided model.
Returns a [AI::Ollama::GenerateCompletionResponse](https://metacpan.org/pod/AI%3A%3AOllama%3A%3AGenerateCompletionResponse).
## `pullModel`
my $res = $client->pullModel(
name => 'llama',
)->get;
Download a model from the ollama library.
Returns a [AI::Ollama::PullModelResponse](https://metacpan.org/pod/AI%3A%3AOllama%3A%3APullModelResponse).
## `pushModel`
my $res = $client->pushModel()->get;
Upload a model to a model library.
Returns a [AI::Ollama::PushModelResponse](https://metacpan.org/pod/AI%3A%3AOllama%3A%3APushModelResponse).
## `showModelInfo`
my $info = $client->showModelInfo()->get;
say $info->modelfile;
lib/AI/Ollama/Client.pm view on Meta::CPAN
package AI::Ollama::Client 0.05;
use 5.020;
use Moo 2;
use experimental 'signatures';
use MIME::Base64 'encode_base64';
extends 'AI::Ollama::Client::Impl';
=head1 NAME
AI::Ollama::Client - Client for AI::Ollama
=head1 SYNOPSIS
use 5.020;
use AI::Ollama::Client;
my $client = AI::Ollama::Client->new(
server => 'https://example.com/',
);
my $res = $client->someMethod()->get;
say $res;
=head1 METHODS
=head2 C<< checkBlob >>
my $res = $client->checkBlob()->get;
Check to see if a blob exists on the Ollama server which is useful when creating models.
=cut
around 'checkBlob' => sub ( $super, $self, %options ) {
$super->( $self, %options )->then( sub( $res ) {
if( $res->code =~ /^2\d\d$/ ) {
return Future->done( 1 )
} else {
return Future->done( 0 )
}
});
};
=head2 C<< createBlob >>
my $res = $client->createBlob()->get;
Create a blob from a file. Returns the server file path.
=cut
=head2 C<< generateChatCompletion >>
my $res = $client->generateChatCompletion()->get;
Generate the next message in a chat with a provided model.
Returns a L<< AI::Ollama::GenerateChatCompletionResponse >>.
=cut
=head2 C<< copyModel >>
my $res = $client->copyModel()->get;
Creates a model with another name from an existing model.
=cut
=head2 C<< createModel >>
my $res = $client->createModel()->get;
Create a model from a Modelfile.
Returns a L<< AI::Ollama::CreateModelResponse >>.
=cut
=head2 C<< deleteModel >>
my $res = $client->deleteModel()->get;
Delete a model and its data.
=cut
=head2 C<< generateEmbedding >>
my $res = $client->generateEmbedding()->get;
Generate embeddings from a model.
Returns a L<< AI::Ollama::GenerateEmbeddingResponse >>.
=cut
=head2 C<< generateCompletion >>
use Future::Utils 'repeat';
my $responses = $client->generateCompletion();
repeat {
my ($res) = $responses->shift;
if( $res ) {
my $str = $res->get;
say $str;
}
Future::Mojo->done( defined $res );
} until => sub($done) { $done->get };
Generate a response for a given prompt with a provided model.
Returns a L<< AI::Ollama::GenerateCompletionResponse >>.
=cut
around 'generateCompletion' => sub ( $super, $self, %options ) {
# Encode images as base64, if images exist:
# (but create a copy so we don't over write the input array)
if (my $images = $options{images}) {
lib/AI/Ollama/Client.pm view on Meta::CPAN
$item = Mojo::File->new($item->{filename})->slurp();
};
encode_base64($item)
} @$images ];
}
return $super->($self, %options);
};
=head2 C<< pullModel >>
my $res = $client->pullModel(
name => 'llama',
)->get;
Download a model from the ollama library.
Returns a L<< AI::Ollama::PullModelResponse >>.
=cut
=head2 C<< pushModel >>
my $res = $client->pushModel()->get;
Upload a model to a model library.
Returns a L<< AI::Ollama::PushModelResponse >>.
=cut
=head2 C<< showModelInfo >>
my $info = $client->showModelInfo()->get;
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
package AI::Ollama::Client::Impl 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
with 'Role::EventEmitter';
use experimental 'signatures';
use PerlX::Maybe;
use Carp 'croak';
# These should go into a ::Role
use YAML::PP;
use Mojo::UserAgent;
use Mojo::URL;
use URI::Template;
use Mojo::JSON 'encode_json', 'decode_json';
use OpenAPI::Modern;
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
YAML::PP->new( boolean => 'JSON::PP' )->load_file($fn);
}
},
);
has 'validate_requests' => (
is => 'rw',
default => 1,
);
has 'validate_responses' => (
is => 'rw',
default => 1,
);
has 'openapi' => (
is => 'lazy',
default => sub {
if( my $schema = $_[0]->schema ) {
OpenAPI::Modern->new( openapi_schema => $schema, openapi_uri => '' )
}
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
);
=head1 METHODS
=head2 C<< build_checkBlob_request >>
Build an HTTP request as L<Mojo::Request> object. For the parameters see below.
=head2 C<< checkBlob >>
my $res = $client->checkBlob(
'digest' => '...',
)->get;
Check to see if a blob exists on the Ollama server which is useful when creating models.
=head3 Parameters
=over 4
=item B<< digest >>
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
$self->validate_request( $tx );
return $tx
}
sub checkBlob( $self, %options ) {
my $tx = $self->build_checkBlob_request(%options);
my $res = Future::Mojo->new();
my $r1 = Future::Mojo->new();
$r1->then( sub( $tx ) {
my $resp = $tx->res;
$self->emit(response => $resp);
# Should we validate using OpenAPI::Modern here?!
if( $resp->code == 200 ) {
# Blob exists on the server
$res->done($resp);
} elsif( $resp->code == 404 ) {
# Blob was not found
$res->done($resp);
} else {
# An unknown/unhandled response, likely an error
$res->fail( sprintf( "unknown_unhandled code %d: %s", $resp->code, $resp->body ), $resp);
}
})->retain;
# Start our transaction
$self->emit(request => $tx);
$tx = $self->ua->start_p($tx)->then(sub($tx) {
$r1->resolve( $tx );
undef $r1;
})->catch(sub($err) {
$self->emit(response => $tx, $err);
$r1->fail( $err => $tx );
undef $r1;
});
return $res
}
=head2 C<< build_createBlob_request >>
Build an HTTP request as L<Mojo::Request> object. For the parameters see below.
=head2 C<< createBlob >>
my $res = $client->createBlob(
'digest' => '...',
)->get;
Create a blob from a file. Returns the server file path.
=head3 Parameters
=over 4
=item B<< digest >>
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
$self->validate_request( $tx );
return $tx
}
sub createBlob( $self, %options ) {
my $tx = $self->build_createBlob_request(%options);
my $res = Future::Mojo->new();
my $r1 = Future::Mojo->new();
$r1->then( sub( $tx ) {
my $resp = $tx->res;
$self->emit(response => $resp);
# Should we validate using OpenAPI::Modern here?!
if( $resp->code == 201 ) {
# Blob was successfully created
$res->done($resp);
} else {
# An unknown/unhandled response, likely an error
$res->fail( sprintf( "unknown_unhandled code %d: %s", $resp->code, $resp->body ), $resp);
}
})->retain;
# Start our transaction
$self->emit(request => $tx);
$tx = $self->ua->start_p($tx)->then(sub($tx) {
$r1->resolve( $tx );
undef $r1;
})->catch(sub($err) {
$self->emit(response => $tx, $err);
$r1->fail( $err => $tx );
undef $r1;
});
return $res
}
=head2 C<< build_generateChatCompletion_request >>
Build an HTTP request as L<Mojo::Request> object. For the parameters see below.
=head2 C<< generateChatCompletion >>
use Future::Utils 'repeat';
my $response = $client->generateChatCompletion();
my $streamed = $response->get();
repeat {
my ($res) = $streamed->shift;
if( $res ) {
my $str = $res->get;
say $str;
}
Future::Mojo->done( defined $res );
} until => sub($done) { $done->get };
Generate the next message in a chat with a provided model.
This is a streaming endpoint, so there will be a series of responses. The final response object will include statistics and additional data from the request.
=head3 Options
=over 4
=item C<< format >>
The format to return a response in. Currently the only accepted value is json.
Enable JSON mode by setting the format parameter to json. This will structure the response as valid JSON.
Note: it's important to instruct the model to use JSON in the prompt. Otherwise, the model may generate large amounts whitespace.
=item C<< keep_alive >>
How long (in minutes) to keep the model loaded in memory.
=over
=item -
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
The model name.
Model names follow a C<model:tag> format. Some examples are C<orca-mini:3b-q4_1> and C<llama2:70b>. The tag is optional and, if not provided, will default to C<latest>. The tag is used to identify a specific version.
=item C<< options >>
Additional model parameters listed in the documentation for the Modelfile such as C<temperature>.
=item C<< stream >>
If C<false> the response will be returned as a single response object, otherwise the response will be streamed as a series of objects.
=back
Returns a L<< AI::Ollama::GenerateChatCompletionResponse >> on success.
=cut
sub build_generateChatCompletion_request( $self, %options ) {
my $method = 'POST';
my $path = '/chat';
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
$self->validate_request( $tx );
return $tx
}
sub generateChatCompletion( $self, %options ) {
my $tx = $self->build_generateChatCompletion_request(%options);
my $res = Future::Mojo->new();
my $r1 = Future::Mojo->new();
our @store; # we should use ->retain() instead
push @store, $r1->then( sub( $tx ) {
my $resp = $tx->res;
$self->emit(response => $resp);
# Should we validate using OpenAPI::Modern here?!
if( $resp->code == 200 ) {
# Successful operation.
my $queue = Future::Queue->new( prototype => 'Future::Mojo' );
$res->done( $queue );
my $ct = $resp->headers->content_type;
return unless $ct;
$ct =~ s/;\s+.*//;
if( $ct eq 'application/x-ndjson' ) {
# we only handle ndjson currently
my $handled_offset = 0;
$resp->on(progress => sub($msg,@) {
my $fresh = substr( $msg->body, $handled_offset );
my $body = $msg->body;
$body =~ s/[^\r\n]+\z//; # Strip any unfinished line
$handled_offset = length $body;
my @lines = split /\n/, $fresh;
for (@lines) {
my $payload = decode_json( $_ );
$self->validate_response( $payload, $tx );
$queue->push(
AI::Ollama::GenerateChatCompletionResponse->new($payload),
);
};
if( $msg->{state} eq 'finished' ) {
$queue->finish();
}
});
} else {
# Unknown/unhandled content type
$res->fail( sprintf("unknown_unhandled content type '%s'", $resp->content_type), $resp );
}
} else {
# An unknown/unhandled response, likely an error
$res->fail( sprintf( "unknown_unhandled code %d", $resp->code ), $resp);
}
});
my $_tx;
$tx->res->once( progress => sub($msg, @) {
$r1->resolve( $tx );
undef $_tx;
undef $r1;
});
$self->emit(request => $tx);
$_tx = $self->ua->start_p($tx);
return $res
}
=head2 C<< build_copyModel_request >>
Build an HTTP request as L<Mojo::Request> object. For the parameters see below.
=head2 C<< copyModel >>
my $res = $client->copyModel()->get;
Creates a model with another name from an existing model.
=head3 Options
=over 4
=item C<< destination >>
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
$self->validate_request( $tx );
return $tx
}
sub copyModel( $self, %options ) {
my $tx = $self->build_copyModel_request(%options);
my $res = Future::Mojo->new();
my $r1 = Future::Mojo->new();
$r1->then( sub( $tx ) {
my $resp = $tx->res;
$self->emit(response => $resp);
# Should we validate using OpenAPI::Modern here?!
if( $resp->code == 200 ) {
# Successful operation.
$res->done($resp);
} else {
# An unknown/unhandled response, likely an error
$res->fail( sprintf( "unknown_unhandled code %d: %s", $resp->code, $resp->body ), $resp);
}
})->retain;
# Start our transaction
$self->emit(request => $tx);
$tx = $self->ua->start_p($tx)->then(sub($tx) {
$r1->resolve( $tx );
undef $r1;
})->catch(sub($err) {
$self->emit(response => $tx, $err);
$r1->fail( $err => $tx );
undef $r1;
});
return $res
}
=head2 C<< build_createModel_request >>
Build an HTTP request as L<Mojo::Request> object. For the parameters see below.
=head2 C<< createModel >>
use Future::Utils 'repeat';
my $response = $client->createModel();
my $streamed = $response->get();
repeat {
my ($res) = $streamed->shift;
if( $res ) {
my $str = $res->get;
say $str;
}
Future::Mojo->done( defined $res );
} until => sub($done) { $done->get };
Create a model from a Modelfile.
It is recommended to set C<modelfile> to the content of the Modelfile rather than just set C<path>. This is a requirement for remote create. Remote model creation should also create any file blobs, fields such as C<FROM> and C<ADAPTER>, explicitly wi...
=head3 Options
=over 4
=item C<< modelfile >>
The contents of the Modelfile.
=item C<< name >>
The model name.
Model names follow a C<model:tag> format. Some examples are C<orca-mini:3b-q4_1> and C<llama2:70b>. The tag is optional and, if not provided, will default to C<latest>. The tag is used to identify a specific version.
=item C<< stream >>
If C<false> the response will be returned as a single response object, otherwise the response will be streamed as a series of objects.
=back
Returns a L<< AI::Ollama::CreateModelResponse >> on success.
=cut
sub build_createModel_request( $self, %options ) {
my $method = 'POST';
my $path = '/create';
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
$self->validate_request( $tx );
return $tx
}
sub createModel( $self, %options ) {
my $tx = $self->build_createModel_request(%options);
my $res = Future::Mojo->new();
my $r1 = Future::Mojo->new();
our @store; # we should use ->retain() instead
push @store, $r1->then( sub( $tx ) {
my $resp = $tx->res;
$self->emit(response => $resp);
# Should we validate using OpenAPI::Modern here?!
if( $resp->code == 200 ) {
# Successful operation.
my $queue = Future::Queue->new( prototype => 'Future::Mojo' );
$res->done( $queue );
my $ct = $resp->headers->content_type;
return unless $ct;
$ct =~ s/;\s+.*//;
if( $ct eq 'application/x-ndjson' ) {
# we only handle ndjson currently
my $handled_offset = 0;
$resp->on(progress => sub($msg,@) {
my $fresh = substr( $msg->body, $handled_offset );
my $body = $msg->body;
$body =~ s/[^\r\n]+\z//; # Strip any unfinished line
$handled_offset = length $body;
my @lines = split /\n/, $fresh;
for (@lines) {
my $payload = decode_json( $_ );
$self->validate_response( $payload, $tx );
$queue->push(
AI::Ollama::CreateModelResponse->new($payload),
);
};
if( $msg->{state} eq 'finished' ) {
$queue->finish();
}
});
} else {
# Unknown/unhandled content type
$res->fail( sprintf("unknown_unhandled content type '%s'", $resp->content_type), $resp );
}
} else {
# An unknown/unhandled response, likely an error
$res->fail( sprintf( "unknown_unhandled code %d", $resp->code ), $resp);
}
});
my $_tx;
$tx->res->once( progress => sub($msg, @) {
$r1->resolve( $tx );
undef $_tx;
undef $r1;
});
$self->emit(request => $tx);
$_tx = $self->ua->start_p($tx);
return $res
}
=head2 C<< build_deleteModel_request >>
Build an HTTP request as L<Mojo::Request> object. For the parameters see below.
=head2 C<< deleteModel >>
my $res = $client->deleteModel()->get;
Delete a model and its data.
=head3 Options
=over 4
=item C<< name >>
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
$self->validate_request( $tx );
return $tx
}
sub deleteModel( $self, %options ) {
my $tx = $self->build_deleteModel_request(%options);
my $res = Future::Mojo->new();
my $r1 = Future::Mojo->new();
$r1->then( sub( $tx ) {
my $resp = $tx->res;
$self->emit(response => $resp);
# Should we validate using OpenAPI::Modern here?!
if( $resp->code == 200 ) {
# Successful operation.
$res->done($resp);
} else {
# An unknown/unhandled response, likely an error
$res->fail( sprintf( "unknown_unhandled code %d: %s", $resp->code, $resp->body ), $resp);
}
})->retain;
# Start our transaction
$self->emit(request => $tx);
$tx = $self->ua->start_p($tx)->then(sub($tx) {
$r1->resolve( $tx );
undef $r1;
})->catch(sub($err) {
$self->emit(response => $tx, $err);
$r1->fail( $err => $tx );
undef $r1;
});
return $res
}
=head2 C<< build_generateEmbedding_request >>
Build an HTTP request as L<Mojo::Request> object. For the parameters see below.
=head2 C<< generateEmbedding >>
my $res = $client->generateEmbedding()->get;
Generate embeddings from a model.
=head3 Options
=over 4
=item C<< model >>
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
$self->validate_request( $tx );
return $tx
}
sub generateEmbedding( $self, %options ) {
my $tx = $self->build_generateEmbedding_request(%options);
my $res = Future::Mojo->new();
my $r1 = Future::Mojo->new();
$r1->then( sub( $tx ) {
my $resp = $tx->res;
$self->emit(response => $resp);
# Should we validate using OpenAPI::Modern here?!
if( $resp->code == 200 ) {
# Successful operation.
my $ct = $resp->headers->content_type;
$ct =~ s/;\s+.*//;
if( $ct eq 'application/json' ) {
my $payload = $resp->json();
$self->validate_response( $payload, $tx );
$res->done(
AI::Ollama::GenerateEmbeddingResponse->new($payload),
);
} else {
# Unknown/unhandled content type
$res->fail( sprintf("unknown_unhandled content type '%s'", $resp->content_type), $resp );
}
} else {
# An unknown/unhandled response, likely an error
$res->fail( sprintf( "unknown_unhandled code %d: %s", $resp->code, $resp->body ), $resp);
}
})->retain;
# Start our transaction
$self->emit(request => $tx);
$tx = $self->ua->start_p($tx)->then(sub($tx) {
$r1->resolve( $tx );
undef $r1;
})->catch(sub($err) {
$self->emit(response => $tx, $err);
$r1->fail( $err => $tx );
undef $r1;
});
return $res
}
=head2 C<< build_generateCompletion_request >>
Build an HTTP request as L<Mojo::Request> object. For the parameters see below.
=head2 C<< generateCompletion >>
use Future::Utils 'repeat';
my $response = $client->generateCompletion();
my $streamed = $response->get();
repeat {
my ($res) = $streamed->shift;
if( $res ) {
my $str = $res->get;
say $str;
}
Future::Mojo->done( defined $res );
} until => sub($done) { $done->get };
Generate a response for a given prompt with a provided model.
The final response object will include statistics and additional data from the request.
=head3 Options
=over 4
=item C<< context >>
The context parameter returned from a previous request to [generateCompletion], this can be used to keep a short conversational memory.
=item C<< format >>
The format to return a response in. Currently the only accepted value is json.
Enable JSON mode by setting the format parameter to json. This will structure the response as valid JSON.
Note: it's important to instruct the model to use JSON in the prompt. Otherwise, the model may generate large amounts whitespace.
=item C<< images >>
(optional) a list of Base64-encoded images to include in the message (for multimodal models such as llava)
=item C<< keep_alive >>
How long (in minutes) to keep the model loaded in memory.
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
The model name.
Model names follow a C<model:tag> format. Some examples are C<orca-mini:3b-q4_1> and C<llama2:70b>. The tag is optional and, if not provided, will default to C<latest>. The tag is used to identify a specific version.
=item C<< options >>
Additional model parameters listed in the documentation for the Modelfile such as C<temperature>.
=item C<< prompt >>
The prompt to generate a response.
=item C<< raw >>
If C<true> no formatting will be applied to the prompt and no context will be returned.
You may choose to use the C<raw> parameter if you are specifying a full templated prompt in your request to the API, and are managing history yourself.
=item C<< stream >>
If C<false> the response will be returned as a single response object, otherwise the response will be streamed as a series of objects.
=item C<< system >>
The system prompt to (overrides what is defined in the Modelfile).
=item C<< template >>
The full prompt or prompt template (overrides what is defined in the Modelfile).
=back
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
$self->validate_request( $tx );
return $tx
}
sub generateCompletion( $self, %options ) {
my $tx = $self->build_generateCompletion_request(%options);
my $res = Future::Mojo->new();
my $r1 = Future::Mojo->new();
our @store; # we should use ->retain() instead
push @store, $r1->then( sub( $tx ) {
my $resp = $tx->res;
$self->emit(response => $resp);
# Should we validate using OpenAPI::Modern here?!
if( $resp->code == 200 ) {
# Successful operation.
my $queue = Future::Queue->new( prototype => 'Future::Mojo' );
$res->done( $queue );
my $ct = $resp->headers->content_type;
return unless $ct;
$ct =~ s/;\s+.*//;
if( $ct eq 'application/x-ndjson' ) {
# we only handle ndjson currently
my $handled_offset = 0;
$resp->on(progress => sub($msg,@) {
my $fresh = substr( $msg->body, $handled_offset );
my $body = $msg->body;
$body =~ s/[^\r\n]+\z//; # Strip any unfinished line
$handled_offset = length $body;
my @lines = split /\n/, $fresh;
for (@lines) {
my $payload = decode_json( $_ );
$self->validate_response( $payload, $tx );
$queue->push(
AI::Ollama::GenerateCompletionResponse->new($payload),
);
};
if( $msg->{state} eq 'finished' ) {
$queue->finish();
}
});
} else {
# Unknown/unhandled content type
$res->fail( sprintf("unknown_unhandled content type '%s'", $resp->content_type), $resp );
}
} else {
# An unknown/unhandled response, likely an error
$res->fail( sprintf( "unknown_unhandled code %d", $resp->code ), $resp);
}
});
my $_tx;
$tx->res->once( progress => sub($msg, @) {
$r1->resolve( $tx );
undef $_tx;
undef $r1;
});
$self->emit(request => $tx);
$_tx = $self->ua->start_p($tx);
return $res
}
=head2 C<< build_pullModel_request >>
Build an HTTP request as L<Mojo::Request> object. For the parameters see below.
=head2 C<< pullModel >>
use Future::Utils 'repeat';
my $response = $client->pullModel();
my $streamed = $response->get();
repeat {
my ($res) = $streamed->shift;
if( $res ) {
my $str = $res->get;
say $str;
}
Future::Mojo->done( defined $res );
} until => sub($done) { $done->get };
Download a model from the ollama library.
Cancelled pulls are resumed from where they left off, and multiple calls will share the same download progress.
=head3 Options
=over 4
=item C<< insecure >>
Allow insecure connections to the library.
Only use this if you are pulling from your own library during development.
=item C<< name >>
The model name.
Model names follow a C<model:tag> format. Some examples are C<orca-mini:3b-q4_1> and C<llama2:70b>. The tag is optional and, if not provided, will default to C<latest>. The tag is used to identify a specific version.
=item C<< stream >>
If C<false> the response will be returned as a single response object, otherwise the response will be streamed as a series of objects.
=back
Returns a L<< AI::Ollama::PullModelResponse >> on success.
=cut
sub build_pullModel_request( $self, %options ) {
my $method = 'POST';
my $path = '/pull';
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
$self->validate_request( $tx );
return $tx
}
sub pullModel( $self, %options ) {
my $tx = $self->build_pullModel_request(%options);
my $res = Future::Mojo->new();
my $r1 = Future::Mojo->new();
our @store; # we should use ->retain() instead
push @store, $r1->then( sub( $tx ) {
my $resp = $tx->res;
$self->emit(response => $resp);
# Should we validate using OpenAPI::Modern here?!
if( $resp->code == 200 ) {
# Successful operation.
my $queue = Future::Queue->new( prototype => 'Future::Mojo' );
$res->done( $queue );
my $ct = $resp->headers->content_type;
return unless $ct;
$ct =~ s/;\s+.*//;
if( $ct eq 'application/x-ndjson' ) {
# we only handle ndjson currently
my $handled_offset = 0;
$resp->on(progress => sub($msg,@) {
my $fresh = substr( $msg->body, $handled_offset );
my $body = $msg->body;
$body =~ s/[^\r\n]+\z//; # Strip any unfinished line
$handled_offset = length $body;
my @lines = split /\n/, $fresh;
for (@lines) {
my $payload = decode_json( $_ );
$self->validate_response( $payload, $tx );
$queue->push(
AI::Ollama::PullModelResponse->new($payload),
);
};
if( $msg->{state} eq 'finished' ) {
$queue->finish();
}
});
} else {
# Unknown/unhandled content type
$res->fail( sprintf("unknown_unhandled content type '%s'", $resp->content_type), $resp );
}
} else {
# An unknown/unhandled response, likely an error
$res->fail( sprintf( "unknown_unhandled code %d", $resp->code ), $resp);
}
});
my $_tx;
$tx->res->once( progress => sub($msg, @) {
$r1->resolve( $tx );
undef $_tx;
undef $r1;
});
$self->emit(request => $tx);
$_tx = $self->ua->start_p($tx);
return $res
}
=head2 C<< build_pushModel_request >>
Build an HTTP request as L<Mojo::Request> object. For the parameters see below.
=head2 C<< pushModel >>
my $res = $client->pushModel()->get;
Upload a model to a model library.
Requires registering for ollama.ai and adding a public key first.
=head3 Options
=over 4
=item C<< insecure >>
Allow insecure connections to the library.
Only use this if you are pushing to your library during development.
=item C<< name >>
The name of the model to push in the form of /:.
=item C<< stream >>
If C<false> the response will be returned as a single response object, otherwise the response will be streamed as a series of objects.
=back
Returns a L<< AI::Ollama::PushModelResponse >> on success.
=cut
sub build_pushModel_request( $self, %options ) {
my $method = 'POST';
my $path = '/push';
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
$self->validate_request( $tx );
return $tx
}
sub pushModel( $self, %options ) {
my $tx = $self->build_pushModel_request(%options);
my $res = Future::Mojo->new();
my $r1 = Future::Mojo->new();
$r1->then( sub( $tx ) {
my $resp = $tx->res;
$self->emit(response => $resp);
# Should we validate using OpenAPI::Modern here?!
if( $resp->code == 200 ) {
# Successful operation.
my $ct = $resp->headers->content_type;
$ct =~ s/;\s+.*//;
if( $ct eq 'application/json' ) {
my $payload = $resp->json();
$self->validate_response( $payload, $tx );
$res->done(
AI::Ollama::PushModelResponse->new($payload),
);
} else {
# Unknown/unhandled content type
$res->fail( sprintf("unknown_unhandled content type '%s'", $resp->content_type), $resp );
}
} else {
# An unknown/unhandled response, likely an error
$res->fail( sprintf( "unknown_unhandled code %d: %s", $resp->code, $resp->body ), $resp);
}
})->retain;
# Start our transaction
$self->emit(request => $tx);
$tx = $self->ua->start_p($tx)->then(sub($tx) {
$r1->resolve( $tx );
undef $r1;
})->catch(sub($err) {
$self->emit(response => $tx, $err);
$r1->fail( $err => $tx );
undef $r1;
});
return $res
}
=head2 C<< build_showModelInfo_request >>
Build an HTTP request as L<Mojo::Request> object. For the parameters see below.
=head2 C<< showModelInfo >>
my $res = $client->showModelInfo()->get;
Show details about a model including modelfile, template, parameters, license, and system prompt.
=head3 Options
=over 4
=item C<< name >>
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
$self->validate_request( $tx );
return $tx
}
sub showModelInfo( $self, %options ) {
my $tx = $self->build_showModelInfo_request(%options);
my $res = Future::Mojo->new();
my $r1 = Future::Mojo->new();
$r1->then( sub( $tx ) {
my $resp = $tx->res;
$self->emit(response => $resp);
# Should we validate using OpenAPI::Modern here?!
if( $resp->code == 200 ) {
# Successful operation.
my $ct = $resp->headers->content_type;
$ct =~ s/;\s+.*//;
if( $ct eq 'application/json' ) {
my $payload = $resp->json();
$self->validate_response( $payload, $tx );
$res->done(
AI::Ollama::ModelInfo->new($payload),
);
} else {
# Unknown/unhandled content type
$res->fail( sprintf("unknown_unhandled content type '%s'", $resp->content_type), $resp );
}
} else {
# An unknown/unhandled response, likely an error
$res->fail( sprintf( "unknown_unhandled code %d: %s", $resp->code, $resp->body ), $resp);
}
})->retain;
# Start our transaction
$self->emit(request => $tx);
$tx = $self->ua->start_p($tx)->then(sub($tx) {
$r1->resolve( $tx );
undef $r1;
})->catch(sub($err) {
$self->emit(response => $tx, $err);
$r1->fail( $err => $tx );
undef $r1;
});
return $res
}
=head2 C<< build_listModels_request >>
Build an HTTP request as L<Mojo::Request> object. For the parameters see below.
=head2 C<< listModels >>
my $res = $client->listModels()->get;
List models that are available locally.
Returns a L<< AI::Ollama::ModelsResponse >> on success.
=cut
sub build_listModels_request( $self, %options ) {
my $method = 'GET';
lib/AI/Ollama/Client/Impl.pm view on Meta::CPAN
$self->validate_request( $tx );
return $tx
}
sub listModels( $self, %options ) {
my $tx = $self->build_listModels_request(%options);
my $res = Future::Mojo->new();
my $r1 = Future::Mojo->new();
$r1->then( sub( $tx ) {
my $resp = $tx->res;
$self->emit(response => $resp);
# Should we validate using OpenAPI::Modern here?!
if( $resp->code == 200 ) {
# Successful operation.
my $ct = $resp->headers->content_type;
$ct =~ s/;\s+.*//;
if( $ct eq 'application/json' ) {
my $payload = $resp->json();
$self->validate_response( $payload, $tx );
$res->done(
AI::Ollama::ModelsResponse->new($payload),
);
} else {
# Unknown/unhandled content type
$res->fail( sprintf("unknown_unhandled content type '%s'", $resp->content_type), $resp );
}
} else {
# An unknown/unhandled response, likely an error
$res->fail( sprintf( "unknown_unhandled code %d: %s", $resp->code, $resp->body ), $resp);
}
})->retain;
# Start our transaction
$self->emit(request => $tx);
$tx = $self->ua->start_p($tx)->then(sub($tx) {
$r1->resolve( $tx );
undef $r1;
})->catch(sub($err) {
$self->emit(response => $tx, $err);
$r1->fail( $err => $tx );
undef $r1;
});
return $res
}
sub validate_response( $self, $payload, $tx ) {
if( $self->validate_responses
and my $openapi = $self->openapi ) {
my $results = $openapi->validate_response($payload, { request => $tx->req });
if( $results->{error}) {
say $results;
say $tx->res->to_string;
};
};
}
sub validate_request( $self, $tx ) {
if( $self->validate_requests
and my $openapi = $self->openapi ) {
my $results = $openapi->validate_request($tx->req);
if( $results->{error}) {
say $results;
say $tx->req->to_string;
};
};
}
1;
lib/AI/Ollama/CopyModelRequest.pm view on Meta::CPAN
package AI::Ollama::CopyModelRequest 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/CreateModelRequest.pm view on Meta::CPAN
package AI::Ollama::CreateModelRequest 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/CreateModelRequest.pm view on Meta::CPAN
=cut
has 'name' => (
is => 'ro',
isa => Str,
required => 1,
);
=head2 C<< stream >>
If `false` the response will be returned as a single response object, otherwise the response will be streamed as a series of objects.
=cut
has 'stream' => (
is => 'ro',
);
1;
lib/AI/Ollama/CreateModelResponse.pm view on Meta::CPAN
package AI::Ollama::CreateModelResponse 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/DeleteModelRequest.pm view on Meta::CPAN
package AI::Ollama::DeleteModelRequest 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/GenerateChatCompletionRequest.pm view on Meta::CPAN
package AI::Ollama::GenerateChatCompletionRequest 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/GenerateChatCompletionRequest.pm view on Meta::CPAN
=cut
sub as_hash( $self ) {
return { $self->%* }
}
=head1 PROPERTIES
=head2 C<< format >>
The format to return a response in. Currently the only accepted value is json.
Enable JSON mode by setting the format parameter to json. This will structure the response as valid JSON.
Note: it's important to instruct the model to use JSON in the prompt. Otherwise, the model may generate large amounts whitespace.
=cut
has 'format' => (
is => 'ro',
isa => Enum[
"json",
],
lib/AI/Ollama/GenerateChatCompletionRequest.pm view on Meta::CPAN
=cut
has 'options' => (
is => 'ro',
isa => HashRef,
);
=head2 C<< stream >>
If `false` the response will be returned as a single response object, otherwise the response will be streamed as a series of objects.
=cut
has 'stream' => (
is => 'ro',
);
1;
lib/AI/Ollama/GenerateChatCompletionResponse.pm view on Meta::CPAN
package AI::Ollama::GenerateChatCompletionResponse 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/GenerateChatCompletionResponse.pm view on Meta::CPAN
=cut
has 'created_at' => (
is => 'ro',
isa => Str,
);
=head2 C<< done >>
Whether the response has completed.
=cut
has 'done' => (
is => 'ro',
);
=head2 C<< eval_count >>
Number of tokens the response.
=cut
has 'eval_count' => (
is => 'ro',
isa => Int,
);
=head2 C<< eval_duration >>
Time in nanoseconds spent generating the response.
=cut
has 'eval_duration' => (
is => 'ro',
isa => Int,
);
=head2 C<< load_duration >>
lib/AI/Ollama/GenerateChatCompletionResponse.pm view on Meta::CPAN
=cut
has 'prompt_eval_duration' => (
is => 'ro',
isa => Int,
);
=head2 C<< total_duration >>
Time spent generating the response.
=cut
has 'total_duration' => (
is => 'ro',
isa => Int,
);
1;
lib/AI/Ollama/GenerateCompletionRequest.pm view on Meta::CPAN
package AI::Ollama::GenerateCompletionRequest 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/GenerateCompletionRequest.pm view on Meta::CPAN
=cut
has 'context' => (
is => 'ro',
isa => ArrayRef[Int],
);
=head2 C<< format >>
The format to return a response in. Currently the only accepted value is json.
Enable JSON mode by setting the format parameter to json. This will structure the response as valid JSON.
Note: it's important to instruct the model to use JSON in the prompt. Otherwise, the model may generate large amounts whitespace.
=cut
has 'format' => (
is => 'ro',
isa => Enum[
"json",
],
lib/AI/Ollama/GenerateCompletionRequest.pm view on Meta::CPAN
=cut
has 'options' => (
is => 'ro',
isa => HashRef,
);
=head2 C<< prompt >>
The prompt to generate a response.
=cut
has 'prompt' => (
is => 'ro',
isa => Str,
required => 1,
);
=head2 C<< raw >>
lib/AI/Ollama/GenerateCompletionRequest.pm view on Meta::CPAN
You may choose to use the `raw` parameter if you are specifying a full templated prompt in your request to the API, and are managing history yourself.
=cut
has 'raw' => (
is => 'ro',
);
=head2 C<< stream >>
If `false` the response will be returned as a single response object, otherwise the response will be streamed as a series of objects.
=cut
has 'stream' => (
is => 'ro',
);
=head2 C<< system >>
The system prompt to (overrides what is defined in the Modelfile).
lib/AI/Ollama/GenerateCompletionResponse.pm view on Meta::CPAN
package AI::Ollama::GenerateCompletionResponse 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/GenerateCompletionResponse.pm view on Meta::CPAN
=cut
sub as_hash( $self ) {
return { $self->%* }
}
=head1 PROPERTIES
=head2 C<< context >>
An encoding of the conversation used in this response, this can be sent in the next request to keep a conversational memory.
=cut
has 'context' => (
is => 'ro',
isa => ArrayRef[Int],
);
=head2 C<< created_at >>
lib/AI/Ollama/GenerateCompletionResponse.pm view on Meta::CPAN
=cut
has 'created_at' => (
is => 'ro',
isa => Str,
);
=head2 C<< done >>
Whether the response has completed.
=cut
has 'done' => (
is => 'ro',
);
=head2 C<< eval_count >>
Number of tokens the response.
=cut
has 'eval_count' => (
is => 'ro',
isa => Int,
);
=head2 C<< eval_duration >>
Time in nanoseconds spent generating the response.
=cut
has 'eval_duration' => (
is => 'ro',
isa => Int,
);
=head2 C<< load_duration >>
lib/AI/Ollama/GenerateCompletionResponse.pm view on Meta::CPAN
Time spent in nanoseconds evaluating the prompt.
=cut
has 'prompt_eval_duration' => (
is => 'ro',
isa => Int,
);
=head2 C<< response >>
The response for a given prompt with a provided model.
=cut
has 'response' => (
is => 'ro',
isa => Str,
);
=head2 C<< total_duration >>
Time spent generating the response.
=cut
has 'total_duration' => (
is => 'ro',
isa => Int,
);
1;
lib/AI/Ollama/GenerateEmbeddingRequest.pm view on Meta::CPAN
package AI::Ollama::GenerateEmbeddingRequest 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/GenerateEmbeddingResponse.pm view on Meta::CPAN
package AI::Ollama::GenerateEmbeddingResponse 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/Message.pm view on Meta::CPAN
package AI::Ollama::Message 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/Model.pm view on Meta::CPAN
package AI::Ollama::Model 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/ModelInfo.pm view on Meta::CPAN
package AI::Ollama::ModelInfo 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/ModelInfoRequest.pm view on Meta::CPAN
package AI::Ollama::ModelInfoRequest 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/ModelsResponse.pm view on Meta::CPAN
package AI::Ollama::ModelsResponse 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/PullModelRequest.pm view on Meta::CPAN
package AI::Ollama::PullModelRequest 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/PullModelRequest.pm view on Meta::CPAN
=cut
has 'name' => (
is => 'ro',
isa => Str,
required => 1,
);
=head2 C<< stream >>
If `false` the response will be returned as a single response object, otherwise the response will be streamed as a series of objects.
=cut
has 'stream' => (
is => 'ro',
);
1;
lib/AI/Ollama/PullModelResponse.pm view on Meta::CPAN
package AI::Ollama::PullModelResponse 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/PushModelRequest.pm view on Meta::CPAN
package AI::Ollama::PushModelRequest 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/PushModelRequest.pm view on Meta::CPAN
=cut
has 'name' => (
is => 'ro',
isa => Str,
required => 1,
);
=head2 C<< stream >>
If `false` the response will be returned as a single response object, otherwise the response will be streamed as a series of objects.
=cut
has 'stream' => (
is => 'ro',
);
1;
lib/AI/Ollama/PushModelResponse.pm view on Meta::CPAN
package AI::Ollama::PushModelResponse 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/RequestOptions.pm view on Meta::CPAN
package AI::Ollama::RequestOptions 0.05;
# DO NOT EDIT! This is an autogenerated file.
use 5.020;
use Moo 2;
use experimental 'signatures';
use stable 'postderef';
use Types::Standard qw(Enum Str Bool Num Int HashRef ArrayRef);
use MooX::TypeTiny;
use namespace::clean;
=encoding utf8
=head1 NAME
lib/AI/Ollama/RequestOptions.pm view on Meta::CPAN
=cut
has 'mirostat' => (
is => 'ro',
isa => Int,
);
=head2 C<< mirostat_eta >>
Influences how quickly the algorithm responds to feedback from the generated text. A lower learning rate will result in slower adjustments, while a higher learning rate will make the algorithm more responsive. (Default: 0.1)
=cut
has 'mirostat_eta' => (
is => 'ro',
isa => Num,
);
=head2 C<< mirostat_tau >>
Controls the balance between coherence and diversity of the output. A lower value will result in more focused and coherent text. (Default: 5.0)
=cut
has 'mirostat_tau' => (
is => 'ro',
isa => Num,
);
=head2 C<< num_batch >>
lib/AI/Ollama/RequestOptions.pm view on Meta::CPAN
=cut
has 'num_predict' => (
is => 'ro',
isa => Int,
);
=head2 C<< num_thread >>
Sets the number of threads to use during computation. By default, Ollama will detect this for optimal performance. It is recommended to set this value to the number of physical CPU cores your system has (as opposed to the logical number of cores).
=cut
has 'num_thread' => (
is => 'ro',
isa => Int,
);
=head2 C<< numa >>
lib/AI/Ollama/RequestOptions.pm view on Meta::CPAN
=head2 C<< penalize_newline >>
Penalize newlines in the output. (Default: false)
=cut
has 'penalize_newline' => (
is => 'ro',
);
=head2 C<< presence_penalty >>
Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.
=cut
has 'presence_penalty' => (
is => 'ro',
isa => Num,
);
=head2 C<< repeat_last_n >>
Sets how far back for the model to look back to prevent repetition. (Default: 64, 0 = disabled, -1 = num_ctx)
=cut
ollama/ollama-curated.json view on Meta::CPAN
{"openapi":"3.0.3","components":{"schemas":{"PushModelResponse":{"properties":{"total":{"type":"integer","description":"total size of the model","example":"2142590208"},"status":{"$ref":"#/components/schemas/PushModelStatus"},"digest":{"example":"sha...
ollama/ollama-curated.yaml view on Meta::CPAN
version: 0.1.9
#servers:
# - url: http://localhost:11434/api
# description: Ollama server URL
tags:
- name: Completions
description: Given a prompt, the model will generate a completion.
- name: Chat
description: Given a list of messages comprising a conversation, the model will return a response.
- name: Embeddings
description: Get a vector representation of a given input.
- name: Models
description: List and describe the various models available.
paths:
/generate:
post:
operationId: generateCompletion
tags:
- Completions
summary: Generate a response for a given prompt with a provided model.
description: The final response object will include statistics and additional data from the request.
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/GenerateCompletionRequest'
responses:
'200':
description: Successful operation.
content:
application/x-ndjson:
schema:
$ref: '#/components/schemas/GenerateCompletionResponse'
/chat:
post:
operationId: generateChatCompletion
tags:
- Chat
summary: Generate the next message in a chat with a provided model.
description: This is a streaming endpoint, so there will be a series of responses. The final response object will include statistics and additional data from the request.
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/GenerateChatCompletionRequest'
responses:
'200':
description: Successful operation.
content:
application/x-ndjson:
schema:
$ref: '#/components/schemas/GenerateChatCompletionResponse'
/embeddings:
post:
operationId: generateEmbedding
tags:
- Embeddings
summary: Generate embeddings from a model.
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/GenerateEmbeddingRequest'
responses:
'200':
description: Successful operation.
content:
application/json:
schema:
$ref: '#/components/schemas/GenerateEmbeddingResponse'
/create:
post:
operationId: createModel
tags:
- Models
summary: Create a model from a Modelfile.
description: It is recommended to set `modelfile` to the content of the Modelfile rather than just set `path`. This is a requirement for remote create. Remote model creation should also create any file blobs, fields such as `FROM` and `ADAPTER`...
requestBody:
description: Create a new model from a Modelfile.
content:
application/json:
schema:
$ref: '#/components/schemas/CreateModelRequest'
responses:
'200':
description: Successful operation.
content:
application/x-ndjson:
schema:
$ref: '#/components/schemas/CreateModelResponse'
/tags:
get:
operationId: listModels
tags:
- Models
summary: List models that are available locally.
responses:
'200':
description: Successful operation.
content:
application/json:
schema:
$ref: '#/components/schemas/ModelsResponse'
/show:
post:
operationId: showModelInfo
tags:
- Models
summary: Show details about a model including modelfile, template, parameters, license, and system prompt.
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ModelInfoRequest'
responses:
'200':
description: Successful operation.
content:
application/json:
schema:
$ref: '#/components/schemas/ModelInfo'
/copy:
post:
operationId: copyModel
tags:
- Models
summary: Creates a model with another name from an existing model.
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/CopyModelRequest'
responses:
'200':
description: Successful operation.
/delete:
delete:
operationId: deleteModel
tags:
- Models
summary: Delete a model and its data.
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/DeleteModelRequest'
responses:
'200':
description: Successful operation.
/pull:
post:
operationId: pullModel
tags:
- Models
summary: Download a model from the ollama library.
description: Cancelled pulls are resumed from where they left off, and multiple calls will share the same download progress.
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/PullModelRequest'
responses:
'200':
description: Successful operation.
content:
application/x-ndjson:
schema:
$ref: '#/components/schemas/PullModelResponse'
/push:
post:
operationId: pushModel
tags:
- Models
summary: Upload a model to a model library.
description: Requires registering for ollama.ai and adding a public key first.
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/PushModelRequest'
responses:
'200':
description: Successful operation.
content:
application/json:
schema:
$ref: '#/components/schemas/PushModelResponse'
"/blobs/{digest}":
head:
operationId: checkBlob
tags:
- Models
summary: Check to see if a blob exists on the Ollama server which is useful when creating models.
parameters:
- in: path
name: digest
schema:
type: string
required: true
description: the SHA256 digest of the blob
example: sha256:c8edda1f17edd2f1b60253b773d837bda7b9d249a61245931a4d7c9a8d350250
responses:
'200':
description: Blob exists on the server
'404':
description: Blob was not found
post:
operationId: createBlob
tags:
- Models
summary: Create a blob from a file. Returns the server file path.
parameters:
ollama/ollama-curated.yaml view on Meta::CPAN
type: string
required: true
description: the SHA256 digest of the blob
example: sha256:c8edda1f17edd2f1b60253b773d837bda7b9d249a61245931a4d7c9a8d350250
requestBody:
content:
application/octet-stream:
schema:
type: string
format: binary
responses:
'201':
description: Blob was successfully created
components:
schemas:
GenerateCompletionRequest:
type: object
description: Request class for the generate endpoint.
properties:
model:
type: string
description: &model_name |
The model name.
Model names follow a `model:tag` format. Some examples are `orca-mini:3b-q4_1` and `llama2:70b`. The tag is optional and, if not provided, will default to `latest`. The tag is used to identify a specific version.
example: llama2:7b
prompt:
type: string
description: The prompt to generate a response.
example: Why is the sky blue?
images:
type: array
description: (optional) a list of Base64-encoded images to include in the message (for multimodal models such as llava)
items:
type: string
contentEncoding: base64
description: Base64-encoded image (for multimodal models such as llava)
example: iVBORw0KGgoAAAANSUhEUgAAAAkAAAANCAIAAAD0YtNRAAAABnRSTlMA/AD+APzoM1ogAAAAWklEQVR4AWP48+8PLkR7uUdzcMvtU8EhdykHKAciEXL3pvw5FQIURaBDJkARoDhY3zEXiCgCHbNBmAlUiyaBkENoxZSDWnOtBmoAQu7TnT+3WuDOA7KBIkAGAGwiNeqjusp/AAAAAElFTkSuQmCC
system:
ollama/ollama-curated.yaml view on Meta::CPAN
$ref: '#/components/schemas/ResponseFormat'
raw:
type: boolean
description: |
If `true` no formatting will be applied to the prompt and no context will be returned.
You may choose to use the `raw` parameter if you are specifying a full templated prompt in your request to the API, and are managing history yourself.
stream:
type: boolean
description: &stream |
If `false` the response will be returned as a single response object, otherwise the response will be streamed as a series of objects.
default: false
keep_alive:
type: integer
description: &keep_alive |
How long (in minutes) to keep the model loaded in memory.
- If set to a positive duration (e.g. 20), the model will stay loaded for the provided duration.
- If set to a negative duration (e.g. -1), the model will stay loaded indefinitely.
- If set to 0, the model will be unloaded immediately once finished.
- If not set, the model will stay loaded for 5 minutes by default
ollama/ollama-curated.yaml view on Meta::CPAN
temperature:
type: number
format: float
description: |
The temperature of the model. Increasing the temperature will make the model answer more creatively. (Default: 0.8)
repeat_penalty:
type: number
format: float
description: |
Sets how strongly to penalize repetitions. A higher value (e.g., 1.5) will penalize repetitions more strongly, while a lower value (e.g., 0.9) will be more lenient. (Default: 1.1)
presence_penalty:
type: number
format: float
description: |
Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics.
frequency_penalty:
type: number
format: float
description: |
Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.
mirostat:
type: integer
description: |
Enable Mirostat sampling for controlling perplexity. (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
mirostat_tau:
type: number
format: float
description: |
Controls the balance between coherence and diversity of the output. A lower value will result in more focused and coherent text. (Default: 5.0)
mirostat_eta:
type: number
format: float
description: |
Influences how quickly the algorithm responds to feedback from the generated text. A lower learning rate will result in slower adjustments, while a higher learning rate will make the algorithm more responsive. (Default: 0.1)
penalize_newline:
type: boolean
description: |
Penalize newlines in the output. (Default: false)
stop:
type: array
description: Sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence.
items:
type: string
numa:
ollama/ollama-curated.yaml view on Meta::CPAN
description: |
The base of the rope frequency scale. (Default: 1.0)
rope_frequency_scale:
type: number
format: float
description: |
The scale of the rope frequency. (Default: 1.0)
num_thread:
type: integer
description: |
Sets the number of threads to use during computation. By default, Ollama will detect this for optimal performance. It is recommended to set this value to the number of physical CPU cores your system has (as opposed to the logical number o...
ResponseFormat:
type: string
description: |
The format to return a response in. Currently the only accepted value is json.
Enable JSON mode by setting the format parameter to json. This will structure the response as valid JSON.
Note: it's important to instruct the model to use JSON in the prompt. Otherwise, the model may generate large amounts whitespace.
enum:
- json
GenerateCompletionResponse:
type: object
description: The response class for the generate endpoint.
properties:
model:
type: string
description: *model_name
example: llama2:7b
created_at:
type: string
format: date-time
description: Date on which a model was created.
example: 2023-08-04T19:22:45.499127Z
response:
type: string
description: The response for a given prompt with a provided model.
example: The sky appears blue because of a phenomenon called Rayleigh scattering.
done:
type: boolean
description: Whether the response has completed.
example: true
context:
type: array
description: |
An encoding of the conversation used in this response, this can be sent in the next request to keep a conversational memory.
items:
type: integer
example: [ 1, 2, 3 ]
total_duration:
type: integer
description: Time spent generating the response.
example: 5589157167
load_duration:
type: integer
description: Time spent in nanoseconds loading the model.
example: 3013701500
prompt_eval_count:
type: integer
description: Number of tokens in the prompt.
example: 46
prompt_eval_duration:
type: integer
description: Time spent in nanoseconds evaluating the prompt.
example: 1160282000
eval_count:
type: integer
description: Number of tokens the response.
example: 113
eval_duration:
type: integer
description: Time in nanoseconds spent generating the response.
example: 1325948000
GenerateChatCompletionRequest:
type: object
description: Request class for the chat endpoint.
properties:
model:
type: string
description: *model_name
example: llama2:7b
messages:
ollama/ollama-curated.yaml view on Meta::CPAN
description: *stream
default: false
keep_alive:
type: integer
description: *keep_alive
required:
- model
- messages
GenerateChatCompletionResponse:
type: object
description: The response class for the chat endpoint.
properties:
message:
$ref: '#/components/schemas/Message'
model:
type: string
description: *model_name
example: llama2:7b
created_at:
type: string
format: date-time
description: Date on which a model was created.
example: 2023-08-04T19:22:45.499127Z
done:
type: boolean
description: Whether the response has completed.
example: true
total_duration:
type: integer
description: Time spent generating the response.
example: 5589157167
load_duration:
type: integer
description: Time spent in nanoseconds loading the model.
example: 3013701500
prompt_eval_count:
type: integer
description: Number of tokens in the prompt.
example: 46
prompt_eval_duration:
type: integer
description: Time spent in nanoseconds evaluating the prompt.
example: 1160282000
eval_count:
type: integer
description: Number of tokens the response.
example: 113
eval_duration:
type: integer
description: Time in nanoseconds spent generating the response.
example: 1325948000
Message:
type: object
description: A message in the chat endpoint
properties:
role:
type: string
description: The role of the message
enum: [ "system", "user", "assistant" ]
content:
ollama/ollama-curated.yaml view on Meta::CPAN
stream:
type: boolean
description: *stream
default: false
required:
- name
PullModelResponse:
description: |
Response class for pulling a model.
The first object is the manifest. Then there is a series of downloading responses. Until any of the download is completed, the `completed` key may not be included.
The number of files to be downloaded depends on the number of layers specified in the manifest.
type: object
properties:
status:
$ref: '#/components/schemas/PullModelStatus'
digest:
type: string
description: The model's digest.
example: 'sha256:bc07c81de745696fdf5afca05e065818a8149fb0c77266fb584d9b2cba3711a'
( run in 0.820 second using v1.01-cache-2.11-cpan-49f99fa48dc )