ClickHouse-Encoder

 view release on metacpan or  search on metacpan

README  view on Meta::CPAN

    transient errors, keep-alive, optional compression. ->summary
    rolls up CH X-ClickHouse-Summary stats across batches;
    ->last_response gives the most recent flush's HTTP response with
    parsed CH metadata attached at ->{ch}{query-id,server,summary,...}.
    ClickHouse::Encoder->for_query($select_sql, host=>..., port=>...)
    runs describe ($select_sql) and returns an encoder configured for
    that result shape; useful when the schema isn't a real table.
    ClickHouse::Encoder->ping(host=>..., port=>...)
    liveness probe via /ping; returns 1 or croaks.

    All HTTP entry points accept scheme=>'https' (needs IO::Socket::SSL
    + Net::SSLeay), ssl_options/verify_SSL pass-throughs to HTTP::Tiny,
    settings=>{...} for per-query CH settings, and dedup_token=>$id for
    idempotent inserts.

SCHEMA INTROSPECTION
    ClickHouse::Encoder->for_table($table, via => 'client', ...)
    ClickHouse::Encoder->for_table($table, via => 'http', port => 8123, ...)
    ClickHouse::Encoder->server_version(host => ..., port => ...)
    fetches select version() over HTTP, returns {major,minor,patch,...}.
    ClickHouse::Encoder->types                  list of supported type names

lib/ClickHouse/Encoder.pm  view on Meta::CPAN

Useful when the schema doesn't correspond to a real table (CTEs,
joins, computed columns). Options are the same as L</for_table>; the
caller is responsible for the SQL.

=head2 insert_http

    my $resp = ClickHouse::Encoder->insert_http(
        host => 'localhost', port => 8123, table => 'events',
        columns => \@cols, rows => \@rows,
        compress => 'zstd',   # optional
        scheme   => 'https',  # optional; needs IO::Socket::SSL + Net::SSLeay
        settings => {         # optional per-query CH settings
            max_execution_time => 30,
            max_memory_usage   => '10G',
        },
        dedup_token => $batch_id,  # optional idempotency token
    );
    die "insert failed (status $resp->{status}): $resp->{content}"
        unless $resp->{success};
    # CH response metadata, when present, is also attached:
    my $qid     = $resp->{ch}{'query-id'};
    my $written = $resp->{ch}{summary}{written_rows};

Thin convenience wrapper that builds an encoder, encodes rows, and
POSTs the bytes to C<< http(s)://host:port/?query=insert into $table format native >>.
Pass C<encoder => $enc> instead of C<columns> to reuse one. C<compress>
accepts C<'raw'> (default), C<'zstd'>, or C<'gzip'>. C<scheme =E<gt> 'https'>
enables TLS via L<HTTP::Tiny>'s SSL support (install
L<IO::Socket::SSL> and L<Net::SSLeay>). C<ssl_options> / C<verify_SSL>
pass through to L<HTTP::Tiny>. C<settings> applies per-query CH
settings via URL params. C<dedup_token> stamps the request with an
C<insert_deduplication_token> so an identical retry is rejected
server-side. Returns the L<HTTP::Tiny> response hashref with a
C<ch =E<gt> { query-id, server, format, exception-code, summary,
progress, ... }> slot containing parsed
C<X-ClickHouse-*> response headers; no automatic retries (do HTTP
policy in the caller, or use L</bulk_inserter>).

=head2 for_table

lib/ClickHouse/Encoder.pm  view on Meta::CPAN

the exponential growth from C<retry_wait>.

=item C<compress>

C<'raw'> (default), C<'zstd'>, or C<'gzip'>. Sets the
C<Content-Encoding> header accordingly.

=item C<scheme>, C<ssl_options>, C<verify_SSL>

C<scheme =E<gt> 'https'> enables TLS via L<HTTP::Tiny> (install
L<IO::Socket::SSL> and L<Net::SSLeay>). C<ssl_options> and
C<verify_SSL> pass through to L<HTTP::Tiny>.

=item C<settings>

Hashref of per-query CH settings (C<max_execution_time>,
C<max_memory_usage>, ...) appended to every flush as URL params.

=item C<dedup_token>

Stamps every POST with C<insert_deduplication_token>. Identical

t/ssl-options.t  view on Meta::CPAN

# the private _http_tiny builder honours them (no network needed - we
# inspect the constructed object's attributes).

# verify_SSL passthrough.
{
    my $t = ClickHouse::Encoder::_http_tiny(verify_SSL => 1);
    isa_ok($t, 'HTTP::Tiny', 'returns an HTTP::Tiny');
    is($t->{verify_SSL}, 1, 'verify_SSL forwarded to constructor');
}

# ssl_options passthrough (a hashref of IO::Socket::SSL options).
{
    my $opts = { SSL_ca_file => '/etc/ssl/certs/ca.pem' };
    my $t = ClickHouse::Encoder::_http_tiny(ssl_options => $opts);
    is_deeply($t->{SSL_options}, $opts, 'ssl_options forwarded as SSL_options');
}

# timeout + keep_alive still work alongside SSL options.
{
    my $t = ClickHouse::Encoder::_http_tiny(
        timeout => 17, keep_alive => 1, verify_SSL => 0);

t/ssl-options.t  view on Meta::CPAN

# No SSL options -> a plain HTTP::Tiny with the default timeout.
{
    my $t = ClickHouse::Encoder::_http_tiny();
    isa_ok($t, 'HTTP::Tiny', 'plain builder');
    ok(!exists $t->{SSL_options},
       'no SSL_options key when ssl_options not given');
}

# https scheme is accepted by the endpoint guard (TLS itself is
# HTTP::Tiny's concern and needs IO::Socket::SSL at request time).
{
    my ($scheme) = ClickHouse::Encoder::_check_endpoint(
        { scheme => 'https', host => 'db', port => 8443 });
    is($scheme, 'https', 'https scheme passes endpoint validation');
}

done_testing();