App-Sqitch
view release on metacpan or search on metacpan
lib/App/Sqitch/Engine/clickhouse.pm view on Meta::CPAN
sub _is_true($) {
my $val = shift || return 0;
# https://github.com/ClickHouse/ClickHouse/blob/ce5a43c/base/poco/Util/src/AbstractConfiguration.cpp#L528C29-L547
return $val != 0 || 0 if looks_like_number $val;
$val = lc $val;
return $val eq 'true' || $val eq 'yes' || $val eq 'on' || 0;
}
# Connection name defaults to host name from url, or else hostname from config
# or else localhost. Then look for that name in a connection under
# `connections_credentials`. If it exists, copy/overwrite `hostname`, `port`,
# `secure`, `user`, `password`, and `database`. Fall back on root object
# values `host` (not `hostname`) `port`, `secure`, `user`, `password`, and
# `database`.
#
# https://github.com/ClickHouse/ClickHouse/blob/d0facf0/programs/client/Client.cpp#L139-L212
sub _conn_cfg {
my ($cfg, $host) = @_;
# Copy root-level configs.
my $conn = {
(exists $cfg->{secure} ? (secure => _is_true $cfg->{secure}) : ()),
map { ( $_ => $cfg->{$_}) } grep { $cfg->{$_} } qw(host port user password database),
};
# Copy client TLS config if exists.
if (my $tls = $cfg->{openSSL}) {
$conn->{tls} = $tls->{client} if $tls->{client};
}
# Copy connection credentials for this host if they exists.
$host ||= $cfg->{host} || 'localhost';
my $creds = $cfg->{connections_credentials} or return $conn;
my $conns = $creds->{connection} or return $conn;
for my $c (@{ ref $conns eq 'ARRAY' ? $conns : [$conns] }) {
next unless ($c->{name} || '') eq $host;
if (exists $c->{secure}) {
$conn->{secure} = _is_true $c->{secure}
}
$conn->{host} = $c->{hostname} if $c->{hostname};
$conn->{$_} = $c->{$_} for grep { $c->{$_} } qw(port user password database);
}
return $conn;
lib/sqitch-authentication.pod view on Meta::CPAN
=encoding UTF-8
=head1 Name
sqitch-authentication - Guide to using database authentication credentials with Sqitch
=head1 Description
For database engines that require authentication, Sqitch supports a number
of credential-specification options, and searches for them in a specific
sequence. These searches are performed in two parts: a search for a username
and a search for a password.
=head1 Usernames
lib/sqitch-authentication.pod view on Meta::CPAN
Sadly, both the C<SNOWSQL_PRIVATE_KEY_PASSPHRASE> environment variable and
the C<priv_key_file_pwd> ODBC parameter must be set, as Sqitch uses ODBC to
maintain its registry and SnowSQL to execute change scripts.
=item ClickHouse
ClickHouse does not support password-less authentication, but does support
L<TLS Client authentication| https://clickhouse.com/docs/guides/sre/ssl-user-auth>.
Sqitch reads the C<$.openSSL.client> section of the
C<Client configuration|https://clickhouse.com/docs/interfaces/cli#connection-credentials>
to also configure is ODBC connection. For example, given this config:
<config>
<openSSL>
<client>
<certificateFile>my_cert_name.crt</certificateFile>
<privateKeyFile>my_cert_name.key</privateKeyFile>
</client>
</openSSL>
</config>
lib/sqitch-authentication.pod view on Meta::CPAN
For ODBC connections from Sqitch, we can use connection settings in F<~/.odbc.ini>:
[flipr_test]
DRIVER = Exasol
EXAHOST = localhost:8563
EXAUID = sys
EXAPWD = exasol
AUTHMETHOD = refreshtoken
When combining the above, Sqitch doesn't need to know any credentials; they are
stored somewhat safely in F<~/.exasol/profiles.xml> and F<~/.odbc.ini>:
> sqitch status db:exasol:flipr_test
# On database db:exasol:flipr_test
# Project: flipr
# ...
#
Nothing to deploy (up-to-date)
>Â sqitch rebase --onto '@HEAD^' -y db:exasol:flipr_test
Reverting changes to hashtags @v1.0.0-dev2 from db:exasol:flipr_test
lib/sqitch-authentication.pod view on Meta::CPAN
=item C<connections.warehousename>
=item C<connections.dbname>
=back
=item ClickHouse
ClickHouse supports password configuration via its
L<client configuration file|https://clickhouse.com/docs/interfaces/cli#connection-credentials>.
Sqitch will read the C<host>, C<port>, C<database>, and C<password> parameters
from either the root of the configuration file, as well as the C<secure> and
C<openSSL.client> parameters, for example:
<config>
<user>username</user>
<password>password</password>
<secure>true</secure>
<openSSL>
<client>
lib/sqitch-authentication.pod view on Meta::CPAN
</openSSL>
</config>
All but the C<openSSL.client> parameters may be set on a named connection
basis, matched by the host name in the target URL (or the
C<$CLICKHOUSE_HOSTNAME> environment variable). All the parameters are the
same except C<host> becomes C<hostname>. For example, to set parameters for
the host C<click.house>:
<config>
<connections_credentials>
<connection>
<name>click.house</name>
<hostname>clickhouse.example.com</hostname>
<port>9440</port>
<secure>1</secure>
<user>default</user>
<password>password</password>
</connection>
</connections_credentials>
</config>
Note that setting C<port> to 9440 will implicitly set the ODBC port to 8443
unless it's explicitly set to something else.
=back
=head2 Use C<$SQITCH_PASSWORD>
The C<$SQITCH_PASSWORD> environment variable can be used to specify the
lib/sqitchguides.pod view on Meta::CPAN
sqitchguides - List of common Sqitch guides
=end private
The common Sqitch guides are:
changes Specifying changes for Sqitch
configuration Hierarchical engine and target configuration
environment Environment variables
authentication Specifying target authentication credentials
tutorial PostgreSQL Tutorial
tutorial-firebird Firebird Tutorial
tutorial-mysql MySQL Tutorial
tutorial-oracle Oracle Tutorial
tutorial-sqlite SQLite Tutorial
tutorial-vertica Vertica Tutorial
tutorial-exasol Exasol Tutorial
tutorial-snowflake Snowflake Tutorial
See C<< sqitch help <command> >> or C<< sqitch help <concept> >> to read about
t/click-conf/client.xml view on Meta::CPAN
<preferServerCiphers>true</preferServerCiphers>
<!-- Use for self-signed: <verificationMode>none</verificationMode> -->
<invalidCertificateHandler>
<!-- Use for self-signed: <name>AcceptCertificateHandler</name> -->
<name>RejectCertificateHandler</name>
</invalidCertificateHandler>
</client>
</openSSL>
<!--
Custom prompt settings for clickhouse-client. See also <connections_credentials>.
Possible macros:
{host}
{port}
{user}
{display_name}
You can also use colored prompt, like in [1].
[1]: https://misc.flogisoft.com/bash/tip_colors_and_formatting
t/click-conf/client.xml view on Meta::CPAN
The same can be done on user-level configuration, just create & adjust: ~/.clickhouse-client/config.xml
-->
<!-- Directory containing the proto files for the well-known Protobuf types.
-->
<google_protos_path>/usr/share/clickhouse/protos/</google_protos_path>
<!-- Analog of .netrc -->
<![CDATA[
<connections_credentials>
<connection>
<!-- Name of the connection, host option for the client.
"host" is not the same as "hostname" since you may want to have different settings for one host,
and in this case you can add "prod" and "prod_readonly".
Default: "hostname" will be used. -->
<name>default</name>
<!-- For self-signed server certificate when connecting to secure tcp: <accept-invalid-certificate>1</accept-invalid-certificate> -->
<!-- Host that will be used for connection. -->
<hostname>127.0.0.1</hostname>
<port>9000</port>
<secure>1</secure>
<user>default</user>
<password></password>
<database></database>
<!-- '~' is expanded to HOME, like in any shell -->
<history_file></history_file>
<prompt></prompt>
</connection>
</connections_credentials>
]]>
</config>
t/click-conf/creds.pl view on Meta::CPAN
{
connections_credentials => {
connection => {
name => 'default',
hostname => 'hostname',
port => 9440,
secure =>1,
user => 'default',
password => 'password',
},
},
}
t/click-conf/creds.xml view on Meta::CPAN
<config>
<connections_credentials>
<connection>
<name>default</name>
<hostname>hostname</hostname>
<port>9440</port>
<secure>1</secure>
<user>default</user>
<password>password</password>
<!-- <history_file></history_file> -->
<!-- <history_max_entries></history_max_entries> -->
<!-- <accept-invalid-certificate>false</accept-invalid-certificate> -->
<!-- <prompt></prompt> -->
</connection>
</connections_credentials>
</config>
t/click-conf/multi-creds.pl view on Meta::CPAN
{
connections_credentials => {
connection => [
{
name => 'default',
user => 'default',
password => 'password',
},
{
name => 'sqitch',
user => 'sqitch',
password => 's3cr3t',
t/click-conf/multi-creds.xml view on Meta::CPAN
<config>
<connections_credentials>
<connection>
<name>default</name>
<user>default</user>
<password>password</password>
</connection>
<connection>
<name>sqitch</name>
<user>sqitch</user>
<password>s3cr3t</password>
</connection>
<connection>
<name>blanket</name>
<user>blanket</user>
<password>howdy</password>
</connection>
</connections_credentials>
</config>
t/clickhouse.t view on Meta::CPAN
['home', $tmp_home, '.clickhouse-client'],
) {
for my $ext (qw(xml yaml yml)) {
my $path = file $spec->[1], "$spec->[2].$ext";
open my $fh, '>:utf8', $path or die "Cannot open $path: $!";
if ($ext eq 'xml') {
print {$fh} qq{
<config>
<user>$spec->[0]</user>
<password>$ext</password>
<connections_credentials>
<connection>
<name>lol.cats</name>
<hostname>cats.example</hostname>
</connection>
</connections_credentials>
</config>
};
} else {
print {$fh} qq{
user: $spec->[0]
password: $ext
connections_credentials:
connection:
- name: lol.cats
hostname: cats.example
};
}
}
}
# Now find them in order.
for my $spec (
t/clickhouse.t view on Meta::CPAN
},
exp => { secure => 0 },
},
{
test => 'default connection',
config => {
secure => 'true',
user => 'sushi',
password => 's3cr3t',
database => 'pets',
connections_credentials => { connection => {
name => 'localhost',
secure => 'false',
hostname => 'cats.lol',
port => 8181,
user => 'biscuit',
password => 'meow',
database => 'cats',
}}
},
exp => {
t/clickhouse.t view on Meta::CPAN
},
},
{
test => 'different host',
config => {
secure => 'true',
host => 'bagel.cat',
user => 'sushi',
password => 's3cr3t',
database => 'pets',
connections_credentials => { connection => {
name => 'localhost',
secure => 'false',
hostname => 'cats.lol',
user => 'biscuit',
password => 'meow',
database => 'cats',
}}
},
exp => {
secure => 1,
host => 'bagel.cat',
user => 'sushi',
password => 's3cr3t',
database => 'pets',
},
},
{
test => 'multiple connections',
config => {
connections_credentials => { connection => [
{
name => 'localhost',
user => 'biscuit',
},
{
name => 'pumpkin',
user => 'pumpkin',
},
]},
},
exp => {
user => 'biscuit',
},
},
{
test => 'repeat connection',
config => {
host => 'cats.lol',
connections_credentials => { connection => [
{
name => 'localhost',
user => 'biscuit',
},
{
user => 'jimmy',
},
{
name => 'cats.lol',
user => 'pumpkin',
t/clickhouse.t view on Meta::CPAN
user => 'strawberry',
},
]},
},
exp => {
host => 'cats.lol',
user => 'strawberry',
},
},
{
test => 'empty connections_credentials',
config => {
host => 'cats.lol',
connections_credentials => {},
},
exp => {
host => 'cats.lol',
},
},
) {
my $got = App::Sqitch::Engine::clickhouse::_conn_cfg(
$tc->{config}, $tc->{host},
);
is_deeply $got, $tc->{exp}, "Should process $tc->{test} config";
( run in 0.497 second using v1.01-cache-2.11-cpan-0ffa90cfd1c )