ACME-QuoteDB
view release on metacpan or search on metacpan
#TODO give user choices about installing the quotes database
#i.e. location/permissions/owner, others?
use strict;
use warnings;
use Module::Build;
my $class = Module::Build->subclass(code => <<'EOF');
use File::Spec;
use File::Copy;
use File::Basename qw/dirname/;
chmod(0777, dirname($_t));
copy($qdb, $_t);
chmod(0666, $_t);
}
sub ACTION_install {
my $self = shift;
$self->SUPER::ACTION_install(@_);
my $ddir = $self->install_destination('lib');
#TODO give user choices about installing the quotes database
#my $ques = 'Where do you want to install the quotes database?';
#my $quote_dest = $self->prompt($ques, $ddir);
my $db = q{};
my $perms = 0666;
my $d_perms = 0777;
#if ($quote_dest != $ddir) {
# die $! unless -d dirname $quote_dest;
# $db = $quote_dest;
# $perms = $self->prompt('File Permissions for the quotes database?',
# $perms);
#}
if (!$db) {
$db =
File::Spec->catfile($ddir, 'ACME', 'QuoteDB', 'DB', 'quotedb', 'quotes.db');
}
##chown('THIS_UNIX_USER' $db);
## prompt for the chown
# XXX we need to change/fix this
# ideas, let installer/user decide
Revision history for ACME-QuoteDB
0.1.2 Wed Sep 30 23:26:11 PDT 2009
bug fixes:
* Build.PL install changes - SQLite3 could not actually write to the database,
even though, the db file was world writeable. The container dir
also needs to be writable, now it is.
* ensure the database is 0666 for tests as well
0.1.1 Fri Sep 18 02:11:02 PDT 2009
bug fixes:
* default constructor values were not getting set on the object as they should
* loosen untaint filepath - for the dist test failures
0.1.0 Wed Sep 9 23:43:56 PDT 2009
Initial public pre-release (minor version)
META.yml
README
t/00-load.t
t/01-load_quotes.t
t/02-get_quotes.t
t/03-load_quotes_env.t
t/04-get_quotes_more.t
t/04-load_get_quote_utf8.t
t/05-load_quotes_remote.t
t/boilerplate.t
t/data/futurama
t/data/python_quotes.txt
t/data/simpsons_quotes.csv
t/data/simpsons_quotes.tsv.csv
t/data/utf8.csv
t/data/www.amk.ca/quotations/python-quotes/index.html
t/data/www.amk.ca/quotations/python-quotes/page-10.html
t/data/www.amk.ca/quotations/python-quotes/page-2.html
t/data/www.amk.ca/quotations/python-quotes/page-3.html
t/pod-coverage.t
t/pod.t
ACME-QuoteDB
ACME::QuoteDB − API implements CRUD for a Collection of Quotes
(adages/proverbs/sayings/epigrams, etc)
This module provides an easy to use programmitic interface to a dataâ€
base (sqlite3 or mysql) of ’quotes’. (any content really, that can fit
into our "defined format")
For simplicty you can think of it as a modern fancy perl version of
fortune (with a management interface, remote database connection supâ€
port, plus additional features and some not (yet) supported)
Supported actions include: (CRUD)
1 Create
* Adding quote(s)
* ’Batch’ Loading quotes from a file (stream, other database, etc)
1 Read
* Displaying a single quote, random or based on some criteria
* Displaying multiple quotes, based on some criteria
* Displaying a specific number of quotes, based on some search criteria
1 Update
* Update an existing quote
1 Delete
lib/ACME/QuoteDB.pm view on Meta::CPAN
else {
if ($qc_obj->catg_id eq $catg_ids){
# use cat_id if already exists
push @{$quote_ids}, $qc_obj->quot_id;
}
}
}
return $quote_ids;
}
sub _untaint_data {
my ($arr_ref) = @_;
my $ut_ref = ();
foreach my $q (@{$arr_ref}){
if ($q =~ m{\A([0-9]+)\z}sm){
push @{$ut_ref}, $1;
}
}
return $ut_ref;
}
lib/ACME/QuoteDB.pm view on Meta::CPAN
}
if ($source) {
$source =~ s{'}{''}gsm; # sql escape single quote
$source = qq/ AND source = '$source' /;
}
my $qids = q{};
if ($catgs) {
$catgs = _get_ids_if_catgs_exist($catgs);
my $qid_ref = _get_quote_id_from_catg_id($catgs);
$qids = join ',', @{_untaint_data($qid_ref)};
$qids = qq/ AND quot_id IN ($qids) /;
}
($lower, $upper) = _get_if_rating($lower, $upper);
if ($contain) { $contain = qq/ AND quote LIKE '%$contain%' / }
if ($limit) { $limit = qq/ LIMIT '$limit' / };
my @q = Quote->retrieve_from_sql(
qq{ $attr_name $lower $upper $source $qids $contain $limit },
lib/ACME/QuoteDB.pm view on Meta::CPAN
=head1 SYNOPSIS
Easy access to a collection of quotes (the 'Read' part)
As quick one liner:
# randomly display one quote from all available. (like motd, 'fortune')
perl -MACME::QuoteDB -le 'print quote()'
# Say you have populated your quotes database with some quotes from
# 'The Simpsons'
# randomly display one quote from all available for person 'Ralph'
perl -MACME::QuoteDB -le 'print quote({AttrName => "ralph"})'
# example of output
Prinskipper Skippel... Primdable Skimpsker... I found something!
-- Ralph Wiggum
# get 1 quote, only using these categories (you have defined)
perl -MACME::QuoteDB -le 'print quote({Category => [qw(Humor Cartoon ROTFLMAO)]})'
lib/ACME/QuoteDB.pm view on Meta::CPAN
# list all sources
print $sq->list_attr_sources;
# list all categories
print $sq->list_categories;
=head1 DESCRIPTION
This module provides an easy to use programmitic interface
to a database (sqlite3 or mysql) of 'quotes'. (any content really,
that can fit into our L<"defined format"|/"record format">)
For simplicty you can think of it as a modern fancy perl version
of L<fortune|/fortune>
(with a management interface, remote database
connection support,
plus additional features and some not (yet) supported)
Originally, this module was designed for a collection of quotes from a well
known TV show, once I became aware that distributing it as such would be
L<copyright infringement|/'copyright infringement'>, I generalized the module, so it can be loaded
with 'any' content. (in the quote-ish L<format|/"record format">)
=head4 Supported actions include: (CRUD)
=over 4
=item 1 Create
* Adding quote(s)
* 'Batch' Loading quotes from a file (stream, other database, etc)
=item 1 Read
* Displaying a single quote, random or based on some criteria
* Displaying multiple quotes, based on some criteria
* Displaying a specific number of quotes, based on some search criteria
=item 1 Update
* Update an existing quote
lib/ACME/QuoteDB.pm view on Meta::CPAN
$sq->get_quotes({Rating => '7.0'});
# get all quotes containing some specific text:
$sq->get_quotes_contain({Contain => 'til the cow'});
=head4 Examples of L<Create|/Create>
(See L<ACME::QuoteDB::LoadDB> for batch loading)
# add a quote to the database
my $id_of_added = $sq->add_quote({
Quote => 'Hi, I'm Peter,...",
AttrName => 'Peter Griffin',
Source => 'Family American Dad Guy',
Rating => '1.6',
Category => 'TV Humor',
});
=head4 Example of L<Update|/Update>
# update a quote in the database
my $quote_id = $sq->get_quote_id({Quote => 'Hi, I'm Peter,..."});
$sq->update_quote({
QuoteId => $quote_id,
Quote => 'Hi, I'm Peter, and your not!',
AttrName => 'Peter Griffin',
Source => 'Family Guy',
Rating => '5.7',
Category => [qw(TV Humor Crude Adolescent)]
});
# category/quote is a many to many relationship:
# 1 quote can be in many categories. (and of course 1 category can have many quotes)
=head4 Example of L<Delete|/Delete>
# delete a quote from the database
$sq->delete_quote({QuoteId => $quote_id});
=over 2
=item record format
One full quote database record currently consits of 5 fields:
Quote, AttrName, Source, Rating, Category
Quote => 'the quote desired' # mandatory
AttrName => 'who said it' # mandatory
Source => 'where was it said'
Rating => 'how you rate the quote/if at all',
Category => 'what category is the quote in',
For example:
Quote => 'Hi, I'm Peter,...",
AttrName => 'Peter Griffin',
Source => 'Family Guy',
Rating => '8.6',
Category => 'TV Humor',
=item * NOTE: In order for this module to be useful one has to load some quotes
to the database. Hey, just once though :) (see below - L<Loading Quotes|/"LOADING QUOTES">)
=back
=head1 OVERVIEW
Easy, quick auto-CRUD access to a collection of quotes. (which you provide)
Some ideal uses for this module could be:
=over 4
lib/ACME/QuoteDB.pm view on Meta::CPAN
=head1 USAGE
use ACME::QuoteDB;
my $sq = ACME::QuoteDB->new;
print $sq->get_quote;
# examples are based on quotes data in the test database.
# (see tests t/data/)
# get specific quote based on basic text search.
# search all 'ralph' quotes for string 'wookie'
print $sq->get_quotes_contain({
Contain => 'wookie',
AttrName => 'ralph',
Limit => 1 # only return 1 quote (if any)
});
# output:
I bent my wookie.
lib/ACME/QuoteDB.pm view on Meta::CPAN
foreach my $q ( @{$sq->get_quotes({Limit => 6})} ) {
print "$q\n";
}
# get list of available attributions (that have quotes provided by this module)
print $sq->list_attr_names;
# any unique part of name will work
# i.e these will all return the same results (because of our limited
# quotes db data set)
print $sq->get_quotes({AttrName => 'comic book guy'});
print $sq->get_quotes({AttrName => 'comic book'});
print $sq->get_quotes({AttrName => 'comic'});
print $sq->get_quotes({AttrName => 'book'});
print $sq->get_quotes({AttrName => 'book guy'});
print $sq->get_quotes({AttrName => 'guy'});
# get all quotes, only using these categories (you have defined)
print @{$sq->get_quotes({ Category => [qw(Humor ROTFLMAO)] })};
lib/ACME/QuoteDB.pm view on Meta::CPAN
=begin comment
# XXX this is a bug with sub _get_attribution_ids_from_name
#print $sq->get_quotes({AttrName => 'guy'}); would not match 'Guy Smiley'
=end comment
=head2 add_quote
Adds the supplied record to the database
possible Key arguments consist of:
Quote, AttrName, Source, Rating, Category
with only Quote and AttrName being mandatory (all are useful though):
For Example:
my $q = 'Lois: Peter, what did you promise me?' .
"\nPeter: That I wouldn't drink at the stag party." .
lib/ACME/QuoteDB.pm view on Meta::CPAN
my $q = 'Lois: Peter, what did you promise me?' .
"\nPeter: That I wouldn't drink at the stag party." .
"\nLois: And what did you do?" .
"\nPeter: Drank at the stag pa-- ... Whoa. I almost walked into that one.";
my $qid = $sq->get_quote_id({Quote => $q});
print $qid; # 30
=head2 delete_quote (very beta)
deletes an existing quote in the database
takes an valid quote id (see L</get_quote_id>)
possible Key arguments consist of: QuoteId
$sq->delete_quote({QuoteId => $qid});
=head2 update_quote (very beta)
updates an existing quote in the database
possible Key arguments consist of: QuoteId, Quote
my $q = 'Lois: Peter, what did you promise me?' .
"\nPeter: That I wouldn't drink at the stag party." .
"\nLois: And what did you do?" .
"\nPeter: Drank at the stag pa-- ... Whoa. I almost walked into that one.";
$q =~ s/Lois/Marge/xmsg;
$q =~ s/Peter/Homer/xmsg;
lib/ACME/QuoteDB.pm view on Meta::CPAN
=head2 list_attr_names
returns a list of attributions (name) for which we have quotes.
# get list of available attributions (that have quotes provided by this module)
print $sq->list_attr_names;
=head2 list_categories
returns a list of categories defined in the database
# get list of available categories (that have quotes provided by this module)
print $sq->list_categories;
=head2 list_attr_sources
returns a list of attribution sources defined in the database
# get list of attribution sources (that have quotes provided by this module)
print $sq->list_attr_sources;
=head1 LOADING QUOTES
In order to actually use this module, one has to load quotes content,
hopefully this is relativly easy,... (see t/01-load_quotes.t in tests)
lib/ACME/QuoteDB.pm view on Meta::CPAN
format of file must be as follows: (headers)
"Quote", "Attribution Name", "Attribution Source", "Category", "Rating"
for example:
"Quote", "Attribution Name", "Attribution Source", "Category", "Rating"
"I hope this has taught you kids a lesson: kids never learn.","Chief Wiggum","The Simpsons","Humor",9
"Sideshow Bob has no decency. He called me Chief Piggum. (laughs) Oh wait, I get it, he's all right.","Chief Wiggum","The Simpsons","Humor",8
=item 1 if these dont suit your needs, ACME::QuoteDB::LoadDB is sub-classable,
so one can extract data anyway they like and populate the db themselves.
(there is a test that illustrates overriding the stub method, 'dbload')
you need to populate a record data structure:
$self->set_record(quote => q{}); # mandatory
$self->set_record(name => q{}); # mandatory
$self->set_record(source => q{}); # optional but useful
$self->set_record(catg => q{}); # optional but useful
$self->set_record(rating => q{}); # optional but useful
# then to write the record you call
$self->write_record;
lib/ACME/QuoteDB.pm view on Meta::CPAN
=head2 QuoteCatg
=end comment
=head1 DIAGNOSTICS
An error such as:
C<DBD::SQLite::db prepare_cached failed: no such table: ,...>
probably means that you do not have a database created in the correct format.
basically, you need to create the database, usually, on a first run
you need to add the flag (to the loader):
create_db => 1, # first run, create the db
appending to an existing database is the default behaviour
see L<ACME::QuoteDB::LoadDB/create_db_tables>
=head1 CONFIGURATION AND ENVIRONMENT
if you are running perl > 5.8.5 and have access to
install cpan modules, you should have no problem installing this module
(utf-8 support in DBD::SQLite not avaible until 5.8 - we don't support 'non
utf-8 mode)
=over 1
=item * By default, the quotes database used by this module installs in the
system path, 'lib', (See L<Module::Build/"INSTALL PATHS">)
as world writable - i.e. 0666 (and probably owned by root)
If you don't like this, you can modify Build.PL to not chmod the file and it
will install as 444/readonly, you can also set a chown in there for whoever
you want to have RW access to the quotes db.
Alternativly, one can specify a location to a quotes database (file) to use.
(Since the local mode is sqlite3, the file doesn't even need to exist, just
needs read/write access to the path on the filesystem)
Set the environmental variable:
$ENV{ACME_QUOTEDB_PATH} (untested on windows)
(this has to be set before trying a database load and also (everytime before
using this module, obviouly)
Something such as:
BEGIN {
# give alternate path to the DB
# doesn't need to exist, will create
$ENV{ACME_QUOTEDB_PATH} = '/home/me/my_stuff/my_quote_db'
}
* (NOTE: be sure this (BEGIN) exists *before* the 'use ACME::QuoteDB' lines)
The default is to use sqlite3.
In order to connect to a mysql database, several environmental variables
are required.
BEGIN {
# have to set this to use remote database
$ENV{ACME_QUOTEDB_REMOTE} = 'mysql';
$ENV{ACME_QUOTEDB_DB} = 'acme_quotedb';
$ENV{ACME_QUOTEDB_HOST} = 'localhost';
$ENV{ACME_QUOTEDB_USER} = 'acme_user';
$ENV{ACME_QUOTEDB_PASS} = 'acme';
}
Set the above in a begin block.
The database connection is transparent.
Module usage wise, all operations are the same but now
you will be writing to the remote mysql database specified.
(The user will need read/write permissions to the db/tables)
(mysql admin duties are beyond the scope of this module)
The only supported databases at this time are sqlite and mysql.
It is trivial to add support for others
=back
=head1 DEPENDENCIES
L<Carp>
L<Data::Dumper>
lib/ACME/QuoteDB.pm view on Meta::CPAN
=head1 AUTHOR
David Wright, C<< <david_v_wright at yahoo.com> >>
=head1 TODO
=over 2
=item 1 if the database cannot be found, no error is printed!!!
or if you have no write access to it!
"you'll just get 'no attribute can be found,,...", which is cryptic to say
the least!
=item 1 add a dump backup to csv
a backup mechanism for your db to a regular text csv file.
=item 1 clean up tests 'skip if module X' not installed
lib/ACME/QuoteDB.pm view on Meta::CPAN
get_quotes_contain uses %search% to do it's pattern mattching, so that will
miss some obvious searches, which it should find.
i.e.
'Bill' will not find 'Bill' , beginning and endings of words will be off.
XXX - look at search_like, instead of what you are doing now
=end comment
currently, I am not encapsulating the record data structure used
by LoadDB->write. (i.e. it's a typical perl5 ojbect, the blessed hash)
I will for sure be encapsulating all data in a future version.
(so, don't have code that does $self->{record}->{name} = 'value', or you won't
be happy down the road). Instead use $self->get_record('name') (getter) or
$self->set_record(name => 'my attrib') (setter)
When we are using a SQLite database backend ('regular' local usage), we
should probably be using, ORLite instead of Class::DBI
(although we have not seen any issues yet).
Please report any bugs or feature requests to C<bug-acme-quotedb at rt.cpan.org>, or through
the web interface at L<http://rt.cpan.org/NoAuth/ReportBug.html?Queue=ACME-QuoteDB>.
I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.
=head1 SUPPORT
lib/ACME/QuoteDB/DB/DBI.pm view on Meta::CPAN
use File::Basename qw/dirname/;
use Carp qw/croak/;
use Cwd 'abs_path';
use File::Spec;
Readonly my $QUOTES_DATABASE => $ENV{ACME_QUOTEDB_PATH}
|| File::Spec->catfile(_untaint_db_path(),
q(quotedb), q(quotes.db)
);
# set this to use a remote database
# i.e. mysql
Readonly my $REMOTE => $ENV{ACME_QUOTEDB_REMOTE};
# be more specific (or more general) this is mysql
# and 'remote' can be localhost
if ($REMOTE && $REMOTE ne 'mysql') {
croak "mysql is the only remote database supported"
." set ENV{ACME_QUOTEDB_REMOTE} = 'mysql'";
}
elsif ($REMOTE && $REMOTE eq 'mysql') {
my $database = $ENV{ACME_QUOTEDB_DB};
my $host = $ENV{ACME_QUOTEDB_HOST};
my $user = $ENV{ACME_QUOTEDB_USER};
my $pass = $ENV{ACME_QUOTEDB_PASS};
ACME::QuoteDB::DB::DBI->connection(
"DBI:mysql:database=$database;host=$host",$user,$pass,
{
RaiseError => 1,
mysql_enable_utf8 => 1,
}
)
|| croak "can not connect to: $database $!";
}
else {
ACME::QuoteDB::DB::DBI->connection(
'dbi:SQLite:dbname='.$QUOTES_DATABASE, '', '',
{
RaiseError => 1,
unicode => 1,
# func/pragma's may not work here,..(probably isnt' smart anyway)
#count_changes => 0,
lib/ACME/QuoteDB/DB/DBI.pm view on Meta::CPAN
Also see t/01* included with the distribution.
(available from the CPAN if not included on your system)
=head1 SUBROUTINES/METHODS
see L<ACME::QuoteDB>
=head2 get_current_db_path
returns the path to our current database.
determined first by $ENV{ACME_QUOTEDB_PATH}
and next by the default system path to 'quotes.db'
=head1 DIAGNOSTICS
None currently known
=head1 CONFIGURATION AND ENVIRONMENT
By default, the quotes database used by this module installs in a system path,
which means you'll need to be root (sudo :) to load and modify it.
Alternativly, one can specify a location to a quotes database (file) to use.
Set the environmental variable:
$ENV{ACME_QUOTEDB_PATH} (untested on windows)
(this has to be set before trying a database load and also (everytime) before
using this module, obviouly)
see L<ACME::QuoteDB>
and
see L<ACME::QuoteDB::LoadDB>
=head1 DEPENDENCIES
lib/ACME/QuoteDB/LoadDB.pm view on Meta::CPAN
Readonly my @QUOTE_FIELDS => qw/quote name source catg rating/;
# XXX refactor
sub new {
my ($class, $args) = @_;
# TODO encapsulation
my $self = bless {}, $class;
# store each record we extract - keys map to database fields
# TODO proper encapsulation
$self->{record} = {};
$self->{record}->{quote} = q{};
$self->{record}->{rating} = q{};
$self->{record}->{name} = q{};
$self->{record}->{source} = q{};
$self->{record}->{catg} = q{};
$self->{file} = $args->{file};
$self->{dir} = $args->{dir};
$self->{data} = $args->{data};
$self->{file_format} = $args->{file_format};
$FILE_ENCODING = $args->{file_encoding} || $FILE_ENCODING;
$self->{delim} = $args->{delimiter};
$self->{verbose} = $args->{verbose};
$self->{category} = $args->{category};
$self->{rating} = $args->{rating};
$self->{attr_source} = $args->{attr_source};
$self->{orig_args} = $args;
$self->{success} = undef;
lib/ACME/QuoteDB/LoadDB.pm view on Meta::CPAN
}
sub get_record {
my ($self, $field) = @_;
if (not $field){return $self}
return $self->{record}->{$field};
}
sub data_to_db {
my ($self) = @_;
if ($self->{file} and $self->{data} and $self->{dir}){
croak 'only file, data or dir as arg but not both'
}
elsif (! ($self->{file} or $self->{data} or $self->{dir})) {
croak 'file, data or dir needed as arg'
}
if ($self->{file}) {
$self->_parse_file($self->{file});
}
elsif ($self->{data}) {
$self->_parse_data($self->{data});
}
elsif ($self->{dir}) {
my $dir = $self->{dir};
my $e = q{};
foreach my $f (<$dir*>) {
#if (! (-e $f) || -z $f) # no worky - need path info
$self->_parse_file($f);
$e++;
}
if (! $e){croak 'no files to parse in: ', Dumper $dir;};
lib/ACME/QuoteDB/LoadDB.pm view on Meta::CPAN
# supply your own
$self->dbload($file);
}
else {
croak 'unsupported file format requested, format must be csv or tsv';
}
return;
}
sub _parse_data {
my ($self, $data) = @_;
if (!$data) {croak "data empty $data"}
if ($self->{verbose}){carp 'processing data:'};
if ($self->{file_format} =~ /(?:csv|tsv)/sm) {
croak 'TODO: not yet supported';
#$self->dbload_from_csv($data);
}
elsif (($self->{file_format} eq 'html') || ($self->{file_format} eq 'custom')){
# not supported, too many possibilities
# supply your own
$self->dbload($data);
}
else {
croak 'unsupported file format requested, '
.'format must be csv, tsv. html, custom also possible';
}
return $self;
}
sub _confirm_header_order {
lib/ACME/QuoteDB/LoadDB.pm view on Meta::CPAN
$self->create_db_tables_mysql();
}
else {
create_db_tables_sqlite();
}
return $self;
}
# XXX we want the user to supply a pre created database.
# created as such 'CREATE DATABASE $dbn CHARACTER SET utf8 COLLATE utf8_general_ci'
# this get's into too many isseuwith privs and database creation
#Sat Aug 22 13:42:37 PDT 2009
# did this:
#mysql> CREATE DATABASE acme_quotedb CHARACTER SET utf8 COLLATE utf8_general_ci;
#mysql> grant usage on *.* to acme_user@localhost identified by 'acme';
#mysql> grant all privileges on acme_quotedb.* to acme_user@localhost ;
#sub create_db_mysql {
# my ($self) = @_;
#
# # hmmmm, what about priv's access, etc
# # maybe user need to supply a db, they have
# # access to, already created (just the db though)
# ## create our db
# #my $dbhc = DBI->connect('DBI:mysql:database=mysql;host='
# # .$self->{host}, $self->{user}, $self->{pass})
# # || croak "db cannot be accessed $! $DBI::errstr";
#
# #my $dbn = $self->{db};
# #my $db = qq(CREATE DATABASE $dbn CHARACTER SET utf8 COLLATE utf8_general_ci);
# # eval {
# # $dbhc->do($db) or croak $dbhc->errstr;
# # };
# # $@ and croak 'Cannot create database!';
# # $dbhc->disconnect; $dbhc = undef;
#
# my $drh = DBI->install_driver('mysql');
# my $rc = $drh->func("dropdb", $self->{db},
# [$self->{host}, $self->{user}, $self->{password}],
# 'admin'
# );
#
# $rc = $drh->func("createdb", $self->{db},
# [$self->{host}, $self->{user}, $self->{password}],
lib/ACME/QuoteDB/LoadDB.pm view on Meta::CPAN
# );
#}
# XXX refactor with sqlite
sub create_db_tables_mysql {
my ($self) = @_;
# connect to our db
my $c = $self->{db}.';host='.$self->{host};
my $dbh = DBI->connect(
"DBI:mysql:database=$c", $self->{user}, $self->{pass})
|| croak "db cannot be accessed $! $DBI::errstr";
eval {
$dbh->do('DROP TABLE IF EXISTS quote;') or croak $dbh->errstr;
$dbh->do('CREATE TABLE IF NOT EXISTS quote (
quot_id INTEGER NOT NULL AUTO_INCREMENT,
attr_id INTEGER,
quote TEXT,
source TEXT,
lib/ACME/QuoteDB/LoadDB.pm view on Meta::CPAN
quot_id INTEGER,
PRIMARY KEY(id)
);') or croak $dbh->errstr;
$dbh->disconnect or warn $dbh->errstr;
$dbh = undef;
};
return $@ and croak 'Cannot create database tables!';
}
sub create_db_tables_sqlite {
my $db = QDBI->get_current_db_path;
#XXX is there really no way to do this with the existing
# connection?!(class dbi)
my $dbh = DBI->connect('dbi:SQLite:dbname='.$db, '', '')
|| croak "$db cannot be accessed $! $DBI::errstr";
#-- sqlite does not have a varchar datatype: VARCHAR(255)
#-- A column declared INTEGER PRIMARY KEY will autoincrement.
eval {
$dbh->do('DROP TABLE IF EXISTS quote;') or croak $dbh->errstr;
$dbh->do('CREATE TABLE IF NOT EXISTS quote (
quot_id INTEGER PRIMARY KEY,
attr_id INTEGER,
quote TEXT,
source TEXT,
rating REAL
lib/ACME/QuoteDB/LoadDB.pm view on Meta::CPAN
id INTEGER PRIMARY KEY,
catg_id INTEGER,
quot_id INTEGER
);') or croak $dbh->errstr;
$dbh->disconnect or carp $dbh->errstr;
$dbh = undef;
};
return $@ and croak 'Cannot create database tables!';
}
q(My cat's breath smells like cat food. --Ralph Wiggum);
__END__
=head1 NAME
ACME::QuoteDB::LoadDB - Database loader for ACME::QuoteDB
=head1 VERSION
Version 0.1.1
=head1 SYNOPSIS
load a csv file to quotes database
my $load_db = ACME::QuoteDB::LoadDB->new({
file => '/home/me/data/simpsons_quotes.csv',
file_format => 'csv',
});
$load_db->data_to_db;
print $load_db->success; # bool
header columns of the csv file as follows:
"Quote", "Attribution Name", "Attribution Source", "Category", "Rating"
=head1 DESCRIPTION
This module is part of L<ACME::QuoteDB>. This is a Database loader, it
takes (quotes) data and loads into a database
(currently L<sqlite3 or mysql|/'CONFIGURATION AND ENVIRONMENT'>),
which is then accessed by L<ACME::QuoteDB>.
There are several ways to get quote data into the db via this loader:
(There are more aimed towards 'batch' operations, i.e load a bunch of
records quickly)
=over 4
=item 1
* csv file (pre determined format)
pros: quick and easy to load.
cons: getting the quotes data into the correct format need by this module
=item 2
* any source.
One can take quote data from any source, override
L<ACME::QuoteDB::LoadDB/dbload> loader methods to populate a record
and write it to the db.
pros: can get any quote data into the db.
cons: you supply the method. depending on the complexity of the data
source and munging required this will take longer then the other
methods.
=back
=head3 load from csv file
The pre defined csv file format is:
format of file is as follows: (headers)
"Quote", "Attribution Name", "Attribution Source", "Category", "Rating"
for example:
"Quote", "Attribution Name", "Attribution Source", "Category", "Rating"
"I hope this has taught you kids a lesson: kids never learn.","Chief Wiggum","The Simpsons","Humor",9
"Sideshow Bob has no decency. He called me Chief Piggum. (laughs) Oh wait, I get it, he's all right.","Chief Wiggum","The Simpsons","Humor",8
my $load_db = ACME::QuoteDB::LoadDB->new({
file => dirname(__FILE__).'/data/simpsons_quotes.csv',
file_format => 'csv',
});
$load_db->data_to_db;
if (!$load_db->success){print 'failed'}
=head3 load from any source
If those dont catch your interest, ACME::QuoteDB::LoadDB is sub-classable,
so one can extract data anyway they like and populate the db themselves.
(there is a test that illustrates overriding the stub method, 'dbload')
you need to populate a record data structure:
$self->set_record(quote => q{}); # mandatory
$self->set_record(name => q{}); # mandatory
$self->set_record(source => q{}); # optional but useful
$self->set_record(catg => q{}); # optional but useful
$self->set_record(rating => q{}); # optional but useful
# then to write the record you call
$self->write_record;
lib/ACME/QuoteDB/LoadDB.pm view on Meta::CPAN
=head1 OVERVIEW
You have a collection of quotes (adages/sayings/quips/epigrams, etc) for
whatever reason, you use these quotes for whatever reason, you want to
access these quotes in a variety of ways,...
This module is part of L<ACME::QuoteDB>.
This is a Database loader, it takes data (quotes) and loads into a database,
which is then accessed by L<ACME::QuoteDB>.
See L<ACME::QuoteDB>.
=head1 USAGE
General usage, csv/tsv file in the expected format loaded to the database
my $load_db = ACME::QuoteDB::LoadDB->new({
file => '/home/me/data/sorta_funny_quotes.tsv',
file_format => 'tsv',
delimiter => "\t",
# provide a attr_source for all (if not in data)
# data is used first, if not defined use below
attr_source => 'Things Randomly Overheard',
# provide a category for all (if not in data)
category => 'Humor',
# provide a rating for all
rating => 5, # scale 1-10
});
$load_db->data_to_db;
if (!$load_db->success){print 'failed'}
Also see t/01-load_quotes.t included with the distribution.
(available from the CPAN if not included on your system)
=head1 SUBROUTINES/METHODS
lib/ACME/QuoteDB/LoadDB.pm view on Meta::CPAN
=over 4
=item file or directory - one or the other required (not both)
if file, must be in our defined format, full path is needed.
if directory, full path is needed, can supply a basic glob type filter.
example:
{ file => '/home/me/data/simpsons_quotes.csv' }
{ dir => '/home/me/data/*.csv' }
=item file_format - required
can be one of: 'csv', 'tsv', 'custom', or 'html'
if 'html' or 'custom' you must supply the method for parsing.
(see tests for examples)
example:
lib/ACME/QuoteDB/LoadDB.pm view on Meta::CPAN
=item delimiter - optional, default is a comma for csv
csv/tsv options tested: comma(,) and tab(\t)
'html' - not applicable
example:
{ delimiter => "\t" }
=item category - optional, extracted from data if exists, otherwise will use what you
specify
TODO one quote to multiple categories
=item attr_source - extracted from data if exists, otherwise will use what you
specify
example:
{attr_source => 'The Simpsons'}
=item file_encoding - optional
Files being loaded are assumed to be utf8 encoded. if utf8 flag is not detected,
falls back to latin1 (iso-8859-1). If neither of these is correct, set this
option to the encoding your file is in.
=back
=head4 Operation Related Parameters
=over 4
=item dry_run - optional
do not write to the database. Use with verbose flag to see what would have beed
written.
This can be helpful for testing the outcome of Loading results.
i.e. like to confirm that the parsing of your data is correct
example:
{
dry_run => 1,
verbose => 1
}
=item verbose - optional
lib/ACME/QuoteDB/LoadDB.pm view on Meta::CPAN
This can be helpful for testing quotes extraction from file parsing
example:
{verbose => 1}
=item create_db - optional (boolean)
L<ACME::QuoteDB::LoadDB> default behaviour is to always assume there is a
database and append new data to that. (It is usually only needed the first
time one load's data)
setting this parameter to a true value will create a new database.
(so while this is an optional param, it is required at least once ;)
B<NOTE: it is not intelligent, if you hand it a populated database,
it will happily overwrite all data>
B<AGAIN: setting this param will destroy the current database, creating a new
empty one>
example:
{create_db => 1}
=back
=head2 data_to_db
takes the data input provided to new, process' it and writes to the database.
should appropriatly blow up if not successful
=head2 dbload_from_csv
takes a csv file (in our defined format) as an argument, parses it and writes
the data to the database. (uses L<Text::CSV> with pure perl parser)
utf-8 safe. (opens file as utf8)
will croak with message if not successful
=head2 dbload
if your file format is set to 'html' or 'custom' you must
define this method to do your parsing in a sub class.
Load from html is not supported because there are too many
ways to represt the data. (same with 'custom')
(see tests for examples - there is a test for loading a 'fortune' file format)
One can subclass ACME::QuoteDB::LoadDB and override dbload,
to do our html parsing
=head2 debug_record
dump record (show what is set on the internal data structure)
e.g. Data::Dumper
=head2 set_record
only needed it one plans to sub-class this module.
otherwise, is transparent in usage.
if you are sub-classing this module, you would have to populate
this record. (L</write_record> knows about/uses this data structure)
possible fields consist of:
$self->set_record(quote => q{});
$self->set_record(rating => q{});
$self->set_record(name => q{});
$self->set_record(source => q{});
$self->set_record(catg => q{});
currently can only set one attribute at a time.
lib/ACME/QuoteDB/LoadDB.pm view on Meta::CPAN
});
=head2 get_record
only useful it one plans to sub-class this module.
otherwise, is transparent in usage.
if you are sub-classing this module, you would have to populate
this record. [see L</set_record>]
(L</write_record> knows about/uses this data structure)
possible fields consist of:
$self->get_record('quote');
$self->get_record('rating');
$self->get_record('name');
$self->get_record('source');
$self->get_record('catg');
=head2 success
indicates that the database load was successfull
is undef on failure or if trying a L</dry_run>
=head2 write_record
takes the data structure 'record' '$self->get_record'
(which must exist). checks if attribution name ($self->get_record('name')) exists,
if so, uses existing attribution name, otherwsie creates a new one
Load from html is not supported because there are too many
ways to represt the data. (see tests for examples)
One can subclass ACME::QuoteDB::LoadDB and override dbload,
to do our html parsing
=head2 create_db_tables
create an empty quotes database (with correct tables).
(usually only performed the first time you load data)
B<NOTE: will overwrite ALL existing data>
Set 'create_db' parameter (boolean) to a true value upon instantiation
to enable.
The default action is to assume the database (and tables) exist and just
append new L<ACME::QuoteDB::LoadDB> loads to that.
=begin comment
keep pod coverage happy.
# Coverage for ACME::QuoteDB::LoadDB is 71.4%, with 3 naked subroutines:
# Catg
# Quote
# Attr
lib/ACME/QuoteDB/LoadDB.pm view on Meta::CPAN
=head2 create_db_tables_mysql
=end comment
=head1 DIAGNOSTICS
An error such as:
C<DBD::SQLite::db prepare_cached failed: no such table: ,...>
probably means that you do not have a database created in the correct format.
basically, you need to create the database, usually, on a first run
you need to add the flag:
create_db => 1, # first run, create the db
appending to an existing database is the default behaviour
see L</create_db_tables>
=head1 CONFIGURATION AND ENVIRONMENT
if you are running perl > 5.8.5 and have access to
install cpan modules, you should have no problem installing this module
(utf-8 support in Text::CSV not avaible until 5.8 - we don't support 'non
utf-8 mode)
=over 1
=item * By default, the quotes database used by this module installs in the
system path, 'lib', (See L<Module::Build/"INSTALL PATHS">)
as world writable - i.e. 0666 (and probably owned by root)
If you don't like this, you can modify Build.PL to not chmod the file and it
will install as 444/readonly, you can also set a chown in there for whoever
you want to have RW access to the quotes db.
Alternativly, one can specify a location to a quotes database (file) to use.
(Since the local mode is sqlite3, the file doesn't even need to exist, just
needs read/write access to the path)
Set the environmental variable:
$ENV{ACME_QUOTEDB_PATH} (untested on windows)
(this has to be set before trying a database load and also (everytime) before
using this module, obviouly)
Something such as:
BEGIN {
# give alternate path to the DB
# doesn't need to exist, will create
$ENV{ACME_QUOTEDB_PATH} = '/home/me/my_stuff/my_quote_db'
}
* (NOTE: be sure this (BEGIN) exists *before* the 'use ACME::QuoteDB' lines)
The default is to use sqlite3.
In order to connect to a mysql database, several environmental variables
are required.
BEGIN {
# have to set this to use remote database
$ENV{ACME_QUOTEDB_REMOTE} = 'mysql';
$ENV{ACME_QUOTEDB_DB} = 'acme_quotedb';
$ENV{ACME_QUOTEDB_HOST} = 'localhost';
$ENV{ACME_QUOTEDB_USER} = 'acme_user';
$ENV{ACME_QUOTEDB_PASS} = 'acme';
}
Set the above in a begin block and all operations are the same but now
you will be writing to the remote mysql database specified.
(The user will need read/write permissions to the db/tables)
(mysql admin duties are beyond the scope of this module)
The only supported databases at this time are sqlite and mysql.
It is trivial to add support for others
see: L<LOADING QUOTES|ACME::QuoteDB/LOADING QUOTES>
=back
t/01-load_quotes.t view on Meta::CPAN
use ACME::QuoteDB;
use ACME::QuoteDB::LoadDB;
#use Test::More 'no_plan';
use Test::More tests => 29;
use File::Basename qw/dirname/;
use Data::Dumper qw/Dumper/;
use File::Spec;
# A. test dry run, show if parsing is succesful but don't load the database
{
my $q = File::Spec->catfile((dirname(__FILE__),'data'),
'simpsons_quotes.tsv.csv'
);
# only 2 supported formats: 'simple' text (which is the default) and 'tsv'
my $load_db = ACME::QuoteDB::LoadDB->new({
file => $q,
file_format => 'tsv', # the only supported format
delimiter => "\t",
# provide a category for all (if not in data)
category => 'Humor',
# provide a attr_source for all (if not in data)
attr_source => 'The Simpsons',
dry_run => 1, # don't write to the database
#verbose => 1, # show what is being done
create_db => 1, # need to create the database
});
isa_ok $load_db, 'ACME::QuoteDB::LoadDB';
$load_db->data_to_db;
#flag not set on dry_run
is $load_db->success, undef; # success only after a database write,
my $sq = ACME::QuoteDB->new;
isa_ok $sq, 'ACME::QuoteDB';
ok ! $sq->list_attr_names;
}
{
my $load_db = ACME::QuoteDB::LoadDB->new({
file =>
#dirname(__FILE__).'/data/simpsons_quotes.tsv.csv',
File::Spec->catfile(
(dirname(__FILE__),'data'),
'simpsons_quotes.tsv.csv'
),
file_format => 'tsv',
delimiter => "\t",
#verbose => 1,
create_db => 1, # first run, create the db
# provide a attr_source for all (if not in data)
attr_source => 'The Simpsons',
# provide a category for all (if not in data)
category => 'Humor',
# provide a rating for all
rating => 6,
});
isa_ok $load_db, 'ACME::QuoteDB::LoadDB';
$load_db->data_to_db;
ok $load_db->success;
is $load_db->success, 1;
my $sq = ACME::QuoteDB->new;
isa_ok $sq, 'ACME::QuoteDB';
# expected attribution list from our data
my @expected_attribution_list = (
'Apu Nahasapemapetilon',
'Chief Wiggum',
'Comic Book Guy',
'Grandpa Simpson',
'Ralph Wiggum',
);
is( $sq->list_attr_names, join "\n", sort @expected_attribution_list);
}
{
#my $sqf = dirname(__FILE__) . '/data/simpsons_quotes.csv';
my $sqf = File::Spec->catfile((dirname(__FILE__),'data'),
'simpsons_quotes.csv'
);
my $load_db = ACME::QuoteDB::LoadDB->new({
file => $sqf,
file_format => 'csv',
#delimiter => ",", # comma is default
#verbose => 1,
create_db => 1, # first run, create the db
});
isa_ok $load_db, 'ACME::QuoteDB::LoadDB';
$load_db->data_to_db;
ok $load_db->success;
is $load_db->success, 1;
my $sq = ACME::QuoteDB->new;
isa_ok $sq, 'ACME::QuoteDB';
# expected attribution list from our data
my @expected_attribution_list = (
'Apu Nahasapemapetilon',
'Chief Wiggum',
'Comic Book Guy',
'Grandpa Simpson',
'Ralph Wiggum',
);
is( $sq->list_attr_names, join("\n", sort(@expected_attribution_list)));
}
{ # load from html is not supported because there are too many
# ways to represt the data.
# this is an example of extracting quotes from html:
# subclass ACME::QuoteDB::LoadDB and override dbload,
# to do our html parsing
package LoadQuoteDBFromHtml;
use base 'ACME::QuoteDB::LoadDB';
use Carp qw/croak/;
use Data::Dumper qw/Dumper/;
use HTML::TokeParser;
sub dbload {
my ($self, $file) = @_;
my $p = HTML::TokeParser->new($file) || croak $!;
while (my $token = $p->get_tag("p")) {
my $idn = $token->[1]{class} || q{};
my $id = $token->[1]{id} || q{}; # if a quotation is continued (id
#is not set)
next unless $idn and ( $idn eq 'quotation' || $idn eq 'source');
#my $data = $p->get_trimmed_text("/p");
my $data = $p->get_text('p', 'cite');
#warn Dumper $data;
# XXX see $self->set_record in ACME::QuoteDB::LoadDB for fields
# to populate
if ($idn eq 'quotation' and $id) {
$self->set_record(quote => $data);
}
elsif ($idn eq 'quotation' and not $id) {
my $d = $self->get_record('quote') || q{};
$self->set_record(quote => qq{$d $data});
}
elsif ($idn eq 'source'){
my ($name, $source) = split /,/, $data;
if ($name) {
chomp $name;
$name =~ s/\A\s+//xms;
$name =~ s/\s+\z//xms;
}
$self->set_record(name => $name);
$self->set_record(source => $source);
# TODO
#$self->set_record({
# name => $name,
# source => $source
#});
}
if ($self->get_record('quote') and $self->get_record('name')) {
# we provided a category and rating, otherwise would have to
# parse from data too
$self->set_record(catg => $self->{category});
$self->set_record(rating => $self->{rating});
# TODO
#$self->set_record({
# catg => $self->{category},
# rating => $self->{rating}
#});
#$self->debug_record;
t/01-load_quotes.t view on Meta::CPAN
}
}
}
package main;
use File::Basename qw/dirname/;
use File::Spec;
# simple glob pattern accepted
my $py_quot = File::Spec->catfile(
dirname(__FILE__), 'data', 'www.amk.ca', 'quotations',
'python-quotes', '*.html'
);
my $load_db = LoadQuoteDBFromHtml->new({
dir => $py_quot,
file_format => 'html',
create_db => 1, # first run, create the db
# provide a category for all (if not in data)
category => 'Python',
# provide a rating for all (if not in data)
# and desired
rating => 5,
});
isa_ok $load_db, 'ACME::QuoteDB::LoadDB';
$load_db->data_to_db;
ok $load_db->success;
is $load_db->success, 1;
my $sq = ACME::QuoteDB->new;
isa_ok $sq, 'ACME::QuoteDB';
# expected attribution list from our data (ok, so the data has some
# 'inconsistancies',...
#grep "'source'" *.html|sed -e 's/,.*$//g' -e 's/<\/p>//g' -e s'/^.*>//g'| sort -u
#seems more accurate: grep "'source'" *.html|sed -e "s/^.*source'>//g" -e 's/,.*$//g' | sort -u
my @expected_attribution_list = (
'Aaron Watters',
'Alex Martelli',
'Allan Bailey',
'A.M. Kuchling',
'Andrew Mullhaupt',
'Anthony Baxter',
t/01-load_quotes.t view on Meta::CPAN
$self->write_record;
$q = q{};
}
close $source || croak $!;
}
package main;
use File::Basename qw/dirname/;
use File::Spec;
my $fd = File::Spec->catfile(dirname(__FILE__), 'data', 'futurama');
my $load_db = Fortune2QuoteDB->new({
file => $fd,
file_format => 'custom',
delimiter => "%",
#verbose => 1,
create_db => 1, # first run, create the db
# provide a attr_source for all (if not in data)
# use fortune filename for 'source'
attr_source => 'Futurama',
# provide a category for all (if not in data)
category => 'Humor',
# provide a rating for all
rating => 6.2,
});
isa_ok $load_db, 'ACME::QuoteDB::LoadDB';
$load_db->data_to_db;
is $load_db->success, 1;
my $sq = ACME::QuoteDB->new;
isa_ok $sq, 'ACME::QuoteDB';
is( $sq->list_attr_sources, 'Futurama');
is( $sq->list_categories, 'Humor');
is scalar @{$sq->get_quotes({AttrName => 'Leela'})}, 2;
is scalar @{$sq->get_quotes({AttrName => 'Professor'})}, 2;
t/02-get_quotes.t view on Meta::CPAN
sub make_test_db_rw {
use ACME::QuoteDB::DB::DBI;
# yeah, this is supposed to be covered by the build process
# but is failing sometimes,...
chmod 0666, ACME::QuoteDB::DB::DBI->get_current_db_path;
}
{
make_test_db_rw;
my $q = File::Spec->catfile((dirname(__FILE__),'data'), 'simpsons_quotes.csv');
my $load_db = ACME::QuoteDB::LoadDB->new({
file => $q,
file_format => 'csv',
create_db => 1, # first run, create the db
});
isa_ok $load_db, 'ACME::QuoteDB::LoadDB';
$load_db->data_to_db;
is $load_db->success, 1;
}
my $sq = ACME::QuoteDB->new;
# some 'the simpsons' characters
my @expected_attr_name_list = (
'Apu Nahasapemapetilon',
'Chief Wiggum',
'Comic Book Guy',
t/03-load_quotes_env.t view on Meta::CPAN
ok unlink $def_db;
}
else {
ok 'already gone';
}
}
ok -z $ENV{ACME_QUOTEDB_PATH};
{
my $q = File::Spec->catfile((dirname(__FILE__),'data'),
'simpsons_quotes.tsv.csv'
);
my $load_db = ACME::QuoteDB::LoadDB->new({
file => $q,
file_format => 'tsv',
create_db => 1,
delimiter => "\t",
attribution => 'The Simpsons',
category => 'Humor',
rating => 6,
#verbose =>1
});
isa_ok $load_db, 'ACME::QuoteDB::LoadDB';
$load_db->data_to_db;
ok $load_db->success;
is $load_db->success, 1;
my $sq = ACME::QuoteDB->new;
isa_ok $sq, 'ACME::QuoteDB';
# expected attribution list from our data
my @expected_attribution_list = (
'Apu Nahasapemapetilon',
'Chief Wiggum',
'Comic Book Guy',
'Grandpa Simpson',
'Ralph Wiggum',
);
is( $sq->list_attr_names, join "\n", sort @expected_attribution_list);
t/04-get_quotes_more.t view on Meta::CPAN
"\nPeter: Drank at the stag pa-- ... Whoa. I almost walked into that one.";
{
#make test db writeable
use ACME::QuoteDB::DB::DBI;
# yeah, this is supposed to be covered by the build process
# but is failing sometimes,...
chmod 0666, ACME::QuoteDB::DB::DBI->get_current_db_path;
my $q = File::Spec->catfile((dirname(__FILE__),'data'),
'simpsons_quotes.csv'
);
my $load_db = ACME::QuoteDB::LoadDB->new({
file => $q,
file_format => 'csv',
create_db => 1,
});
isa_ok $load_db, 'ACME::QuoteDB::LoadDB';
$load_db->data_to_db;
is $load_db->success, 1;
}
my $sq = ACME::QuoteDB->new;
is $sq->get_quote({Rating => '8.7'}),
"Me fail English? That's unpossible.\n-- Ralph Wiggum";
is( $sq->list_attr_sources, 'The Simpsons');
is( $sq->list_categories, 'Humor');
t/04-load_get_quote_utf8.t view on Meta::CPAN
$@ and croak 'DBD::SQLite is a required dependancy';
# give alternate path to the DB
$ENV{ACME_QUOTEDB_PATH} =
File::Temp->new( UNLINK => 0,
EXLOCK => 0,
SUFFIX => '.dat',
);
}
# matches the data in our utf8.csv file, soon to be in our quote db
my $utf8_quotes = [
'¥ · £ · € · $ · ¢ · ₡ · ₢ · ₣ · ₤ · ₥ · ₦ · ₧ · ₨ · ₩ · ₪ · ₫ · â‚ · ₮ · ₯',
'我能åžä¸‹çŽ»ç’ƒè€Œä¸ä¼¤èº«ä½“。',
'ç§ã¯ã‚¬ãƒ©ã‚¹ã‚’食ã¹ã‚‰ã‚Œã¾ã™ã€‚ãã‚Œã¯ç§ã‚’å‚·ã¤ã‘ã¾ã›ã‚“。',
'나는 ìœ ë¦¬ë¥¼ ë¨¹ì„ ìˆ˜ 있어요. ê·¸ëž˜ë„ ì•„í”„ì§€ ì•Šì•„ìš”',
'Tsésǫʼ yishÄ…ÌÄ…go bÃÃnÃshghah dóó doo shiÅ‚ neezgai da. ',
'ΜποÏÏŽ να φάω σπασμÎνα γυαλιά χωÏίς να πάθω τίποτα.',
'मैं काà¤à¤š खा सकता हूà¤, मà¥à¤à¥‡ उस से कोई पीडा नहीं होती.',
'×× ×™ יכול ל×כול זכוכית וזה ×œ× ×ž×–×™×§ לי',
];# any takers for specifying each multibyte code sequence for the above,.. ;)
{
#make test db writeable
use ACME::QuoteDB::DB::DBI;
# yeah, this is supposed to be covered by the build process
# but is failing sometimes,...
chmod 0666, ACME::QuoteDB::DB::DBI->get_current_db_path;
my $q = File::Spec->catfile((dirname(__FILE__),'data'),
'utf8.csv'
);
my $load_db = ACME::QuoteDB::LoadDB->new({
file => $q,
file_format => 'csv',
delimiter => "\t",
create_db => 1
});
isa_ok $load_db, 'ACME::QuoteDB::LoadDB';
$load_db->data_to_db;
is $load_db->success, 1;
}
my $sq = ACME::QuoteDB->new;
# matches the data in our utf8.csv file, attribution's to the 'quotes' above
my @expected_attribution_list = (
'UTF-8 Sampler Currency',
'I can eat grass (Chinese)',
'I can eat grass (Japanese)',
'I can eat grass (Korean)',
'I can eat grass (Navajo)',
'I can eat grass (Greek)',
'I can eat grass (Hindi)',
'I can eat grass (Hebrew)',
);
t/05-load_quotes_remote.t view on Meta::CPAN
#use Test::More qw/no_plan/;
use File::Basename qw/dirname/;
use DBI;
use File::Temp;
use File::Spec;
BEGIN {
eval "use DBI";
$@ and plan skip_all => 'DBI/mysql is required for this test';
# have to set this to use remote database
$ENV{ACME_QUOTEDB_REMOTE} = 'mysql';
$ENV{ACME_QUOTEDB_DB} = 'acme_quotedb';
$ENV{ACME_QUOTEDB_HOST} = 'localhost';
$ENV{ACME_QUOTEDB_USER} = 'acme_user';
$ENV{ACME_QUOTEDB_PASS} = 'acme';
}
my $database = $ENV{ACME_QUOTEDB_DB};
my $host = $ENV{ACME_QUOTEDB_HOST};
my $user = $ENV{ACME_QUOTEDB_USER};
my $pass = $ENV{ACME_QUOTEDB_PASS};
# XXX these use's must happen after the BEGIN,...
use ACME::QuoteDB::LoadDB;
use ACME::QuoteDB;
eval {
my $q = File::Spec->catfile((dirname(__FILE__),'data'),
'simpsons_quotes.csv'
);
my $load_db = ACME::QuoteDB::LoadDB->new({
file => $q,
file_format => 'csv',
create_db => 1,
});
};
$@ and plan skip_all => 'mysql not installed or not configured for test user';
# ok, still here? let's run some tests
plan tests => 7;
{ # create it
my $q = File::Spec->catfile((dirname(__FILE__),'data'),
'simpsons_quotes.csv'
);
my $load_db = ACME::QuoteDB::LoadDB->new({
file => $q,
file_format => 'csv',
create_db => 1,
});
isa_ok $load_db, 'ACME::QuoteDB::LoadDB';
$load_db->data_to_db;
ok $load_db->success;
is $load_db->success, 1;
my $sq = ACME::QuoteDB->new;
isa_ok $sq, 'ACME::QuoteDB';
# expected attribution list from our data
my @expected_attribution_list = (
'Apu Nahasapemapetilon',
'Chief Wiggum',
'Comic Book Guy',
'Grandpa Simpson',
'Ralph Wiggum',
);
is( $sq->list_attr_names, join "\n", sort @expected_attribution_list);
$load_db = undef;
}
my $dbh = DBI->connect("DBI:mysql:database=$database;host=$host",$user,$pass)
|| croak "can not connect to: $database $!";
my $count = $dbh->selectrow_hashref('SELECT COUNT(*) AS COUNT FROM quote');
is $count->{COUNT}, 29 ;
my $qc = $dbh->selectrow_hashref('SELECT COUNT(*) AS COUNT FROM quote_catg');
is $qc->{COUNT}, 29 ;
t/data/python_quotes.txt view on Meta::CPAN
... we need more people like him, who are willing to explore without being
driven to argue with people about it.
-- William Tanksley on Chuck Moore, inventor of Forth, 2 Jul 1999
Sorry for the term, I picked it up from Jim Fulton back when it was an
about-to-be-added feature for Principia/Aqueduct. As with so many Fultonisms,
it's vivid and tends to stick in one's (non-pluggable) brain.
-- Paul Everitt on the term "pluggable brains", 5 Jul 1999
I picture a lump of inanimate flesh (a result from a relational database query)
being infused with the spark of life (object behavior, aka class).
-- Jim Fulton on the term "pluggable brains", 5 Jul 1999
This is good. It means that while Ionesco is dead, his spirit lives on.
-- Gordon McMillan on how Windows attaches meaning to 3-character
file extensions, 30 Jul 1999
(On the statement print "42 monkeys"+"1 snake") BTW, both Perl and Python get
this wrong. Perl gives 43 and Python gives "42 monkeys1 snake", when the answer
is clearly "41 monkeys and 1 fat snake".
t/data/python_quotes.txt view on Meta::CPAN
The secret to good performance is to prototype and prototype, then code the
bottlenecks in a faster language. The secret to large systems is to prototype
and prototype, until you've got clean separation of the system into managable
pieces, then code in whatever language most suits the need of each piece.
-- Gordon McMillan, 15 Dec 1999
When Jim [Fulton] says "tricky" it means your brain could explode.
-- Michel Pelletier, 15 Dec 1999
You have start-tags, attributes, end-tags and character data. We have all seen
"XML applications" and "XML parsers" which handle this gang- of-four concepts.
... Now we can peer over the parapet and shout "your parser smells of
elderberries" or "I wave my mixed content at your ankles", as long as we like
but the simple gang-of-four base apps will not go away.
-- Sean McGrath, 19 Dec 1999
Abstraction is one of those notions that Python tosses out the window, yet
expresses very well.
-- Gordon McMillan, 6 Jan 2000
t/data/python_quotes.txt view on Meta::CPAN
Me? I hate the whole lambda calculus, not because of what it is, but
because of what many people think it is. They think that it's the whole of
computer science, the ultimate way to express and reason about programs, when
in reality it's merely a shabby and incomplete model of how Fortran fails to
work. The first thing SICP has to do is teach everyone how bad the lambda
calculus model is -- as part of teaching them about a language allegedly based
on lambda calculus.
I'm sorry, was my bias showing again? :-)
-- William Tanksley, 13 May 2000
I never got beyond starting the data-structures in C++, I never got beyond
seeing how it would work in Scheme. I finished it in one Python -filled
afternoon, and discovered the idea sucked big time. I was glad I did it in
Python, because it only cost me one afternoon to discover the idea sucks.
-- Moshe Zadka, 13 May 2000
In truth, we use 'j' to represent sqrt(-1) for exactly the same reason we use a
convention for the direction of current which is exactly the opposite of the
direction the electrons actually travel: because it drives physicists crazy.
(And if we pick up a few mathematicians or whatever along the way, well, that's
just gravy. ;-)
t/data/python_quotes.txt view on Meta::CPAN
software after two bugs we would not be able to turn our computers.
-- Brett Cannon, 13 Jul 2006
... I've come to believe that some people have the personality traits that let
them tolerate redoing the same work over and over again for no reason other
than management "furniture rearranging", whereas others start to resent having
their (working) life repeatedly flashed before their eyes, but in slightly
different colours, over a longer period of time.
-- Paul Boddie, 29 Aug 2006
I am the very model of a modern major database,
For gigabytes of information gathered out in userspace.
For banking applications to a website crackers will deface,
You access me from console or spiffy user interface.
My multi-threaded architecture offers you concurrency,
And loads of RAM for caching things reduces query latency.
The data is correctly typed, a fact that I will guarantee,
Each datum has a data type, it's specified explicitly.
-- Tim Chase, 12 Sep 2006
t/data/www.amk.ca/quotations/python-quotes/page-10.html view on Meta::CPAN
<p class='source'>Brett Cannon, 13 Jul 2006</p>
<p class='quotation' id='q319'>... I've come to believe that some
people have the personality traits that let them tolerate redoing
the same work over and over again for no reason other than
management "furniture rearranging", whereas others start to resent
having their (working) life repeatedly flashed before their eyes,
but in slightly different colours, over a longer period of
time.</p>
<p class='source'>Paul Boddie, 29 Aug 2006</p>
<p class='quotation' id='q320'>I am the very model of a modern
major database,
For gigabytes of information gathered out in userspace.
For banking applications to a website crackers will deface,
You access me from console or spiffy user interface.
My multi-threaded architecture offers you
concurrency, And loads of RAM for caching things reduces query latency.<br />
The data is correctly typed, a fact that I will guarantee,<br />
Each datum has a data type, it's specified explicitly.</p>
<p class='source'>Tim Chase, 12 Sep 2006</p>
<hr /></div>
<small>[<a href="mailto:comments@amk.ca">Contact me</a>]</small>
</body>
</html>
( run in 0.434 second using v1.01-cache-2.11-cpan-496ff517765 )