view release on metacpan or  search on metacpan

doc/catalog.texinfo  view on Meta::CPAN

Catalog that is installed with the product at http://localhost/Catalog/ and
follow the @strong{DMOZ Control Panel} link. Alternatively you can jump 
directly to http://localhost/cgi-bin/DMOZ/dmoz?context=ccontrol_panel.

We have loaded a version of dmoz.org that contains approximately 1 500 000
records and around 250 000 categories on a Pentium 450. It leads to a
500Mb MySQL database. It takes about one hour to load.  The response
time when navigating the categories is excellent, provided you are using
Apache + mod_perl.

The memory used during the load is around 70Mb during the conversion
and 10Mb for loading. 

In order to load dmoz.org data using Catalog you must follow the steps listed
below. This procedure assumes that you have created a database named @strong{dmoz}
and a catalog of named @strong{dmoz} within this database. A @strong{dmoz}
database has been created during the installation process. If you don't have it
create it with the following command:
@example
mysql -e "create database dmoz" 
@end example
If you have a database named dmoz, go to the URL
http://localhost/cgi-bin/DMOZ/dmoz?context=ccontrol_panel and a catalog named
@strong{dmoz} will automatically be created.

@itemize @bullet

@item
Load the content.rdf.gz file from http://dmoz.org/rdf.html.

@item
Run
@example
convert_dmoz -exclude '^/Adult' -what content content.rdf.gz
@end example
It prints a dot from time to time to show that it does not hang.
This script produces the following files:

@itemize @bullet
@item
 category.txt (table catalog_category_dmoz)
@item
 entry2category.txt (table catalog_entry2category_dmoz)
@item
 category2category.txt (table catalog_category2category_dmoz)
@item
 dmozrecords.txt (table dmozrecords)
@end itemize

@item
Load the files into the database using the following command
@example
convert_dmoz -load all ~/dmoz
@end example

@item
Click on @strong{browse} link in the Control Panel and check that
the catalog displays well. @strong{Warning} the first time you click
on @strong{browse} Catalog will rebuild some internal tables and it
will take some time to display. While working Catalog sends white
space characters to keep the connection busy and prevent timeouts.
These characters also tells you that Catalog is working and not hanging.
One character is printed for each category. If you have 200 000 categories
you should expect to download 200KB.

@item
Click on the @strong{count} link in the Control Panel to calculate
how many entries each category contains. It taks about the same time
to complete.
One character is printed for each category. If you have 200 000 categories
you should expect to download 200KB.

@end itemize

After a while, you will want to reload a new version of the dmoz.org
data. It can be done using the same commands. The problem is that while
you do that the catalog will be unavailable to the users. The data are
first removed and then populated. Catalog does not currently offer support
for user transparent reloading. Instead we suggest you follow these steps:

@itemize @bullet

@item
Load dmoz on another machine.

@item
Swap the MySQL database files when done.

@end itemize

@node Template files introduction, Catalog template files, External representation, Top
@chapter Template files introduction

@cindex template files 
@cindex template files definitions
The template files are the HTML files used by the cgi-bin to interact
with the user. Every interaction step is associated with the file name
of a template, the content of this file is read and analyzed by the 
cgi-bin and portions of it are replaced according to the action taken. 

@cindex simple template file

The simplest of all templates is the error template. It looks like this:

@cindex _MESSAGE_

@example
<title>Error</title>
<center><h3>
Error<p>
_MESSAGE_
</center></h3>
@end example

When an error occurs in a cgi-bin the error template is used. The
@strong{_MESSAGE_} tag is replaced by the actual error message and the
result is sent to the user.

@cindex location of template files
@cindex template files location
@cindex TEMPLATESDIR