App-phoebe
view release on metacpan or search on metacpan
* another item
A line starting with ">", followed by a space and some text is a quote.
The monologue at the end is fantastic, with the city lights and the rain.
> I've seen things you people wouldn't believe.
# Editing the wiki
How do you edit a Phoebe wiki? You need to use a Titan-enabled client.
Titan is a companion protocol to Gemini: it allows clients to upload files to
Gemini sites, if servers allow this. On Phoebe, you can edit "raw" pages. That
is, at the bottom of a page you'll see a link to the "raw" page. If you follow
it, you'll see the page content as plain text. You can submit a changed version
of this text to the same URL using Titan. There is more information for
developers available on Community Wiki. [https://communitywiki.org/wiki/Titan](https://communitywiki.org/wiki/Titan)
Known clients:
This repository comes with a Perl script called `titan` to upload files.
[https://alexschroeder.ch/cgit/phoebe/plain/titan](https://alexschroeder.ch/cgit/phoebe/plain/titan)
_Gemini Write_ is an extension for the Emacs Gopher and Gemini client
_Elpher_. [https://alexschroeder.ch/cgit/gemini-write/](https://alexschroeder.ch/cgit/gemini-write/)
[https://thelambdalab.xyz/elpher/](https://thelambdalab.xyz/elpher/)
Gemini & Titan for Bash are two shell functions that allow you to download and
upload files. [https://alexschroeder.ch/cgit/gemini-titan/about/](https://alexschroeder.ch/cgit/gemini-titan/about/)
## Editing via the web
The Configuration section of the Phoebe space on _The Transjovian Council_ has
an example config on how to enable editing via the web.
- [https://transjovian.org:1965/phoebe/page/Configuration](https://transjovian.org:1965/phoebe/page/Configuration)
- [gemini://transjovian.org/phoebe/page/Configuration](gemini://transjovian.org/phoebe/page/Configuration)
# Installation
Using `cpan`:
cpan App::phoebe
Manual install:
perl Makefile.PL
make
make install
## Dependencies
If you are not using `cpan` or `cpanm` to install Phoebe, you'll need to
install the following dependencies:
- [Algorithm::Diff](https://metacpan.org/pod/Algorithm%3A%3ADiff), or `libalgorithm-diff-xs-perl`
- [File::ReadBackwards](https://metacpan.org/pod/File%3A%3AReadBackwards), or `libfile-readbackwards-perl`
- [File::Slurper](https://metacpan.org/pod/File%3A%3ASlurper), or `libfile-slurper-perl`
- [Mojolicious](https://metacpan.org/pod/Mojolicious), or `libmojolicious-perl`
- [IO::Socket::SSL](https://metacpan.org/pod/IO%3A%3ASocket%3A%3ASSL), or `libio-socket-ssl-perl`
- [Modern::Perl](https://metacpan.org/pod/Modern%3A%3APerl), or `libmodern-perl-perl`
- [URI::Escape](https://metacpan.org/pod/URI%3A%3AEscape), or `liburi-escape-xs-perl`
- [Net::IDN::Encode](https://metacpan.org/pod/Net%3A%3AIDN%3A%3AEncode), or `libnet-idn-encode-perl`
- [Encode::Locale](https://metacpan.org/pod/Encode%3A%3ALocale), or `libencode-locale-perl`
I'm going to be using `curl` and `openssl` in the ["Quickstart"](#quickstart) instructions,
so you'll need those tools as well. And finally, when people download their
data, the code calls `tar` (available from packages with the same name on
Debian derived systems).
The `update-readme.pl` script I use to generate `README.md` also requires some
libraries:
- [Pod::Markdown](https://metacpan.org/pod/Pod%3A%3AMarkdown), or `libpod-markdown-perl`
- [Text::Slugify](https://metacpan.org/pod/Text%3A%3ASlugify), which has no Debian package, apparently ð
## Quickstart
I'm going to assume that you're going to create a new user just to be safe.
sudo adduser --disabled-login --disabled-password phoebe
sudo su phoebe --shell=/bin/bash
cd
Now you're in your home directory, `/home/phoebe`. We're going to install
things right here.
cpan App::phoebe
Start Phoebe. It's going to prompt you for a hostname and create certificates
for you. If in doubt, answer `localhost`. The certificate and a private key are
stored in the `cert.pem` and `key.pem` files, using elliptic curves, valid for
five years, without password protection.
perl5/bin/phoebe
This starts the server in the foreground. If it aborts, see the
["Troubleshooting"](#troubleshooting) section below. If it runs, open a second terminal and test
it:
perl5/bin/gemini gemini://localhost/
You should see a Gemini page starting with the following:
20 text/gemini; charset=UTF-8
Welcome to Phoebe!
Success!! ð ðð
Let's create a new page using the Titan protocol, from the command line:
echo "Welcome to the wiki!" > test.txt
echo "Please be kind." >> test.txt
perl5/bin/titan --url=titan://localhost/raw/Welcome --token=hello test.txt
You should get a nice redirect message, with an appropriate date.
30 gemini://localhost:1965/page/Welcome
You can check the page, now (replacing the appropriate date):
User-agent: *
Disallow: /raw
Disallow: /html
Disallow: /diff
Disallow: /history
Disallow: /do/comment
Disallow: /do/changes
Disallow: /do/all/changes
Disallow: /do/all/latest/changes
Disallow: /do/rss
Disallow: /do/atom
Disallow: /do/all/atom
Disallow: /do/new
Disallow: /do/more
Disallow: /do/match
Disallow: /do/search
# allowing do/index!
Crawl-delay: 10
In fact, as long as you don't create a page called `robots` then this is what
gets served. I think it's a good enough way to start. If you're using spaces,
the `robots` pages of all the spaces are concatenated.
If you want to be more paranoid, create a page called `robots` and put this on
it:
User-agent: *
Disallow: /
Note that if you've created your own `robots` page, and you haven't decided to
disallow them all, then you also have to do the right thing for all your spaces,
if you use them at all.
## Configuration
See [App::Phoebe](https://metacpan.org/pod/App%3A%3APhoebe) for more information.
## Wiki Spaces
Wiki spaces are separate wikis managed by the same Phoebe server, on the
same machine, but with data stored in a different directory. If you used
`--wiki_space=alex` and `--wiki_space=berta`, for example, then you'd have
three wikis in total:
- `gemini://localhost/` is the main space that continues to be available
- `gemini://localhost/alex/` is the wiki space for Alex
- `gemini://localhost/berta/` is the wiki space for Berta
Note that all three spaces are still editable by anybody who knows any of the
[tokens](#security).
## Tokens per Wiki Space
Per default, there is simply one set of tokens which allows the editing of the
wiki, and all the wiki spaces you defined. If you want to give users a token
just for their space, you can do that, too. Doing this is starting to strain the
command line interface, however, and therefore the following illustrates how to
do more advanced configuration using the config file:
package App::Phoebe;
use Modern::Perl;
our ($server);
$server->{wiki_space_token}->{alex} = ["*secret*"];
The code above sets up the `wiki_space_token` property. It's a hash reference
where keys are existing wiki spaces and values are array references listing the
valid tokens for that space (in addition to the global tokens that you can set
up using `--wiki_token` which defaults to the token "hello"). Thus, the above
code sets up the token `*secret*` for the `alex` wiki space.
You can use the config file to change the values of other properties as well,
even if these properties are set via the command line.
package App::Phoebe;
use Modern::Perl;
our ($server);
$server->{wiki_token} = [];
This code simply deactivates the token list. No more tokens!
## Virtual Hosting
Sometimes you want have a machine reachable under different domain names and you
want each domain name to have their own wiki space, automatically. You can do
this by using multiple `--host` options.
Here's a simple, stand-alone setup that will work on your local machine. These
are usually reachable using the IPv4 `127.0.0.1` or the name `localhost`. The
following command tells Phoebe to serve both `127.0.0.1` and `localhost`
(the default is to just serve `localhost`).
phoebe --host=127.0.0.1 --host=localhost
Visit both at [gemini://localhost/](gemini://localhost/) and [gemini://127.0.0.1/](gemini://127.0.0.1/), and create a
new page in each one, then examine the data directory `wiki`. You'll see both
`wiki/localhost` and `wiki/127.0.0.1`.
If you're using more wiki spaces, you need to prefix them with the respective
hostname if you use more than one:
phoebe --host=127.0.0.1 --host=localhost \
--wiki_space=127.0.0.1/alex --wiki_space=localhost/berta
In this situation, you can visit [gemini://127.0.0.1/](gemini://127.0.0.1/),
[gemini://127.0.0.1/alex/](gemini://127.0.0.1/alex/), [gemini://localhost/](gemini://localhost/), and
[gemini://localhost/berta/](gemini://localhost/berta/), and they will all be different.
If this is confusing, remember that not using virtual hosting and not using
spaces is fine, too. ð
## Multiple Certificates
If you're using virtual hosting as discussed above, you have two options: you
can use one certificate for all your hostnames, or you can use different
certificates for the hosts. If you want to use just one certificate for all your
hosts, you don't need to do anything else. If you want to use different
certificates for different hosts, you have to specify them all on the command
line. Generally speaking, use `--host` to specifiy one or more hosts, followed
by both `--cert_file` and `--key_file` to specifiy the certificate and key to
use for the hosts.
For example:
phoebe --host=transjovian.org \
--cert_file=/var/lib/dehydrated/certs/transjovian.org/cert.pem \
--key_file=/var/lib/dehydrated/certs/transjovian.org/privkey.pem \
--host=alexschroeder.ch \
--cert_file=/var/lib/dehydrated/certs/alexschroeder.ch/cert.pem \
--key_file=/var/lib/dehydrated/certs/alexschroeder.ch/privkey.pem
# See also
As you might have guessed, the system is easy to tinker with, if you know some
Perl. The Transjovian Council has a wiki space dedicated to Phoebe, and it
includes a section with more configuration examples.
./gemini "$url" > $(basename "$url")
done
In the shell script above, the first call to gemini gets the page with all the
links, grep then filters for the links to thumbnails, extract the URL using cut
(assuming a space between "=>" and the URL), and download each URL, and save the
output in the filename indicated by the URL.
## Client Certificates
You can provide a certificate and a key file:
gemini --cert_file=cert.pem --key_file=key.pem \
gemini://campaignwiki.org/play/ijirait
# Gemini
All `gemini-chat` does is repeatedly post stuff to a Gemini URL. For example,
assume that there is a Phoebe wiki with chat enabled via [App::Phoebe::Chat](https://metacpan.org/pod/App%3A%3APhoebe%3A%3AChat).
First, you connect to the _listen_ URL using `gemini`:
gemini --cert_file=cert.pem --key_file=key.pem \
gemini://localhost/do/chat/listen
Then you connect the chat client to the _say_ URL using `gemini-chat`:
gemini-chat --cert=cert.pem --key=key.pem \
gemini://transjovian.org/do/chat/say
To generate your client certificate for 100 days and using âAlexâ as your common
name:
openssl req -new -x509 -newkey ec -subj "/CN=Alex" \
-pkeyopt ec_paramgen_curve:prime256v1 -days 100 \
-nodes -out cert.pem -keyout key.pem
# Ijirait
This is a command-line client for Ijirait, a Gemini-based MUSH that can be run
by Phoebe. See [App::Phoebe::Ijirait](https://metacpan.org/pod/App%3A%3APhoebe%3A%3AIjirait).
First, generate your client certificate for as many or as few days as you like:
openssl req -new -x509 -newkey ec -subj "/CN=Alex" \
-pkeyopt ec_paramgen_curve:prime256v1 -days 100 \
-nodes -out cert.pem -keyout key.pem
Then start this program to play:
ijirait --cert=cert.pem --key=key.pem \
--url=gemini://campaignwiki.org/play/ijirait
You can also use it to stream, i.e. get notified of events in real time:
ijirait --cert=cert.pem --key=key.pem --stream \
--url=gemini://campaignwiki.org/play/ijirait/stream
Here are the Debian package names to satisfy the dependencies. Use `cpan` or
`cpanm` to install them.
- [Modern::Perl](https://metacpan.org/pod/Modern%3A%3APerl) from `libmodern-perl-perl`
- [Mojo::IOLoop](https://metacpan.org/pod/Mojo%3A%3AIOLoop) from `libmojolicious-perl`
- [Term::ReadLine::Gnu](https://metacpan.org/pod/Term%3A%3AReadLine%3A%3AGnu) from `libterm-readline-gnu-perl`
- [URI::Escape::XS](https://metacpan.org/pod/URI%3A%3AEscape%3A%3AXS) from `liburi-escape-xs-perl`
- [Encode::Locale](https://metacpan.org/pod/Encode%3A%3ALocale) from `libencode-locale-perl`
- [Text::Wrapper](https://metacpan.org/pod/Text%3A%3AWrapper) from `libtext-wrapper-perl`
# phoebe-ctl
This script helps you maintain your Phoebe installation.
- **--wiki\_dir=**_DIR_
This the wiki data directory to use; the default is either the value of the
`GEMINI_WIKI_DATA_DIR` environment variable, or the `./wiki` subdirectory. Use
it to specify a space, too.
- **--log=**_NUMBER_
This is the log level to use. 1 only prints errors; 2 also prints warnings (this
is the default); 3 prints any kind of information; 4 prints all sorts of info
the developer wanted to see as they were fixing bugs.
## Commands
**phoebe-ctl help**
This is what you're reading right now.
**phoebe-ctl update-changes**
This command looks at all the pages in the `page` directory and generates new
entries for your changes log into `changes.log`.
**phoebe-ctl erase-page**
This command removes pages from the `page` directory, removes all the kept
revisions in the `keep` directory, and all the mentions in the `change.log`.
Use this if spammers and vandals created page names you want to eliminate.
**phoebe-ctl html-export** \[**--source=**`subdirectory` ...\]
\[**--target=**`directory`\] \[**--no-extension**\]
This command converts all the pages in the subdirectories provided to HTML and
writes the HTML files into the target directory. The subdirectories must exist
inside your wiki data directory. The default wiki data directory is `wiki` and
the default source subdirectory is undefined, so the actual files to be
processed are `wiki/page/*.gmi`; if you're using virtual hosting, the
subdirectory might be your host name; if you're using spaces, those need to be
appended as well.
Example:
phoebe-ctl html-export --wiki_dir=/home/alex/phoebe \
--source=transjovian.org \
--source=transjovian.org/phoebe \
--source=transjovian.org/gemini \
--source=transjovian.org/titan \
--target=/home/alex/transjovian.org
This will create HTML files in `/home/alex/transjovian.org`,
Or from a pipe:
echo "This is my test." \
| titan --url=titan://transjovian.org/test/raw/testing --token=hello
# App::Phoebe
This module contains the core of the Phoebe wiki. Import functions and variables
from this module to write extensions, or to run it some other way. Usually,
`script/phoebe` is used to start a Phoebe server. This is why all the
documentation regarding server startup can be found there.
This section describes some hooks you can use to customize your wiki using the
`config` file, or using a Perl file (ending in `*.pl` or `*.pm`) in the
`conf.d` directory. Once you're happy with the changes you've made, restart the
server, or send a SIGHUP if you know the PID.
Here are the ways you can hook into Phoebe code:
`@extensions` is a list of code references allowing you to handle additional
URLs; return 1 if you handle a URL; each code reference gets called with $stream
([Mojo::IOLoop::Stream](https://metacpan.org/pod/Mojo%3A%3AIOLoop%3A%3AStream)), the first line of the request (a Gemini URL, a Gopher
selector, a finger user, a HTTP request line), a hash reference for the headers
(in the case of HTTP requests), and a buffer of bytes (e.g. for Titan or HTTP
PUT or POST requests).
`@main_menu` adds more lines to the main menu, possibly links that aren't
simply links to existing pages.
`@footer` is a list of code references allowing you to add things like licenses
or contact information to every page; each code reference gets called with
$stream ([Mojo::IOLoop::Stream](https://metacpan.org/pod/Mojo%3A%3AIOLoop%3A%3AStream)), $host, $space, $id, $revision, and $format
('gemini' or 'html') used to serve the page; return a gemtext string to append
at the end; the alternative is to overwrite the `footer` or `html_footer` subs
â the default implementation for Gemini adds History, Raw text and HTML link,
and `@footer` to the bottom of every page; the default implementation for HTTP
just adds `@footer` to the bottom of every page.
If you do hook into Phoebe's code, you probably want to make use of the
following variables:
`$server` stores the command line options provided by the user.
`$log` is how you log things.
A very simple example to add a contact mail at the bottom of every page; this
works for both Gemini and the web:
# tested by t/example-footer.t
use App::Phoebe::Web;
use App::Phoebe qw(@footer);
push(@footer, sub { '=> mailto:alex@alexschroeder.ch Mail' });
This prints a very simply footer instead of the usual footer for Gemini, as the
`footer` function is redefined. At the same time, the `@footer` array is still
used for the web:
# tested by t/example-footer2.t
package App::Phoebe;
use App::Phoebe::Web;
use Modern::Perl;
our (@footer); # HTML only
push(@footer, sub { '=> https://alexschroeder.ch/wiki/Contact Contact' });
# footer sub is Gemini only
no warnings qw(redefine);
sub footer {
return "\n" . 'â' x 10 . "\n" . '=> mailto:alex@alexschroeder.ch Mail';
}
This example shows you how to add a new route (a new path served by the wiki).
Instead of just writing "Test" to the page, you could of course run arbitrary
Perl code.
# tested by t/example-route.t
our @config = (<<'EOT');
use App::Phoebe qw(@extensions @main_menu port host_regex success);
use Modern::Perl;
push(@main_menu, "=> /do/test Test");
push(@extensions, \&serve_test);
sub serve_test {
my $stream = shift;
my $url = shift;
my $hosts = host_regex();
my $port = port($stream);
if ($url =~ m!^gemini://($hosts):$port/do/test$!) {
success($stream, 'text/plain; charset=UTF-8');
$stream->write("Test\n");
return 1;
}
return;
}
EOT
This example also shows how to redefine existing code in your config file
without the warning "Subroutine ⦠redefined".
Here's a more elaborate example to add a new action the main menu and a handler
for it, for Gemini only:
# tested by t/example-new-action.t
package App::Phoebe;
use Modern::Perl;
our (@extensions, @main_menu);
push(@main_menu, "=> gemini://localhost/do/test Test");
push(@extensions, \&serve_test);
sub serve_test {
my $stream = shift;
my $url = shift;
my $headers = shift;
my $host = host_regex();
my $port = port($stream);
if ($url =~ m!^gemini://($host)(?::$port)?/do/test$!) {
result($stream, "20", "text/plain");
$stream->write("Test\n");
return 1;
}
return;
}
1;
# App::Phoebe::BlockFediverse
This extension blocks the Fediverse user agent from your website (Mastodon,
Friendica, Pleroma). The reason is this: when these sites federate a status
linking to your site, each instance will fetch a preview, so your site will get
hit by hundreds of requests from all over the Internet. Blocking them helps us
weather the storm.
There is no configuration. Simply add it to your `config` file:
use App::Phoebe::BlockFediverse;
Sure, we could also think of better caching and all that. I hate the fact that
other developers are forcing us to build âsoftware that scalesâ â I hate how
they think that I have nothing better to do than think about blocking and
caching. Phoebe is software for the Smolnet, not for people that keep thinking
about scaling.
The solution implemented is this: if the user agent of a HTTP request matches
the regular expression, quit immediatly. The result:
$ curl --header "User-Agent: Pleroma" https://transjovian.org:1965/
Blocking Fediverse previews
Yeah, we could respond with a error, but fediverse developers arenât interested
in a new architecture for this problem. They think the issue has been solved.
See [#4486](https://github.com/tootsuite/mastodon/issues/4486), âMastodon can be
used as a DDOS tool.â
# App::Phoebe::Chat
For every wiki space, this creates a Gemini-based chat room. Every chat client
needs two URLs, the "listen" and the "say" URL.
The _Listen URL_ is where you need to _stream_: as people say things in the
room, these messages get streamed in one endless Gemini document. You might have
to set an appropriate timeout period for your connection for this to work. 1h,
perhaps?
The URL will look something like this:
`gemini://localhost/do/chat/listen` or
`gemini://localhost/space/do/chat/listen`
our $galleries_host = "alexschroeder.ch";
use App::Phoebe::Galleries;
# App::Phoebe::Gopher
This extension serves your Gemini pages via Gopher and generates a few automatic
pages for you, such as the main page.
To configure, you need to specify the Gopher port(s) in your Phoebe `config` file.
The default port is 70. This is a priviledge port. Thus, you either need to
grant Perl the permission to listen on a priviledged port, or you need to run
Phoebe as a super user. Both are potential security risk, but the first option
is much less of a problem, I think.
If you want to try this, run the following as root:
setcap 'cap_net_bind_service=+ep' $(which perl)
Verify it:
getcap $(which perl)
If you want to undo this:
setcap -r $(which perl)
The alternative is to use a port number above 1024.
If you don't do any of the above, you'll get a permission error on startup:
"Mojo::Reactor::Poll: Timer failed: Can't create listen socket: Permission
deniedâ¦"
If you are virtual hosting note that the Gopher protocol is incapable of doing
that: the server does not know what hostname the client used to look up the IP
number it eventually contacted. This works for HTTP and Gemini because HTTP/1.0
and later added a Host header to pass this information along, and because Gemini
uses a URL including a hostname in its request. It does not work for Gopher.
This is why you need to specify the hostname via `$gopher_host`.
You can set the normal Gopher via `$gopher_port` and the encrypted Gopher ports
via `$gophers_port` (note the extra s). The values either be a single port, or
an array of ports. See the example below.
In this example we first switch to the package namespace, set some variables,
and then we _use_ the package. At this point the ports are specified and the
server processes it starts go up, one for ever IP number serving the hostname.
package App::Phoebe::Gopher;
our $gopher_host = "alexschroeder.ch";
our $gopher_port = [70,79]; # listen on the finger port as well
our $gophers_port = 7443; # listen on port 7443 using TLS
our $gopher_main_page = "Gopher_Welcome";
use App::Phoebe::Gopher;
Note the `finger` port in the example. This works, but it's awkward since you
have to finger `page/alex` instead of `alex`. In order to make that work, we
need some more code.
package App::Phoebe::Gopher;
use App::Phoebe qw(@extensions port $log);
use Modern::Perl;
our $gopher_host = "alexschroeder.ch";
our $gopher_port = [70,79]; # listen on the finger port as well
our $gophers_port = 7443; # listen on port 7443 using TLS
our $gopher_main_page = "Gopher_Welcome";
our @extensions;
push(@extensions, \&finger);
sub finger {
my $stream = shift;
my $selector = shift;
my $port = port($stream);
if ($port == 79 and $selector =~ m!^[^/]+$!) {
$log->debug("Serving $selector via finger");
gopher_serve_page($stream, $gopher_host, undef, decode_utf8(uri_unescape($selector)));
return 1;
}
return 0;
}
use App::Phoebe::Gopher;
# App::Phoebe::HeapDump
Perhaps you find yourself in a desperate situation: your server is leaking
memory and you don't know where. This extension provides a way to use
[Devel::MAT::Dumper](https://metacpan.org/pod/Devel%3A%3AMAT%3A%3ADumper) by allowing users identified with a known fingerprint of
their client certificate to initiate a dump.
You must set the fingerprints in your `config` file.
package App::Phoebe;
our @known_fingerprints = qw(
sha256$fce75346ccbcf0da647e887271c3d3666ef8c7b181f2a3b22e976ddc8fa38401);
use App::Phoebe::HeapDump;
Once have restarted the server, [gemini://localhost/do/heap-dump](gemini://localhost/do/heap-dump) will write a
heap dump to its wiki data directory. See [Devel::MAT::UserGuide](https://metacpan.org/pod/Devel%3A%3AMAT%3A%3AUserGuide) for more.
# App::Phoebe::Iapetus
This allows known editors to upload files and pages using the Iapetus protocol.
See [Iapetus documentation](https://codeberg.org/oppenlab/iapetus).
In order to be a known editor, you need to set `@known_fingerprints` in your
`config` file. Hereâs an example:
package App::Phoebe;
our @known_fingerprints;
@known_fingerprints = qw(
sha256$fce75346ccbcf0da647e887271c3d3666ef8c7b181f2a3b22e976ddc8fa38401
sha256$54c0b95dd56aebac1432a3665107d3aec0d4e28fef905020ed6762db49e84ee1);
use App::Phoebe::Iapetus;
The way to do it is to run the following, assuming the certificate is named
`client-cert.pem`:
openssl x509 -in client-cert.pem -noout -sha256 -fingerprint \
| sed -e 's/://g' -e 's/SHA256 Fingerprint=/sha256$/' \
| tr [:upper:] [:lower:]
This should give you the fingerprint in the correct format to add to the list
above.
( run in 0.594 second using v1.01-cache-2.11-cpan-d7a12ab2c7f )