view release on metacpan or  search on metacpan

examples/async-job-queue/SPEC.md  view on Meta::CPAN

Workers should simulate job processing with `Future::IO->sleep($delay)` rather
than CPU work. The point is event-loop concurrency, not parallel CPU execution.

The app should coordinate all tasks with futures, for example:

- producer future enqueues all jobs
- worker futures loop until they receive a stop sentinel
- heartbeat future loops until the processed count reaches the target
- main future waits for producer, workers, and heartbeat to finish

## Queue Semantics

Startup:

- delete `async-job-queue:jobs`
- delete `async-job-queue:processed`
- delete `async-job-queue:in-flight`
- push jobs `job-1` through `job-N`

Worker loop:

- `BLPOP async-job-queue:jobs 0`
- if the value is a stop sentinel, exit
- add the job to `async-job-queue:in-flight`
- print that the worker started the job
- `await Future::IO->sleep($delay)`
- remove the job from `async-job-queue:in-flight`
- increment `async-job-queue:processed`
- print that the worker finished the job

Shutdown:

- after all real jobs are processed, push one stop sentinel per worker
- workers consume sentinels and exit
- final summary prints elapsed time, processed count, and expected sequential
  time

The stop sentinel should be a value that cannot collide with generated job
names, for example `__async_job_queue_stop__`.

## Output Requirements

Output must be human-readable and timestamped relative to process start.

It should make queueing and async behavior obvious. Example shape:

```text
[ 0.00s] queued 10 jobs
[ 0.01s] worker-1 started job-1
[ 0.01s] worker-2 started job-2
[ 0.25s] heartbeat queue=8 in_flight=2 processed=0
[ 1.51s] worker-1 finished job-1
[ 1.51s] worker-1 started job-3
[ 1.51s] worker-2 finished job-2
[ 1.51s] worker-2 started job-4
[ 1.75s] heartbeat queue=6 in_flight=2 processed=2
[ 7.55s] done processed=10 workers=2 elapsed=7.55s sequential_about=15.00s
```

The README should explain that the heartbeat lines are the key proof: the
process continues running other async work while workers are blocked in Redis
or waiting on simulated work.

## CLI

Keep the first version simple, but allow the demo to be tweaked:

```text
examples/async-job-queue/app.pl [options]

Options:
  --jobs N       number of jobs to enqueue, default 10
  --workers N    number of workers, default 2
  --delay SEC    simulated seconds per job, default 1.5
  --help         show usage
```

Validation:

- `--jobs` must be a positive integer
- `--workers` must be a positive integer
- `--delay` must be a positive number

## Dependencies

Use only modules already required by the distribution or Perl core where
possible:

- `Async::Redis`
- `Future`
- `Future::AsyncAwait`
- `Future::IO`
- `Getopt::Long`
- `Time::HiRes`
- `FindBin`

Do not require PAGI, AnyEvent, IO::Async-specific setup, or a web server.

## Documentation Requirements

`examples/async-job-queue/README.md` should include:

- what the example demonstrates
- how to start Redis with the existing `examples/docker-compose.yml`
- how to run the app from the project root
- sample output
- why separate worker connections are used
- what to look for in the output

`examples/README.md` should add an `async-job-queue` section with a short
description and one run command.

## Test And Verification Plan

Manual smoke test:

```bash
perlbrew use perl-5.40.0@default
docker compose -f examples/docker-compose.yml up -d
REDIS_HOST=localhost perl examples/async-job-queue/app.pl --jobs 6 --workers 2 --delay 0.2
```