view release on metacpan or  search on metacpan

examples/async-job-queue/SPEC.md  view on Meta::CPAN

# Async Job Queue Example Specification

## Goal

Create a small CLI example that makes Async::Redis concurrency visible without
requiring a web framework or extra UI.

The example should prove three things:

- Redis list commands can model a simple job queue.
- multiple async workers can process jobs concurrently in one Perl process.
- a heartbeat task keeps running while workers wait in `BLPOP` and while other
  tasks are sleeping or doing Redis I/O.

## Location

- executable: `examples/async-job-queue/app.pl`
- documentation: `examples/async-job-queue/README.md`
- examples index update: `examples/README.md`

## User Story

As a new Async::Redis user, I can run one command and see jobs being queued,
claimed by workers, processed concurrently, and reported by a heartbeat while
the process remains responsive.

## Runtime Behavior

The app should:

- connect to Redis using `REDIS_HOST` and `REDIS_PORT`, defaulting to
  `localhost` and `6379`
- clean only its own demo keys at startup
- enqueue a burst of jobs immediately so the queue depth is visible
- start a fixed number of worker coroutines
- start one heartbeat coroutine
- exit automatically after all jobs are processed
- clean up its worker unblock sentinels before exit

Default values:

- jobs: `10`
- workers: `2`
- simulated work delay: `1.5` seconds
- heartbeat interval: `0.25` seconds
- queue key: `async-job-queue:jobs`
- processed counter key: `async-job-queue:processed`
- in-flight set key: `async-job-queue:in-flight`

## Async Design

Use separate Redis connections for roles that can block independently:

- one producer/controller connection
- one stats connection
- one Redis connection per worker

Workers must use `BLPOP` so the example demonstrates a real Redis blocking
operation. Each worker connection may sit inside `BLPOP` without preventing the
producer, heartbeat, or other workers from making progress.

Workers should simulate job processing with `Future::IO->sleep($delay)` rather
than CPU work. The point is event-loop concurrency, not parallel CPU execution.

The app should coordinate all tasks with futures, for example:

- producer future enqueues all jobs
- worker futures loop until they receive a stop sentinel
- heartbeat future loops until the processed count reaches the target
- main future waits for producer, workers, and heartbeat to finish

## Queue Semantics

Startup:

- delete `async-job-queue:jobs`
- delete `async-job-queue:processed`
- delete `async-job-queue:in-flight`
- push jobs `job-1` through `job-N`

Worker loop:

- `BLPOP async-job-queue:jobs 0`
- if the value is a stop sentinel, exit
- add the job to `async-job-queue:in-flight`
- print that the worker started the job
- `await Future::IO->sleep($delay)`
- remove the job from `async-job-queue:in-flight`
- increment `async-job-queue:processed`
- print that the worker finished the job

Shutdown:

- after all real jobs are processed, push one stop sentinel per worker
- workers consume sentinels and exit
- final summary prints elapsed time, processed count, and expected sequential