view release on metacpan or  search on metacpan

doc/integration-test-plan.md  view on Meta::CPAN

The repo checkout is not mounted into the container as the app under test.
Only the host-built tarball is mounted into the blank container.
The normal `prove -lr t` and explicit `Devel::Cover` gates are where the
Developer Dashboard distribution tests run in full. The later blank-container
tarball install is now an installation-verification gate, so it uses
`cpanm --notest` to verify packaged dependency resolution and installed
runtime behavior without rerunning the same distribution test suite a second
time. The Windows guest smoke follows the same rule for the tarball install
step, and the optional bootstrap path passes the tarball through the literal
`DD_INSTALL_CPAN_TARGET` environment variable so `install.ps1` still lets
`cpanm --notest` resolve the exact target literally. Outside that override,
the streamed Windows bootstrap defaults to cloning the GitHub `master`
checkout into a temporary local tree instead of relying on a potentially stale
CPAN release.

## Test Data

The integration run creates:

- a temporary home directory under `/tmp`
- a fake project root under `/tmp/fake-project`
- a fake project `./.developer-dashboard` tree with `dashboards`, `config`, and `cli` directories
- a saved page named `sample`
- a saved bookmark page named `project-home`
- a saved stream regression bookmark page
- shared nav bookmark pages under `nav/*.tt`
- a helper user for explicit add/remove testing
- a second helper user for browser login/logout cleanup testing
- a temporary Compose project under `/tmp`

## Execution Flow

1. Build the distribution tarball on the host with `dzil build`.
2. Run `prove -lv t/44-smart-router-two-stage.t` against that freshly built tarball so the extracted-dashboard smart-router contract is verified at the post-build stage. That guard retries one transient `cpanm` fetch or unpack failure inside its Dock...
3. Start the blank container with only that host-built tarball mounted into it.
4. Copy the mounted tarball to a versioned local path inside the container and
   install that staged tarball with `cpanm --notest`. The staged filename must keep the
   concrete `Developer-Dashboard-X.XX.tar.gz` version so `cpanm` cannot drift
   into a CPAN lookup because the bind-mounted filename is generic.
5. Create the fake-project `./.developer-dashboard` tree only after that install step succeeds so the tarball's own tests still run against a clean runtime.
6. Extract the same tarball inside the container for the rest of the installed-command checks.
7. Verify the installed CLI responds to `dashboard help`.
8. Verify bare `dashboard` returns usage output.
9. Verify `dashboard version` reports the installed runtime version.
10. Create a fake project root with a local `./.developer-dashboard` runtime tree.
11. Exercise `dashboard cpan DBD::Driver` inside the fake project and confirm the requested driver plus `DBI` are installed into `./.developer-dashboard/local` and recorded in `./.developer-dashboard/cpanfile`.
12. Seed a user-provided fake-project `./.developer-dashboard/cli/update` command plus `update.d` hooks in the clean container, run `dashboard update`, and confirm the normal top-level command-hook pipeline completes, including later-hook reads throu...
13. Exercise path, prompt, shell, encode/decode, and indicator commands.
14. Exercise collector write/run/read/start/restart/stop flows, including fake-project config collector definitions, TT-backed collector indicator icons rendered from collector stdout JSON, `dashboard collector log`, `dashboard collector log <name>`,...
15. Restart the installed runtime with one intentionally broken Perl config collector and one healthy config collector, then verify the broken collector reports an error without stopping the healthy collector or its green indicator state, even when p...
16. Kill one managed collector loop after startup, confirm the watchdog restarts it automatically, and verify `dashboard collector status <name>` records watchdog restart counters/timestamps. Kill it repeatedly until the watchdog limit is exceeded, t...
17. Exercise page create/save/show/encode/decode/render/source flows inside the fake bookmark directory.
18. Exercise builtin action execution.
19. For Windows-targeted changes, run `integration/windows/run-strawberry-smoke.ps1 -UseInstallBootstrap -BootstrapScript <checkout install.ps1>` so the guest validates the same streamed `Invoke-Expression` bootstrap shape that operators use with `ir...
20. Exercise docker compose dry-run resolution against a temporary project.
21. Start the installed web service.
22. Confirm exact-loopback access reaches the editor page in Chromium.
23. Confirm the browser can render a saved fake-project bookmark page from the fake project bookmark directory.
24. Confirm the browser inserts sorted rendered `nav/*.tt` bookmark fragments between the top chrome and the main page body.
25. Confirm the browser top-right status strip shows configured collector icons, not collector names, that UTF-8 icons such as `🐳` and `💰` are visibly rendered, and that renamed collectors no longer leave stale managed indicators behind.
26. Confirm an installed saved bookmark page can declare `var endpoints = {};`, then use `fetch_value()` and `stream_value()` from `$(document).ready(...)` against saved `/ajax/<file>` routes without inline-script ordering failures or browser console...
27. Confirm an installed long-running saved `/ajax/<file>` route starts streaming the first output chunks promptly instead of buffering until the worker exits.
28. Confirm an installed skill page that ships `config/routes.json` emits the declared canonical custom ajax path, that the custom path resolves, that the smart `/ajax/<repo-name>/...` route still resolves for the same handler, and that a route-level...
29. Confirm non-loopback self-access returns `401` with an empty body and without a login form before any helper user exists in the active runtime.
30. Add a helper user for the outsider browser flow, then confirm non-loopback self-access reaches the helper login page in Chromium.
31. Log in as a helper through the HTTP helper flow.
32. Confirm helper page chrome shows `Logout`.
33. Log out and confirm the helper account is removed.
34. Restart the installed runtime from the extracted tarball tree and confirm the web service comes back.
35. Stop the runtime and confirm the web service is gone.

## Expected Results

- every covered command exits successfully except bare `dashboard`, which should
  return usage with a non-zero status
- `dashboard version` reports the installed release version
- `dashboard init` creates starter state without requiring manual setup
- `dashboard update` succeeds in the container from a user-provided fake-project `./.developer-dashboard/cli/update` command through the normal command-hook path
- the installed `dashboard` binary works without `perl -Ilib`
- the fake project's `./.developer-dashboard` tree becomes the active local runtime root with the home tree as fallback
- layered root-to-leaf `.env` and `.env.pl` files override in order, skill-local env files load only for skill execution paths, nested skill commands expand `foo -> foo.bar -> foo.bar.zzz` env files in order while preserving overwritten parent keys u...
- skill dependency installs follow `aptfile -> apkfile -> dnfile -> wingetfile -> brewfile -> package.json -> requirements.txt -> cpanfile -> cpanfile.local -> Makefile -> ddfile -> ddfile.local`, with `aptfile`, `apkfile`, and `dnfile` probing each ...
- long-running skill dependency steps keep the main epic checklist visible while streaming a Docker-style rolling ten-line detail window under the active task, collapse those detail lines when the task succeeds, and leave the captured detail visible ...
- explicit `dashboard skills install --ddfile` runs process `ddfile` first into the active layered skills root and then `ddfile.local` into the current directory's nested `./skills/` tree
- explicit `dashboard skills install <source> ...` runs can install one or more sources in command-line order, append each exact source to the home root `~/.developer-dashboard/ddfile` without duplicating existing non-comment entries, `dashboard skil...
- when the home runtime already has `.gitignore` or compatibility `.gitiignore`, explicit skill installs append `skills/<repo-name>/` without duplicates so cloned skill trees stay ignored by runtime Git checkouts
- `dashboard skill` is covered as the singular alias for `dashboard skills` management commands while installed command execution remains on the dotted `dashboard <skill>.<command>` path
- streamed `install.sh` runs such as `curl ... | sh` succeed without a local checkout by falling back to embedded `aptfile`, `apkfile`, `dnfile`, and `brewfile` manifests, shipping `tmux` in those bootstrap package sets because `dashboard workspace` ...
- the old-system-Perl Alpine rescue path keeps the locally bootstrapped `perlbrew` and `patchperl` tools on the private `~/perl5/lib/perl5` include path so `curl ... | sh` can still build `perl-5.38.5` instead of dying with missing `App::perlbrew` or...
- Debian-family streamed bootstrap also copes with third-party `nodejs` repositories that conflict with the distro `npm` package by installing `nodejs` first, checking whether `npm` and `npx` are already present, and only then attempting the distro `...
- Alpine streamed bootstrap installs the repo-root `apkfile` package set through `apk add --no-cache` and then proves the same post-install shell finish line as Debian-family hosts
- Debian-family streamed bootstrap uses `perlbrew --notest install perl-5.38.5` for the old-system-Perl rescue path so blank-machine bootstrap does not fail on upstream Perl core test noise before Developer Dashboard itself is installed
- `install.sh` prints a full progress board before it changes the system, then emits only per-step transitions instead of redrawing the whole board, explains any upcoming `sudo` prompt as an operating-system package-manager password request before th...
- blank macOS streamed bootstrap now also covers the no-Homebrew starting state, proving `install.sh` bootstraps Homebrew first, updates `PATH` from the discovered Homebrew prefix in the same run, and only then installs the repo `brewfile` package se...
- `dashboard workspace` tmux sessions move prompt indicators into the first row of a session-local two-line bottom tmux status block, keep the normal indexed session/window row underneath it, keep the inline prompt free of duplicated indicators even ...
- nested installed skill nav trees such as `skills/ho/skills/coverage/dashboards/nav/index.tt` render on the nested skill route itself and also join the shared nav strip above normal saved `/app/<page>` routes
- the staged home-runtime `shell` helper itself must emit that tmux-aware bootstrap after install, not just the repo checkout `bin/dashboard shell ...` path
- a broken config Perl collector reports an error without stopping other configured collectors
- a healthy config collector still reports `ok` and stays green in `dashboard indicator list`, `dashboard ps1`, and `/system/status`, without being clobbered back to `missing` by concurrent config-sync refreshes
- a killed managed collector loop is restarted automatically by the watchdog, and repeated crash loops eventually surface `watchdog_attention_required` in `dashboard collector status <name>` instead of going silent
- a live managed collector loop that stops updating its status or completion timestamps is treated as stalled, recycled automatically by the watchdog, and reported explicitly in `dashboard collector status <name>` instead of sitting silent forever
- `dashboard collector log` prints aggregated collector transcripts, `dashboard collector log <name>` prints the named collector transcript, and configured collectors that have not run yet report an explicit no-log message instead of blank output
- TT-backed collector icons render from stdout JSON and stay rendered through later config-sync reads instead of reverting to raw `[% ... %]` text
- the web service serves the root editor on `127.0.0.1:7890`
- the browser can load both the editor and a saved fake-project bookmark page from the fake project bookmark directory
- the browser sees sorted shared `nav/*.tt` fragments above the main page body on that fake-project bookmark page
- the browser top-right status strip shows configured collector icons and does not leave stale renamed collector indicators behind
- nested `DD-OOP-LAYERS` collector prompts do not let a child-layer placeholder `missing` state override a healthy inherited parent-layer collector indicator when the child config adds no collector override
- under `DD-OOP-LAYERS`, `dashboard path add` writes only the new child-layer alias delta into the deepest child `config/config.json` instead of copying inherited parent config domains into that file
- bookmark pages can use `fetch_value()`, `stream_value()`, and `stream_data()` helpers against saved `/ajax/...` endpoints on first render
- the installed `/ajax/<file>` route streams early output chunks promptly enough to prove browser-visible progress instead of silent buffering
- skill pages that ship `config/routes.json` emit their declared canonical custom ajax paths, while the smart `/ajax/<repo-name>/...` route still works as the parent compatibility resolver and custom paths stay fallback-only before a normal `404`
- layered `config/api.json` files under both runtime roots and installed skills can authorize selected saved `/ajax/...` routes for machine callers, those callers must present matching `X-DD-API-Key` and `X-DD-API-Secret` headers, helper-session auth...
- the built-in `dashboard api` command can list that effective merged registry, hash raw secrets from `--secret` or `--maybe-secret` before saving them, add and remove exact saved ajax routes, and write only to the deepest writable `config/api.json` ...
- non-loopback access produces `401` with an empty body and without a login page until a helper user exists in the active runtime
- under `dashboard serve --ssl`, plain `http://HOST:PORT/...` requests on the public listener return a same-port `307` redirect to `https://HOST:PORT/...`, the generated cert advertises SAN coverage for `localhost`, `127.0.0.1`, and `::1`, and a brow...
- after a helper user exists, non-loopback access produces the helper login page
- helper logout removes both the helper session and the helper account
- `dashboard stop` leaves no active listener on port `7890`
- `dashboard stop` and `dashboard restart` still control the real serving pid
  when the web process has renamed itself into a `starman master` listener
  shape, so container lifecycle checks stay attached to the active listener
- interactive `dashboard stop` and `dashboard restart` runs print the full lifecycle task board on `stderr` before work begins, so managed shutdown and startup waits stay visible instead of looking hung
- `dashboard stop` and `dashboard restart` default to a final terminal table summary, while `-o json` keeps the machine-readable payload
- `dashboard stop web`, `dashboard stop collector`, `dashboard stop collector <name>`, `dashboard restart web`, `dashboard restart collector`, `dashboard restart collector <name>`, `dashboard log`, `dashboard logs`, `dashboard log web`, `dashboard lo...
- runtime stop/restart behavior still works when listener ownership must be
  discovered through `/proc` instead of `ss`
- Linux host lifecycle runs ignore web and collector pids that belong to a
  different pid namespace, so a host-side runtime does not kill or adopt a
  sibling Docker runtime during `dashboard stop` or `dashboard restart`
- `dashboard restart` also succeeds when a listener pid survives the first stop
  sweep and must be discovered by a late port re-probe
- `dashboard restart` only reports success after the replacement runtime still
  has a live managed pid and an accepting listener on the requested port, and
  after that ready state survives a short confirmation window instead of
  trusting an acknowledged pid that dies immediately afterwards
- runtime shutdown uses numeric POSIX signals for managed stop/restart paths so
  Alpine/iSH Perl builds that reject named signal strings still stop web and
  collector processes correctly

## Optional macOS Brewfile Verification

End-to-end `brewfile` verification on macOS is optional manual coverage, not a
required release gate. Use a real macOS host or a disposable macOS guest only
when you need to investigate a Homebrew-specific regression.

One practical route is the `dockur/macos` project:
https://github.com/dockur/macos

The upstream README documents a compose flow using `dockurr/macos`,
`/dev/kvm`, `/dev/net/tun`, `NET_ADMIN`, and the web installer on port `8006`.
Once the guest is installed and reachable, copy the built tarball in, install
Developer Dashboard with `cpanm`, create a skill that ships a `brewfile`, and
confirm `dashboard skills install <skill>` prints the requested Homebrew
packages before running `brew install ...`. When the fixture directory also
contains `ddfile` and `ddfile.local`, run `dashboard skills install --ddfile`
from that directory and confirm the global entries land under the active
layered skills root while the local entries land under `./skills/`.

## Out Of Scope

These are not treated as failures for this blank-environment run:

- outbound integrations not implemented by the current core
- actual privileged Docker daemon execution inside the container

The docker command family is validated through `--dry-run`, which is enough to
prove that the installed CLI resolves the compose stack correctly in a clean
environment.