Recovery catalog format 2

Implemented and tested locally; rollout and a live restore rehearsal are pending. Upgrade the destination's HostSSH binary and engine scripts before producing new packs. The source checks its own binary capability; it cannot establish the destination's installed version.

integrity.json remains the single file-size/SHA-256 catalog. Format 2 adds a random artifact ID, capture time, selected component roots, required adapter identifiers, engine/version fields, consistency scope, and verification level. The existing manifest.json remains descriptive capture metadata and is hashed as payload.

The first adapter is fleet-files-v1. Its components are filesystem roots with engineVersion: "unknown", consistency: "uncoordinated-files", and verification sha256-content-only. These fields do not assert Postgres compatibility, cross-database consistency, or application health. Native database discovery/version checks follow in H3. Credential references are currently an empty array; reusable credentials are never copied from captured file contents into the recovery catalog. Backup payloads can still contain secrets and need their existing encryption and access controls.

H3's Postgres adapter now adds version/digest receipts beside logical dumps. Catalogs containing those receipts also require postgres-logical-v1; the filesystem component's own consistency label remains unchanged.

Publish and verify

The fleet capture engine now requests format 2 by default and refuses an older binary. For a manually prepared, stable directory:

hostssh pack capabilities
hostssh pack write --format 2 --out /snapshot/stage/integrity.json \
  --root /snapshot/stage=var/backups/fleet-clone/example
hostssh pack verify --catalog /restore/var/backups/fleet-clone/example/integrity.json \
  --base /restore

The verification example assumes the stage has been copied into that restore-relative layout. Use an isolated extraction directory owned by the operator, with no concurrent writers.

Publication removes the previous completion marker, hashes and syncs payload files and directories, atomically writes and syncs the catalog, then checks the file inventory/hashes again. Only then does it atomically publish integrity.complete.json. This marker binds the artifact ID to the catalog's exact length and SHA-256. A failed/crashed attempt can leave a catalog without a marker; verification rejects it. Retry publication from stable content.

Restore verifies the marker and catalog before database, volume or configuration apply. It rejects unknown formats/fields/adapters, duplicate JSON keys/paths, overlapping roots, traversal, invalid sizes, missing components, special files, symlinks and mismatched content. Extra files anywhere in the extraction directory are rejected, including outside declared roots. The catalog has a 16 MiB limit; larger inventories need a future format change.

Fleet restore extracts each attempt into a fresh /var/restore/attempt.* directory and requires exactly one stage. That directory is retained for inspection; remove it only after the attempt is resolved and no restore is using it.

Fleet capture holds /var/backups/fleet-clone/.capture-lock-<box> through packing/upload. Normal exit removes the lock. After a killed capture, confirm no process still uses that workspace and inspect the prior attempt before manually removing its lock directory.

Compatibility and limits

• To deliberately produce an old pack, use HOSTSSH_PACK_FORMAT=1 with fleet capture. The direct hostssh pack write command also retains format 1 as its compatibility default; pass --format 2 for publication evidence.
• To restore an old pack, set HOSTSSH_ALLOW_LEGACY_PACK=1, or pass --allow-legacy to hostssh pack verify. A catalog-free image is labeled unverified; schema 1 verifies hashes without publication proof. A catalog still requires the upgraded verifier.
• Legacy opt-in cannot bypass a missing or invalid format-2 marker. Verification errors never trigger a fallback to TSV, Python, or an older binary.
• Format 2 fails closed on symlinks and special files. It does not silently omit them. Check this constraint against the selected filesystem roots before rollout.
• A checksum does not authenticate the artifact's author. Signed packs remain separate work. Keep receiving directories private and use trusted encrypted transport/storage.
• Publication does not freeze live volumes. Changes after publication can cause restore to reject the captured content; use snapshots/quiescence for stable capture. H4 covers stronger consistency coordination. Local process/fsync fault tests are not hardware power-loss proof.
• Restic continues to own deduplication and snapshot dependencies. No manifest base chain, new storage service, database migration or dependency is introduced.