Fleet Copilot

An AI SRE that diagnoses your fleet from live state and proposes one-click fixes you confirm — never auto-executing.

Fleet Copilot

Fleet Copilot is an AI diagnostician for your fleet. Ask it a question — or hit Triage the fleet — and it reasons over a live snapshot of your actual state (servers, deployments, recovery jobs, alerts, and snapshots), tells you what's wrong and why, and where a concrete fix exists, proposes it as a card you confirm.

Opening the panel needs the observability.view privilege (read). Running any proposed action additionally needs platform.deploy — so a read-only operator can see a proposal but can never execute it.

What makes its answers trustworthy

The Copilot is grounded in real state, not documentation or training data. Before each question it gathers a fresh, token-bounded snapshot of your fleet and hands that to the model as context. Answers trace back to what's actually happening right now — the model is instructed to diagnose strictly from the snapshot and to say so when the snapshot doesn't contain the answer.

The snapshot it reasons over includes:

SectionWhat's in it
Fleet summaryCounts: servers, online/degraded/offline, backups healthy, open alerts, running jobs, images.
ServersEach Node: status, provider/region, CPU/mem/disk, backup health, agent version.
DeploymentsEach app: state, health, primary domain, and which Node it's on.
Recent recovery jobsKind, status, source→target, progress.
Open alertsSeverity, title, and source of each unacknowledged alert.
Loadable snapshotsReal image ids (with size, integrity, contents) it can propose a restore against.

Lists are capped so the context stays bounded on a large fleet, and the whole gather is read-only.

Asking and triaging

  • Ask — type a question (e.g. "Which servers are unhealthy, and why?", "Did any recent deploys or recovery jobs fail?", "What should I act on first?"). Send with the button or ⌘/Ctrl+Enter.
  • Triage the fleet — one click runs a structured prompt that lists the top issues in priority order and proposes a fix for each where one exists.
  • Follow-ups — the conversation threads, so you can drill in ("why is that one degraded?") and it keeps the prior turns in context.

Proposals — you confirm, it never executes

This is the core safety model: the Copilot never runs anything itself. When it identifies a fix, it emits a proposal card with a title, a plain-English summary, and a risk badge. You review it and click Confirm & run — that's the only path to execution, and it re-validates the action server-side and runs it through the same permission-gated job that the rest of the dashboard uses.

The six actions it can propose:

ProposalWhat it doesRisk
Redeploy appRe-run an app's build→run→route on its Node (recover a failed/degraded deploy, or pick up a new commit).medium
Cancel deployStop an in-flight deploy that's stuck or wrong (only while it's still queued/building).medium
Harden firewallApply default-deny inbound on a Node (SSH always preserved).high
DNS cutoverPoint a deployment's domain at the Node it runs on (an A-record cutover).high
Provision databaseStand up a managed database (Postgres + pgvector by default) on a Node.low
Restore snapshotLoad a registered snapshot onto a Node — a non-destructive clone.high

A few things that keep proposals honest:

  • It only proposes against a server, app, or snapshot it can see in the snapshot, referenced by exact name (or exact image id for a restore).
  • The title and risk you see are re-derived server-side from validated arguments — a tampered or stale card is rejected with "this action is no longer valid — ask the Copilot again."
  • Names are resolved fail-closed: if a name is missing or matches more than one Node, the action refuses rather than guessing.
  • Every confirmed action is audited.

Scope and limits

Be clear-eyed about what this is and isn't:

  • It diagnoses; it does not act on its own. No proposal ever runs without your click.
  • It sees a snapshot, not everything. It reasons over the capped fleet snapshot above — not raw logs, live shells, or provider dashboards. If the answer isn't in the snapshot, it will say so rather than guess.
  • Restore is deliberately narrow. The Copilot proposes a restore only against a real img_… id from the "Loadable snapshots" list; free-form recovery lives in the Backups & recovery UI.
  • AI is optional. If no AI key is configured on the control plane, the Copilot is honest about it: it returns the live fleet snapshot your question would have been grounded in, so the page is still useful for at-a-glance state.
  • Confirming requires platform.deploy. Read-only operators can ask and see proposals but can't run them.