Fleet Copilot
An AI SRE that diagnoses your fleet from live state and proposes one-click fixes you confirm — never auto-executing.
Fleet Copilot
Fleet Copilot is an AI diagnostician for your fleet. Ask it a question — or hit Triage the fleet — and it reasons over a live snapshot of your actual state (servers, deployments, recovery jobs, alerts, and snapshots), tells you what's wrong and why, and where a concrete fix exists, proposes it as a card you confirm.
Opening the panel needs the observability.view privilege (read). Running any
proposed action additionally needs platform.deploy — so a read-only operator can see
a proposal but can never execute it.
What makes its answers trustworthy
The Copilot is grounded in real state, not documentation or training data. Before each question it gathers a fresh, token-bounded snapshot of your fleet and hands that to the model as context. Answers trace back to what's actually happening right now — the model is instructed to diagnose strictly from the snapshot and to say so when the snapshot doesn't contain the answer.
The snapshot it reasons over includes:
| Section | What's in it |
|---|---|
| Fleet summary | Counts: servers, online/degraded/offline, backups healthy, open alerts, running jobs, images. |
| Servers | Each Node: status, provider/region, CPU/mem/disk, backup health, agent version. |
| Deployments | Each app: state, health, primary domain, and which Node it's on. |
| Recent recovery jobs | Kind, status, source→target, progress. |
| Open alerts | Severity, title, and source of each unacknowledged alert. |
| Loadable snapshots | Real image ids (with size, integrity, contents) it can propose a restore against. |
Lists are capped so the context stays bounded on a large fleet, and the whole gather is read-only.
Asking and triaging
- Ask — type a question (e.g. "Which servers are unhealthy, and why?", "Did any recent deploys or recovery jobs fail?", "What should I act on first?"). Send with the button or ⌘/Ctrl+Enter.
- Triage the fleet — one click runs a structured prompt that lists the top issues in priority order and proposes a fix for each where one exists.
- Follow-ups — the conversation threads, so you can drill in ("why is that one degraded?") and it keeps the prior turns in context.
Proposals — you confirm, it never executes
This is the core safety model: the Copilot never runs anything itself. When it identifies a fix, it emits a proposal card with a title, a plain-English summary, and a risk badge. You review it and click Confirm & run — that's the only path to execution, and it re-validates the action server-side and runs it through the same permission-gated job that the rest of the dashboard uses.
The six actions it can propose:
| Proposal | What it does | Risk |
|---|---|---|
| Redeploy app | Re-run an app's build→run→route on its Node (recover a failed/degraded deploy, or pick up a new commit). | medium |
| Cancel deploy | Stop an in-flight deploy that's stuck or wrong (only while it's still queued/building). | medium |
| Harden firewall | Apply default-deny inbound on a Node (SSH always preserved). | high |
| DNS cutover | Point a deployment's domain at the Node it runs on (an A-record cutover). | high |
| Provision database | Stand up a managed database (Postgres + pgvector by default) on a Node. | low |
| Restore snapshot | Load a registered snapshot onto a Node — a non-destructive clone. | high |
A few things that keep proposals honest:
- It only proposes against a server, app, or snapshot it can see in the snapshot, referenced by exact name (or exact image id for a restore).
- The title and risk you see are re-derived server-side from validated arguments — a tampered or stale card is rejected with "this action is no longer valid — ask the Copilot again."
- Names are resolved fail-closed: if a name is missing or matches more than one Node, the action refuses rather than guessing.
- Every confirmed action is audited.
Scope and limits
Be clear-eyed about what this is and isn't:
- It diagnoses; it does not act on its own. No proposal ever runs without your click.
- It sees a snapshot, not everything. It reasons over the capped fleet snapshot above — not raw logs, live shells, or provider dashboards. If the answer isn't in the snapshot, it will say so rather than guess.
- Restore is deliberately narrow. The Copilot proposes a restore only against a real
img_…id from the "Loadable snapshots" list; free-form recovery lives in the Backups & recovery UI. - AI is optional. If no AI key is configured on the control plane, the Copilot is honest about it: it returns the live fleet snapshot your question would have been grounded in, so the page is still useful for at-a-glance state.
- Confirming requires
platform.deploy. Read-only operators can ask and see proposals but can't run them.
Related
- Observability — the alerts and posture the Copilot triages.
- Fleet & Nodes — the Node detail behind its diagnoses.
- Backups & recovery — the human-driven restore/relocate UI.
- Hardening — the firewall model behind the harden proposal.