kyleandClaude Opus 5.5
d4e821799c
Admins (Microsoft 365 sign-in names in BACKUP_DASH_ADMINS) get an Admin link to /admin.html, where they can create, list and revoke the API keys collectors use, instead of running manage.py on the server. Backend - /api/v1/admin/keys (list, create, revoke), admins only (403 otherwise); the user comes from the headers nginx sets after sign-in - Changes need an X-Requested-By header, which another site's page can't add, so a signed-in admin's browser can't be used to forge them - A new key is returned once; only its hash is stored. Names are unique for good, so revoked keys' history stays unambiguous - Keys record who created/revoked them and the company they last reported for (schema v2; databases migrate automatically on start) - /api/v1/me reports is_admin; manage.py shares the key code and records itself as creator/revoker Frontend - admin.html / admin.js: name validation shown after interaction (:user-invalid, aria-invalid kept in sync), the one-time key with a copy button (falls back to selecting it if clipboard access is refused), a key table flagging keys quiet for over 24 hours, and revoking confirmed in a <dialog>; results announced in a status region - Shared el() and signed-in helpers moved to dom.js / signed-in.js - Mock mode (?mock=1) includes sample keys Tested: 15 new backend tests (access, forged requests, names, duplicates, revoking, last-used company, v1 -> v2 upgrade); in the browser, every admin flow in mock mode and at phone width; end to end through the nginx config with the real API (a key created on the API works for ingest; revoking it makes ingest 401; non-admins see an explanation and no Admin link). Co-Authored-By: Claude Opus 5.5 <[email protected]>
176 lines
9.3 KiB
Markdown
176 lines
9.3 KiB
Markdown
# Backup-Dashboard
|
||
|
||
Proof of concept backup dashboard, intended to display backup jobs from multiple sources.
|
||
|
||
```
|
||
collectors ──POST /api/v1/ingest (API key)──▶ ┌──────────────┐
|
||
│ FastAPI + │
|
||
browser ──nginx (TLS)──▶ /api/* ─────────────▶│ SQLite │
|
||
│ └─ static files: public/ └──────────────┘
|
||
└─ auth_request ─▶ oauth2-proxy ─▶ Microsoft Entra ID (Microsoft 365 sign-in)
|
||
```
|
||
|
||
- `public/`: static frontend (no build step). Add `?mock=1` to the URL to use built-in sample data.
|
||
- `backend/`: FastAPI API backed by a single SQLite file.
|
||
- `deploy/`: nginx site and systemd unit.
|
||
- `collectors/`: PowerShell scripts that report results from backup products ([Veeam B&R](collectors/veeam/README.md), [Azure Backup / MARS](collectors/mars/README.md), [Windows Server Backup](collectors/wsb/README.md), [Veeam Backup for M365](collectors/vb365/README.md)). Code they share (config, logging, posting, state, installing) is in `collectors/common/BackupDashboard.Collector.psm1`.
|
||
|
||
## Data model
|
||
|
||
- **Tenant** (company) → **Job** → **Run**.
|
||
- A job is keyed by `(tenant, source_system, job.external_id)`, so renaming a job at the source updates it instead of creating a duplicate.
|
||
- A run has `status` (`running` | `success` | `warning` | `failed`), `started_at` and `finished_at`.
|
||
The dashboard's **Last Run** is the most recent `finished_at`. While a job is running it shows
|
||
**Running**, and Last Run still shows when the previous run finished.
|
||
- Runs older than `BACKUP_DASH_RETENTION_DAYS` (default 90, minimum 30) are purged every 6 hours.
|
||
|
||
## Reporting results (collectors)
|
||
|
||
Each collector (a script on a Veeam server, a webhook relay, etc.) gets its own API key:
|
||
|
||
```bash
|
||
cd backend && python manage.py create-key veeam-acme
|
||
```
|
||
|
||
Then it POSTs one report per run. Send `running` when a run starts (optional) and the final status
|
||
when it ends. Reports with the same `started_at` (or `run.external_id`) update the same run, and
|
||
retries are safe.
|
||
|
||
```bash
|
||
curl -X POST https://backups.example.com/api/v1/ingest \
|
||
-H "Authorization: Bearer bdk_..." -H "Content-Type: application/json" \
|
||
-d '{
|
||
"tenant": "Acme Corp",
|
||
"source_system": "Veeam B&R",
|
||
"job": { "external_id": "4f1c…", "name": "File Server – Nightly", "next_run_at": "2026-10-05T22:00:00-04:00" },
|
||
"run": {
|
||
"status": "success",
|
||
"started_at": "2026-10-04T22:00:05-04:00",
|
||
"finished_at": "2026-10-04T22:47:12-04:00",
|
||
"message": "Job completed successfully.",
|
||
"size_bytes": 128849018880,
|
||
"details": { "repository": "REPO-01" }
|
||
}
|
||
}'
|
||
```
|
||
|
||
- All timestamps must include a timezone offset (`Get-Date -Format o` in PowerShell does this).
|
||
- `job.next_run_at` controls the **Overdue** flag. If you omit it, the stored value is kept. If you send `null`, it's cleared.
|
||
- `run` can be omitted to only register or update a job.
|
||
- Unknown fields are rejected (HTTP 422), so typos in collector scripts fail loudly.
|
||
|
||
## API (read)
|
||
|
||
| Endpoint | Returns |
|
||
|---|---|
|
||
| `GET /api/v1/jobs` | All jobs with current status, `running_since`, `last_run_at`, `next_run_at` |
|
||
| `GET /api/v1/jobs/{id}` | One job, plus the last finished run's duration, size, message and details |
|
||
| `GET /api/v1/jobs/{id}/runs?days=30` | Run history (no messages), newest first |
|
||
| `GET /api/v1/runs/{id}` | One run including message and details |
|
||
| `GET /api/v1/health` | `{"status": "ok"}` |
|
||
|
||
The OpenAPI schema is at `/api/v1/openapi.json`.
|
||
|
||
## Authentication
|
||
|
||
- **Dashboard users:** Microsoft 365 sign-in. nginx checks every request with
|
||
[oauth2-proxy](https://oauth2-proxy.github.io/oauth2-proxy/), which signs users in with Microsoft
|
||
Entra ID. Who may sign in is controlled in Entra (users or a group assigned to the app). Every
|
||
signed-in user can see every tenant. Sessions last 12 hours; signing in again is usually silent.
|
||
The dashboard shows who's signed in; **Sign out** also signs out of Microsoft in that browser,
|
||
otherwise the next visit would sign straight back in.
|
||
- **Admins:** the sign-in names in `BACKUP_DASH_ADMINS` (in `/etc/backup-dashboard/backup-dashboard.env`,
|
||
comma-separated) get an **Admin** link to the API keys page. Restart the API after changing it.
|
||
- **Collectors:** per-collector bearer keys, checked by the app. Only a SHA-256 hash is stored.
|
||
`/api/v1/ingest` is the only path that doesn't need Microsoft sign-in.
|
||
|
||
### Managing API keys
|
||
|
||
On the **Admin** page (`/admin.html`), admins can:
|
||
- **Create** a key. It's shown once, with a copy button. Paste it when a collector's installer
|
||
asks for the API key.
|
||
- **See** each key's status, when it was last used and for which company, and who created it.
|
||
Active keys quiet for over 24 hours are flagged, since collectors report hourly.
|
||
- **Revoke** a key. The collectors using it are refused from their next run. This can't be undone,
|
||
and names stay taken, so the history stays unambiguous.
|
||
|
||
The same is possible from the server with `manage.py` (`create-key`, `list-keys`, `revoke-key`),
|
||
recorded as created/revoked by `manage.py (<user>)`.
|
||
|
||
### Microsoft 365 sign-in setup
|
||
|
||
**In the Microsoft Entra admin center** (entra.microsoft.com), as an admin of your tenant:
|
||
|
||
1. *App registrations → New registration*. Name: `Backup Dashboard`. Supported account types:
|
||
**Accounts in this organizational directory only** (single tenant). Redirect URI: platform
|
||
**Web**, `https://<your dashboard host>/oauth2/callback`. Register.
|
||
2. On its *Overview*, note the **Application (client) ID** and **Directory (tenant) ID**.
|
||
3. *Certificates & secrets → New client secret*. Copy the secret's **Value** now (it's only shown
|
||
once). Note its expiry date: sign-in stops working when it expires, so set a reminder to
|
||
create a new one before then.
|
||
4. *Enterprise applications → Backup Dashboard → Properties*: set **Assignment required** to
|
||
**Yes**. Then *Users and groups → Add user/group* and add the people (or a group) who should
|
||
see the dashboard. Everyone else in the tenant is refused by Microsoft before reaching it.
|
||
|
||
No API permissions beyond the default `User.Read` are needed.
|
||
|
||
**On the dashboard server:**
|
||
|
||
1. Install oauth2-proxy 7.8 or newer from its
|
||
[releases](https://github.com/oauth2-proxy/oauth2-proxy/releases) as `/usr/local/bin/oauth2-proxy`,
|
||
and create its user: `sudo useradd --system --no-create-home oauth2-proxy`.
|
||
2. Install the config and the client secret, readable only by root and oauth2-proxy:
|
||
```bash
|
||
sudo install -d -m 750 -g oauth2-proxy /etc/oauth2-proxy
|
||
sudo install -m 640 -g oauth2-proxy deploy/oauth2-proxy.cfg /etc/oauth2-proxy/backup-dashboard.cfg
|
||
sudo sh -c 'umask 027; printf %s "<client secret value>" > /etc/oauth2-proxy/client-secret' && sudo chgrp oauth2-proxy /etc/oauth2-proxy/client-secret
|
||
```
|
||
In `/etc/oauth2-proxy/backup-dashboard.cfg`, fill in `<TENANT_ID>`, `<APPLICATION_CLIENT_ID>`,
|
||
`<DASHBOARD_HOST>`, and `<COOKIE_SECRET>`, generated with:
|
||
`openssl rand -base64 32 | tr -- '+/' '-_'`
|
||
3. Install `deploy/oauth2-proxy.service` into `/etc/systemd/system/`, then
|
||
`sudo systemctl daemon-reload && sudo systemctl enable --now oauth2-proxy`.
|
||
Check it with `journalctl -u oauth2-proxy -n 20`: it should say "OAuthProxy configured for
|
||
Microsoft Entra ID".
|
||
4. Install the new `deploy/nginx.conf` (keeping your server name and certificate paths), then
|
||
`sudo nginx -t && sudo systemctl reload nginx`. Basic auth and its htpasswd file are no longer
|
||
used.
|
||
|
||
**Troubleshooting:** sign-in problems show in `journalctl -u oauth2-proxy`. Microsoft's own
|
||
errors (wrong redirect URI, user not assigned, expired secret) appear on its sign-in page with an
|
||
`AADSTS…` code.
|
||
|
||
## Development
|
||
|
||
```bash
|
||
cd backend
|
||
python -m venv .venv && . .venv/bin/activate
|
||
pip install -r requirements.txt -r requirements-dev.txt
|
||
pytest
|
||
python manage.py --create seed-demo # 30 days of fake history in ./backup-dashboard.db
|
||
BACKUP_DASH_SERVE_STATIC=1 uvicorn --factory app.main:create_app --reload
|
||
# → http://localhost:8000
|
||
```
|
||
|
||
Collector tests (PowerShell 7 + Pester; see [collectors/tests](collectors/tests/README.md)):
|
||
|
||
```powershell
|
||
./collectors/tests/Invoke-CollectorTests.ps1
|
||
```
|
||
|
||
## Deployment (Linux, nginx)
|
||
|
||
1. Copy the repo to `/opt/backup-dashboard`, create a venv at `/opt/backup-dashboard/venv`, and run `pip install -r backend/requirements.txt`.
|
||
2. Copy `deploy/backup-dashboard.env` to `/etc/backup-dashboard/backup-dashboard.env`. Both the service and `manage.py` read their settings from it.
|
||
3. Create a `backupdash` system user, then install `deploy/backup-dashboard-api.service` into `/etc/systemd/system/` and enable it. The database is created on first start.
|
||
4. Copy `public/` to `/var/www/backup-dashboard/public`, install `deploy/nginx.conf`, get a certificate with certbot, and set up Microsoft 365 sign-in (see [Microsoft 365 sign-in setup](#microsoft-365-sign-in-setup)).
|
||
5. Run `manage.py` as the service user. It finds the database through the env file:
|
||
```bash
|
||
cd /opt/backup-dashboard/backend
|
||
sudo -u backupdash /opt/backup-dashboard/venv/bin/python manage.py create-key veeam-acme
|
||
```
|
||
It prints the database path it's using and won't create a new database unless you pass `--create`.
|
||
|
||
Back up the SQLite file with `sqlite3 backup-dashboard.db ".backup copy.db"`. Don't use a plain `cp`
|
||
while the service is running.
|