Backup-Dashboard/README.md

Backup-Dashboard

Proof of concept backup dashboard, intended to display backup jobs from multiple sources.

collectors ──POST /api/v1/ingest (API key)──▶ ┌──────────────┐
                                              │ FastAPI +    │
browser ──nginx (TLS)──▶ /api/* ─────────────▶│ SQLite       │
         │  └─ static files: public/          └──────────────┘
         └─ auth_request ─▶ oauth2-proxy ─▶ Microsoft Entra ID (Microsoft 365 sign-in)

• public/: static frontend (no build step). Add ?mock=1 to the URL to use built-in sample data.
• backend/: FastAPI API backed by a single SQLite file.
• deploy/: nginx site and systemd unit.
• collectors/: PowerShell scripts that report results from backup products (Veeam B&R, Azure Backup / MARS, Windows Server Backup, Veeam Backup for M365). Code they share (config, logging, posting, state, installing) is in collectors/common/BackupDashboard.Collector.psm1.

Data model

• Tenant (company) → Job → Run.
• A job is keyed by (tenant, source_system, job.external_id), so renaming a job at the source updates it instead of creating a duplicate.
• A run has status (running | success | warning | failed), started_at and finished_at. The dashboard's Last Run is the most recent finished_at. While a job is running it shows Running, and Last Run still shows when the previous run finished.
• Runs older than BACKUP_DASH_RETENTION_DAYS (default 90, minimum 30) are purged every 6 hours.

Reporting results (collectors)

Each collector (a script on a Veeam server, a webhook relay, etc.) gets its own API key:

cd backend && python manage.py create-key veeam-acme

Then it POSTs one report per run. Send running when a run starts (optional) and the final status when it ends. Reports with the same started_at (or run.external_id) update the same run, and retries are safe.

curl -X POST https://backups.example.com/api/v1/ingest \
  -H "Authorization: Bearer bdk_..." -H "Content-Type: application/json" \
  -d '{
    "tenant": "Acme Corp",
    "source_system": "Veeam B&R",
    "job": { "external_id": "4f1c…", "name": "File Server – Nightly", "next_run_at": "2026-10-05T22:00:00-04:00" },
    "run": {
      "status": "success",
      "started_at": "2026-10-04T22:00:05-04:00",
      "finished_at": "2026-10-04T22:47:12-04:00",
      "message": "Job completed successfully.",
      "size_bytes": 128849018880,
      "details": { "repository": "REPO-01" }
    }
  }'

• All timestamps must include a timezone offset (Get-Date -Format o in PowerShell does this).
• job.next_run_at controls the Overdue flag. If you omit it, the stored value is kept. If you send null, it's cleared.
• run can be omitted to only register or update a job.
• Unknown fields are rejected (HTTP 422), so typos in collector scripts fail loudly.

API (read)

Endpoint	Returns
GET /api/v1/jobs	All jobs with current status, running_since, last_run_at, next_run_at
GET /api/v1/jobs/{id}	One job, plus the last finished run's duration, size, message and details
GET /api/v1/jobs/{id}/runs?days=30	Run history (no messages), newest first
GET /api/v1/runs/{id}	One run including message and details
GET /api/v1/tenants	Companies with jobs, for choosing a report
GET /api/v1/reports/tenant?tenant=…&days=30	One company's jobs and every run in the period (messages for warnings/failures only)
GET /api/v1/reports/tenant.csv?tenant=…&days=30&tz=America/Edmonton	The same as CSV, with full messages and times in tz
GET /api/v1/health	{"status": "ok"}

The OpenAPI schema is at /api/v1/openapi.json.

Reports

Reports in the dashboard's top bar (or a company's name in a job's detail) opens a backup report for one company over the last 7, 30 or 90 days. The address carries the choice, e.g. /report.html?tenant=Alpha%20Construction&days=30, so a report can be bookmarked or linked.

• On the page: a summary (jobs, runs, success rate, warnings, failures, overdue now); per job, its run counts, success rate, last result and last successful backup; a day-by-day strip; and every warning and failure with its message. Print / Save as PDF prints it light and without the page controls, whatever the screen's theme.
• CSV: every run in the period, with full messages and times in the browser's time zone. It opens directly in Excel. Values that a spreadsheet would treat as formulas are prefixed with '.
• Success rate is successful runs ÷ finished runs; runs still in progress don't count against it.
• Days before a job was monitored (before its earliest known run) show as Not monitored yet, not as missed backups.
• Periods are limited by retention (BACKUP_DASH_RETENTION_DAYS, 90 by default). Anyone who can sign in can view reports.

Authentication

• Dashboard users: Microsoft 365 sign-in. nginx checks every request with oauth2-proxy, which signs users in with Microsoft Entra ID. Who may sign in is controlled in Entra (users or a group assigned to the app). Every signed-in user can see every tenant. Sessions last 12 hours; signing in again is usually silent. The dashboard shows who's signed in; Sign out also signs out of Microsoft in that browser, otherwise the next visit would sign straight back in.
• Admins: the sign-in names in BACKUP_DASH_ADMINS (in /etc/backup-dashboard/backup-dashboard.env, comma-separated) get an Admin link to the API keys page. Restart the API after changing it.
• Collectors: per-collector bearer keys, checked by the app. Only a SHA-256 hash is stored. /api/v1/ingest is the only path that doesn't need Microsoft sign-in.

Managing API keys

On the Admin page (/admin.html), admins can:

• Create a key. It's shown once, with a copy button. Paste it when a collector's installer asks for the API key.
• See each key's status, when it was last used and for which company, and who created it. Active keys quiet for over 24 hours are flagged, since collectors report hourly.
• Revoke a key. The collectors using it are refused from their next run. This can't be undone, and names stay taken, so the history stays unambiguous.

The same is possible from the server with manage.py (create-key, list-keys, revoke-key), recorded as created/revoked by manage.py (<user>).

Microsoft 365 sign-in setup

In the Microsoft Entra admin center (entra.microsoft.com), as an admin of your tenant:

1. App registrations → New registration. Name: Backup Dashboard. Supported account types: Accounts in this organizational directory only (single tenant). Redirect URI: platform Web, https://<your dashboard host>/oauth2/callback. Register.
2. On its Overview, note the Application (client) ID and Directory (tenant) ID.
3. Certificates & secrets → New client secret. Copy the secret's Value now (it's only shown once). Note its expiry date: sign-in stops working when it expires, so set a reminder to create a new one before then.
4. Enterprise applications → Backup Dashboard → Properties: set Assignment required to Yes. Then Users and groups → Add user/group and add the people (or a group) who should see the dashboard. Everyone else in the tenant is refused by Microsoft before reaching it.

No API permissions beyond the default User.Read are needed.

On the dashboard server:

1. Install oauth2-proxy 7.8 or newer from its releases as /usr/local/bin/oauth2-proxy, and create its user: sudo useradd --system --no-create-home oauth2-proxy.
2. Install the config and the client secret, readable only by root and oauth2-proxy:

   sudo install -d -m 750 -g oauth2-proxy /etc/oauth2-proxy
   sudo install -m 640 -g oauth2-proxy deploy/oauth2-proxy.cfg /etc/oauth2-proxy/backup-dashboard.cfg
   sudo sh -c 'umask 027; printf %s "<client secret value>" > /etc/oauth2-proxy/client-secret' && sudo chgrp oauth2-proxy /etc/oauth2-proxy/client-secret
   

   In /etc/oauth2-proxy/backup-dashboard.cfg, fill in <TENANT_ID>, <APPLICATION_CLIENT_ID>, <DASHBOARD_HOST>, and <COOKIE_SECRET>, generated with: openssl rand -base64 32 | tr -- '+/' '-_'
3. Install deploy/oauth2-proxy.service into /etc/systemd/system/, then sudo systemctl daemon-reload && sudo systemctl enable --now oauth2-proxy. Check it with journalctl -u oauth2-proxy -n 20: it should say "OAuthProxy configured for Microsoft Entra ID".
4. Install the new deploy/nginx.conf (keeping your server name and certificate paths), then sudo nginx -t && sudo systemctl reload nginx. Basic auth and its htpasswd file are no longer used.

Troubleshooting: sign-in problems show in journalctl -u oauth2-proxy. Microsoft's own errors (wrong redirect URI, user not assigned, expired secret) appear on its sign-in page with an AADSTS… code.

Development

cd backend
python -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt -r requirements-dev.txt
pytest
python manage.py --create seed-demo            # 30 days of fake history in ./backup-dashboard.db
BACKUP_DASH_SERVE_STATIC=1 uvicorn --factory app.main:create_app --reload
# → http://localhost:8000

Collector tests (PowerShell 7 + Pester; see collectors/tests):

./collectors/tests/Invoke-CollectorTests.ps1

Deployment (Linux, nginx)

1. Copy the repo to /opt/backup-dashboard, create a venv at /opt/backup-dashboard/venv, and run pip install -r backend/requirements.txt.
2. Copy deploy/backup-dashboard.env to /etc/backup-dashboard/backup-dashboard.env. Both the service and manage.py read their settings from it.
3. Create a backupdash system user, then install deploy/backup-dashboard-api.service into /etc/systemd/system/ and enable it. The database is created on first start.
4. Copy public/ to /var/www/backup-dashboard/public, install deploy/nginx.conf, get a certificate with certbot, and set up Microsoft 365 sign-in (see Microsoft 365 sign-in setup).
5. Run manage.py as the service user. It finds the database through the env file:

   cd /opt/backup-dashboard/backend
   sudo -u backupdash /opt/backup-dashboard/venv/bin/python manage.py create-key veeam-acme
   

   It prints the database path it's using and won't create a new database unless you pass --create.

Back up the SQLite file with sqlite3 backup-dashboard.db ".backup copy.db". Don't use a plain cp while the service is running.

Updating the server

After changes are merged into main, on the dashboard server:

1. Back up the database. It takes seconds, and an update may upgrade the database's structure when the API restarts, which only this backup can reverse. Needs the sqlite3 package:

   sudo -u backupdash sqlite3 /var/lib/backup-dashboard/backup-dashboard.db ".backup /var/lib/backup-dashboard/before-update.db"
   

2. Pull, and see what changed (the list tells you whether steps 7 and 8 apply):

   sudo git -C /opt/backup-dashboard pull
   

   git -C /opt/backup-dashboard diff --stat ORIG_HEAD HEAD
   

3. Python packages. Does nothing unless backend/requirements.txt changed:

   sudo /opt/backup-dashboard/venv/bin/pip install -q -r /opt/backup-dashboard/backend/requirements.txt
   

4. Web files. nginx serves its own copy in /var/www, so a pull alone doesn't change what browsers get. If this step is skipped, new pages quietly show the dashboard instead (nginx falls back to index.html for files it doesn't have):

   sudo rsync -a /opt/backup-dashboard/public/ /var/www/backup-dashboard/public/
   

   Browsers pick up the new files on their next page load.

5. Restart the API, which loads the new code and settings and upgrades the database if needed:

   sudo systemctl restart backup-dashboard-api
   

6. Check it's healthy. The first should print {"status":"ok"}; the log shouldn't show errors:

   curl -s http://127.0.0.1:8000/api/v1/health
   

   journalctl -u backup-dashboard-api -n 20 --no-pager
   

7. If files in deploy/ changed: those are templates whose installed copies live elsewhere, so a pull doesn't update them. Compare each changed one with its installed copy (e.g. diff /opt/backup-dashboard/deploy/oauth2-proxy.service /etc/systemd/system/oauth2-proxy.service) and bring the changes across:

   Changed in the repo	Installed copy	After updating it
   deploy/backup-dashboard.env	/etc/backup-dashboard/backup-dashboard.env	Add any new settings, keeping your values; restart the API
   deploy/backup-dashboard-api.service	/etc/systemd/system/backup-dashboard-api.service	sudo systemctl daemon-reload, restart the API
   deploy/nginx.conf	Your nginx site config	Keep your server name and certificate paths; sudo nginx -t && sudo systemctl reload nginx
   deploy/oauth2-proxy.cfg	/etc/oauth2-proxy/backup-dashboard.cfg	Keep your IDs and secrets; sudo systemctl restart oauth2-proxy
   deploy/oauth2-proxy.service	/etc/systemd/system/oauth2-proxy.service	sudo systemctl daemon-reload && sudo systemctl restart oauth2-proxy

8. If files in collectors/ changed: the backup servers keep running their installed copy. On each affected server, copy over the whole collectors folder, re-run that collector's installer, and do a dry run (see its README). The changes are already covered by the collector tests (./collectors/tests/Invoke-CollectorTests.ps1) if you want to run them first.

If something goes wrong, go back to the version from before step 2 (ORIG_HEAD). The order matters: an older API won't start on a database a newer one has upgraded, so restore the database before starting it.

1. Stop the API:

   sudo systemctl stop backup-dashboard-api
   

2. Check whether the update upgraded the database: if these two numbers differ, it did.

   for db in backup-dashboard before-update; do sudo -u backupdash sqlite3 /var/lib/backup-dashboard/$db.db "PRAGMA user_version"; done
   

   If so, restore the backup from step 1 (.restore, not cp: it also deals with SQLite's -wal file). Runs reported since the backup drop out of the history, since collectors have already marked them as sent; backups themselves are unaffected.

   sudo -u backupdash sqlite3 /var/lib/backup-dashboard/backup-dashboard.db ".restore /var/lib/backup-dashboard/before-update.db"
   

3. Go back to the previous code and web files, then start the API:

   sudo git -C /opt/backup-dashboard checkout ORIG_HEAD
   

   sudo rsync -a /opt/backup-dashboard/public/ /var/www/backup-dashboard/public/ && sudo systemctl start backup-dashboard-api
   

Once the problem is fixed, return to main with sudo git -C /opt/backup-dashboard checkout main and update as above.